Navigating AI's Energy Challenges: Insights from Washington, DC with Senator Mitt Romney

Last week, I had the privilege of traveling to Washington, DC, to meet with top policymakers, including Senator Mitt Romney (R-UT) to discuss AI, American Innovation and IP Policies.

Sitting down with Sen. Romney, we discussed the future of the technology sector and the transformative impact of AI. We highlighted the importance of investing in R&D, innovation, and education within the U.S.

Regardless of political affiliation, it's clear that prioritizing and protecting American inventors and businesses is crucial. Innovation, IP protection, and a competitive marketplace empower American investors and entrepreneurs, benefiting us all. Key points from our discussion:

AI Energy Problem:

• Massive Energy Consumption: AI computation requires significant electricity, straining the grid. AI servers use 30-100 kW, compared to 7 kW for traditional servers. A single ChatGPT query uses 2.9 watt-hours, nearly 10x more than a Google search.
• Energy Dependence: Training (20%): Involves running models on large datasets using many GPUs over months, consuming substantial energy for computation and cooling. Inference (80%): Applying trained models to solve problems increases power demand as usage grows.
• Growing Energy Use: AI’s computational power needs are doubling every 100 days, with energy demand growing 26%-36% annually. By 2028, AI could consume more power than Iceland did in 2021. Data centers currently use 1-2% of global power, projected to rise to 3-4% by decade's end, potentially doubling carbon dioxide emissions by 2030.

Mitigation Strategies:

• Chip Efficiency: Implementing power draw caps can reduce power consumption by 12%-15% with minimal compute time increase. Cooler running chips also lower cooling energy needs.
• Early Termination of Underperforming Models: Shutting down models that don't learn as fast during hyperparameter optimization can save energy.
• Quantum Computing: Offers a linear relationship between computational power and energy usage, potentially making AI models more compact and efficient. However, it's still in early stages with few practical applications.

Implications for US Policymakers:

• Increase Energy Capacity: Building more, ideally clean, energy capacity to meet growing demands. US utilities need to invest around $50 billion in new generation capacity for data centers.
• Invest in Nuclear Energy: Including nuclear fusion, which could be a game-changer for sustainable energy. Significant advancements like the December 2022 fusion breakthrough show promise, but current government investment is under $1 billion.

Leading Companies in AI Energy Efficiency:

• Etched: Develops specialized hardware for inference, such as the Sohu AI chip, which claims to be 20x faster and cheaper than Nvidia H100 GPUs. Recently raised $120M.
• Martian: Provides an LLM “router” that dynamically selects the best AI model for a given task, optimizing cost-to-performance and reducing inference time and energy use.

Investing in these areas and fostering innovation will ensure that the U.S. remains at the forefront of technological advancement while addressing the challenges posed by AI's energy demands.