Eating our own tail: recipes for AI in the energy transition

On Friday, November 1, I attended a symposium on the topic of AI and the Environment, at Santa Clara University's Markkula Center for Applied Ethics . I welcomed the deeper dive into the quandary that AI poses, at the intersection of technology, decarbonization, and environmental justice.

I had already been thinking about AI's environmental footprint. Much of the work presented was research that we've been using at Salesforce to inform development and market strategies for AI usage in the energy and utilities industry. Spending a whole day reviewing the wider interdisciplinary research was a much-needed practice for me.

In this edition of Re:Energize I summarize some considerations from the symposium, and provide a list of Side Trail resources for us all to explore further as we build the future together.

First, some terminology

Let's start by understanding the terminology and some of the frameworks we can use to think about AI's environmental and energy impact.

Categories in which to consider the carbon footprint of AI:

• Operational - the development and use of training models
• Embodied - hardware, infrastructure, and materials management and disposal
• Downstream - where and how models are applied

Mechanisms by which AI impacts the environment:

• GHG emissions - datacenters, manufacturing
• Water - evaporative cooling, datacenter water consumption
• Deforestation - mineral and resource extraction for hardware
• Rebound effects - societal factors, how consumption and behavior changes

AI's environmental footprint has given us a jolt

I've written about this in Re:Energize before, but it bears repeating: the recent innovations in generative AI has disrupted almost every aspect of our energy transition plans.

Santa Clara University's Susan Kennedy made a convincing argument that the advent of ChatGPT markedly changed the environmental, social, and governance (ESG) game. Technology companies' net-zero ambitions and decarbonization goals did not account for the unprecedented demand in compute power and associated energy usage they would need for generative AI. Giants like Microsoft and Google have slipped in their ability to meet the net-zero pledges they made before GenAI became their business.

The prime directive of tech companies is profit, and the race to be market leaders requires some profligate practices. As University of Pennsylvania's Benjamin Lee pointed out, to come out on top, the big tech companies are giving AI away for free because they are still figuring out what use cases will emerge and how to optimize latency and throughput. As we start thinking about mitigating AI's impact, this is a firehose pointing right at us.

AI datacenters are competing with people, plants and diverse species for fresh water and clean energy sources, even as they degrade the ecosystems that would ensure the continued security and purity of those water sources. In regions like Virginia and Ohio, datacenter energy demand exceeds pre-ChatGPT resource planning, and Dominion Energy and Omaha Public Power District won't have the generation capacity to be able to ensure reliable power with clean energy sources alone.

This has created another disruption, as tech companies seek to meet their power demands with clean energy to maintain their decarbonization commitments while developing viable AI businesses. The conference came on the heels of recent news about Amazon, Google, and Microsoft investing in nuclear energy in their search to be emissions-free. Their other mechanisms are Power Purchase Agreements (PPAs) and buying carbon offsets. There are problems with both PPAs and carbon credits. PPAs don't always represent or reflect local energy impact. Carbon credits just shift responsibility without taking more carbon emissions away. Nevertheless, this is disrupting the energy market, as more companies get into the business of being energy suppliers because of AI.

We don't have a handle on how to measure and report on AI's environmental footprint

At this stage of AI development, it's very difficult to measure and report on the true impact of AI usage. You would expect academics would lament a lack of data, but that's not something to cynically dismiss. Researchers know that people don't measure what they don't care about. Much of the accounting data we should know in order to assess environmental degradation, human rights, and energy usage is in an invisible zone of externalized costs.

AI requires three phases:

1. development of the infrastructure and tools;
2. training the models; and
3. the inference phase - when all of us are running applications and playing around with what AI can do.

In the development phase, Environmental Impact Reports (EIRs) help us understand things like the draw on local water supply, or supply chain data about the materials used to build a datacenter. In the training phase, we might collect data on the energy intensity of manufactured chips and the energy required for compute load and operations. For the inference phase, we would also think about the energy required for all those queries and engagement with models. While the carbon intensity of developing a datacenter or building a chip might be high, the frequency of this use case is low. On the other hand, we have no idea how many billions of queries are going to drive up operational impact, nor how complex or energy-intensive any given query will be. The aggregated and cumulative impact of all those queries will probably exceed the impact of all the datacenters being built, but by how much? Carbon intensity is nuanced, and that's why I'm glad there are economists to help us substantiate claims.

Transparency would help

There are several examples of where mandated reporting is making it possible to research and validate AI's environmental impact. California Senate Bill 253 ("Climate Corporate Data Accountability Act"), the European Commission's Energy Efficiency Directive, and Germany's Energy Efficiency Act were cited.

Carnegie Mellon's Emma Strubell is part of a project trying to introduce an "AI Energy Score ," which would be to AI as EnergyStar?? is to appliances. In order to be able to do that, we have a long ways to go in defining the categories, outcomes, and measurements that would go into such a score. In fact Salesforce Ventures has invested in Hugging Face , a champion of the AI open source community, doing significant work in this area. But the datasets being used are only from Hugging Face's open-source community, and there is a lot of other important information we don't know.

What don't we know? We don't have a lot of the data from further down supply chains, and we are uncertain about the extent to which models will be used or the scale of consumer usage and queries conducted. Just as importantly, some of the most important information we don't know or measure is related to environmental justice (EJ). This is a result of our values.

SCU's Yuhong Liu works on models for a more interactive electricity grid. She says that training data is missing data from EJ communities and is not fully representative of grid and consumer behavior.?In a future where AI models get monetized and need to compete against each other for grid optimization applications, is there the danger of consumers using one model over another because it’s cheaper, but then getting a different level of accurate representation depending on how they trained the model? And in land use decisions, if we intend to make sure EJ communities are represented as powerfully as a well-resourced tech company building a datacenter, we need to choose to ask questions and collect input to expand our view.

But as SCU's Joseph Keller pointed out, even baseline reporting is political. He observed that sustainability conversations and attention to ESG peaked around 2021-22, then ebbed when CEOs got pushback from both the left and right. The left accused companies of greenwashing and not doing enough. The right accused companies of being "too woke," not putting profits first, and being disruptive to business.

We have to think about both water and energy

I've already written a lot in Re:Energize about the energy problem that AI poses but UC Riverside's Shaolei Ren has set my hair on fire about water. AI datacenters and chip manufacturers need to use "blue" water - fresh from a natural source. "Grey" water, while diluted and filtered, still has too many pollutants. And "Green" water, from soil and plants, is also not pure enough. Water is used for server level cooling, and facility level cooling, and the latter is most water intensive. Datacenters essentially compete with humans for access and use of water.

It helps me to think about scale and analogies. With regard to energy, writing 2 short emails using Llama-3-70B uses as much energy as fully charging an iPhone. Just thinking about all the ways I and other corporate drones are being asked to churn out more emails, more blog posts, and just generally more words strung together to make people buy more AI, inspires me give you a rough and completely emotional estimate that the equivalent of a thousand new iPhones drain power every day from my marketing colleagues alone.

What's the story on water? I took a bad photo here, but you can get the idea: depending on where you are, it takes 500 ml of water consumption - one medium water bottle - to support 10 to 34 LLM responses. If you're carrying a re-usable water bottle around so you aren't putting plastic bottles in landfill, you might also want to think about the water: how each image and email you generate takes another sip from water that we also need for drinking and hygiene.

As both Ren and SCU's Iris Stewart-Frey pointed out, water is undervalued and subsidized. Corporations don’t pay the full value of it or for environmental degradation. Companies are burning money to let us play with these tech apps for free, and the true cost is hidden. For example, we have not adequately tackled how to quantify the "social cost of carbon," used by the Environmental Protection Agency to estimate in dollars how much damage is caused over the long run by a ton of CO2 emissions in a given year. It is tied to our politics and how we value communities at the front lines of environmental harm (See "Trump vs. Obama on the Social Cost of Carbon–and Why It Matters ," an essay from the Center on Global Energy Policy at Columbia).

Avenues to solutions

Datacenters and embodied carbon - What should we consider about datacenter siting and management, if we want to make sure datacenters don't use so much energy, ruin local ecosystems, or take water away from people who need it?

Ren made a distinction between co-located datacenters (co-los) in urban areas, helping feed our appetite for digital life and apps, and hyperscaler datacenters like the ones Amazon and Google use. Co-los are multi-tenant situations, typically in metro areas, with limited cooling options and less control over using renewable generation and cooling system design. Hyperscalers are the ones you're reading about where a company can supply their own on-site renewable or clean energy generation, and efficiency measures.

Can we just pick cold places for datacenters? If you care about fish (and, well, the whole food chain that starts in the ocean), then you can't put a datacenter in the cold sea - the water is too erosive, and the thermal pollution would be devastating to ocean health. Someone asked about Microsoft's supposed "zero water" facility in Wisconsin . Ren pointed out that it was a great example of using clean, cold air - but that in equally cold northern China, you couldn't do it. The air would be too polluted and you would end up using a large amount of energy for filtering.

There are more siting considerations for getting to true decarbonization. Benjamin Lee pointed out that local renewable installations are constrained and not enough to match demand growth; this is true almost everywhere. And demand response is not a good option, given the uptime needs for datacenters.

In fact, as SCU's Tseming Yang presented, datacenters primarily use diesel for backup generation, because they can't cycle down operations. A legal loophole here poses another threat of going back to fossil energy: most datacenter regulation is at the state level. In California, all datacenters do have to go through California Environmental Quality Act (CEQA) to Inform the public and decision makers about the potential environmental impacts of proposed projects, but anything under 100MW is exempt from regulation, so most datacenter filings are for <100MW. They all go on diesel generation as backup generation (BUG) for their uptime requirements. The monitors and controls on these diesel generators shut off in emergencies. Nevertheless it’s been documented that in outages and disasters, there are extraordinarily disastrous emissions impacting local communities from these diesel generators.

Computing, operational carbon and embodied carbon - Can we optimize computing processes and chips to improve energy and water efficiency?

Benjamin Lee covered a lot of solutions that I've alluded to before, especially from my colleagues at Salesforce working on sustainable and ethical AI. In fact, the marks of Boris Gamazaychikov 's research was all over this symposium.

The main idea is this: You don't have to use everything, everywhere, all at once, all the time.

To sum up the main ways to reduce power consumption and improve AI energy efficiency, Lee cited

1. Better hardware design - for example, rethinking hyperscaler design; instead of amortizing costs by running at 100% capacity 24/7, trying to get more specific and precise about operations. The Verrus project for flexible data center energy management is an example of this.
2. Quantization, precision - Several presenters posed ideas for using only the necessary arithmetic precision, no more; using smaller language models and leaner training data; or, using a specific piece of the model for the task at hand, not throwing the whole model at the problem.
3. Specialized GPU platforms - chips that are purpose-built, instead of generalized for every possible use case.

Is pricing the most direct way to deal with this problem?

SCU's Helen Popper had a simple proposition: tax what's bad, not what's good. Everyone makes decisions based on context and values, so if you price things right, we can make better decisions. She suggested ways to agree on a carbon price that benefits everyone. If taxes go up in one place, it can go down somewhere else that matters. What if you could reduce everyone’s income tax by 20% if you had a $100 tax per tonne of CO2?

SCU's Hersh Shefrin addressed the hard truth about corporate social responsibility and the true costs to society. Microsoft has been increasing its energy contracts to support AI, but continuing to meet their net zero goals by paying others to cut emissions to offset Microsoft’s.?Most people dealing with Scope 1 and 2 use this carbon credits / offset strategy (and the company I work for has had to do this too). According to Shefrin, SCU bought carbon credits valued at $7 per credit, but by his calculations the number should be more like $200 to account for the social cost. In contrast to this shell game of credit-swapping, he invited us to push for a real carbon price and a carbon tax. In his words, the only thing preventing this is our will: "We are addicted to fossil fuels, and we are the kind of addicts who won’t go to rehab.”

This is where academic thought is being applied to integrated assessment models, where climate science and choice theory and behavioral economics intersect. Throughout the sessions I was thinking about the problem where we isolate and draw lines closer and closer to ourseles, instead of expanding our idea of who is at the table and extending our timeframe. For me, the discussion came together with the ideas of trade-offs, risks, and a "temporal discount" - how much to sacrifice today to benefit future generations.

Let's face it, profits are the priority for tech companies. How does this affect energy markets and pricing? Utilization is a big part of that accounting. Tech companies don’t want to miss out on opportunity in this emergent phase of AI industry growth or lag behind market just because they under-invested in energy. Someone asked Benjamin Lee why Microsoft and Google pay a third of what residents pay for kWh, and he explained that their rate is negotiated and set for peak usage; that’s why there’s no incentive to use less than 100% all the time. Environmental impact is an externality not reflected in accounting, nor have we calculated the energy cost of the new stress on grid capacity and our need for infrastructure. If you embed carbon costs into the running of a datacenter or the making of a GPU, your utilization of that carbon cost is a trade off.

Where do we go from here?

Joseph Keller reminded us that even though there’s this concept that AI is inevitable, we have agency. I thought about how we can manage risk, make policy, and control how AI evolves. Just because something is "inevitable" doesn’t mean we have to let it roll right over us.

Keller also emphasized that trust is essential for climate policy. Everyone should know exactly what’s at stake. This underscored the universal call from presenters for mandated, standardized, public reporting.

Helen Popper also wisely pointed out that AI is young, so we can be forgiving. We don't have to get everything right as we are working on this, and we definitely don't know everything we need to know to make the "best" decisions - but we can learn when things go awry.

And Hersh Shefrin ended with the observation that humans take more risks when behind the 8-ball. AI is a big risk, and climate change is our 8-ball.

For more details about the conference and its proceedings, visit https://www.scu.edu/ethics/events/ai-and-the-environment/agenda/

Side trails

Resources in addition to the links in the article above:

• Climate Change AI - If AI is causing more problems for climate change and we need to eat our own tail with it, then CCAI is our ouroboros, using machine learning to tackle the biggest data mining and pattern-recognition problems in our quest for whole-systems analyses.
• The Economics of Water: Valuing the Hydrological Cycle as a Global Common Good - I need pictures and explication in small chunks to help me with complex systems-thinking, and this resource helps me dig deeper into the concepts that Shaolei Ren and Iris Steward-Fry touched on.
• https://www.datacentermap.com/ - a database that has been mapping worldwide data center locations since 2007.
• The National Artificial Intelligence Research Resource (NAIRR)?Pilot - The NAIRR Pilot aims to connect U.S. researchers and educators to computational, data, and training resources needed to advance AI research and research that employs AI. Federal agencies are collaborating with government-supported and non-governmental partners to implement the Pilot as a preparatory step toward an eventual full NAIRR?implementation.
• NSF Expeditions in Computing: Carbon Connect—An Ecosystem for Sustainable Computing - A collaboration between Penn Engineering and Harvard to establish new standards for carbon accounting in the computing industry, with the aim of influencing future energy policy and legislation. Led by Benjamin Lee , who is already a treasure trove of information and wisdom regarding this complex topic.
• The EU’s Carbon Border Adjustment Mechanism (CBAM) - the EU's tool to put a fair price on the carbon emitted during the production of carbon intensive goods that are entering the EU, and to encourage cleaner industrial production in non-EU countries.?As Helen Popper put it, this strategy for putting a fair price on carbon might appeal to people who like to tax foreigners while also achieving an actual just and equitable outcome for us all.

My coverage of solutions or products are meant to expand awareness and should not be construed as endorsements. The views expressed in my newsletter and posts represent my own opinions and not necessarily those of my employer, Salesforce.