The rising cost of LLM-based search

Large Language Models like ChatGPT are shifting the paradigm of information retrieval and search. These models go beyond merely providing a list of possible answers; they analyse, interpret, and contextualise queries, delivering nuanced and detailed responses. For instance, I regularly use Perplexity to help make sense of questions as diverse as tank survivability rates, what kind of lights I should get for my bookshelves, or understanding the regulatory burden across industrial sectors. LLMs have the potential to move beyond the search engine, towards synthesising information, a process I described three years ago in my essay “A Short History of Knowledge Technologies”.

However, this computational wizardry comes at a significant energetic cost. Each interaction with ChatGPT may consume up to 2.9 watt-hours (Wh) of energy, enough to boil two tablespoons of water and nearly ten times the 0.3 Wh energy cost of a standard Google search. And if Google replaced its current search algorithms with LLMs, SemiAnalysis estimates that each search request would cost up to 8.9 Wh of energy. Given Google’s 9 billion search requests per day this would lead to an annual energy use of 29.2 terawatt-hours (TWh) on their servers, equivalent to the entire annual energy consumption of Ireland.

Of course, several variables could offset this theoretical surge in energy use, including bottlenecks in server manufacturing, improvements in energy efficiency, and Google simply not wanting to pay 30 times the energy costs for a search request. Nevertheless, it illustrates how emerging technologies often come with heightened energetic demands.

As we look ahead, our digital world’s computational appetite shows no signs of abating - even without AI developments. Efficiency gains in data centres, which have historically helped mitigate increased energy consumption, are plateauing. MIT predicts that by 2030, data centres could devour up to 21% of the world’s total electricity supply, up from 1-1.5% currently. Other more conservative forecasts suggest energy demand will still increase by 67% by 2028. Either way, new technologies and the demand for computation will pressure the energy transition.

Yet this pressure can often stimulate the quest for greater efficiency. We’ve seen this dynamic play out in cryptocurrency. Initially, blockchain technologies like Ethereum relied on energy-intensive proof-of-work mechanisms. As energy concerns mounted, the industry pivoted towards more sustainable proof-of-stake models. We are already seeing promising avenues within AI, such as gains in the performance of smaller models. However, this cycle of new technologies raising energy consumption before efficiencies eventually kick in will likely continue into the future.

Of course, the long-term goal is to make these concerns irrelevant in an age of abundant, clean energy — we’re looking at you, nuclear fusion.?