"Millions, billions, trillions of parameters ... isn't it time to talk about frugality for AI too?"

ChatGPT ...There's been no let-up in the last few weeks!

We look at the news on any tech site, one article out of two talks about ChatGPT, we open the thread of posts on LinkedIn, one post out of three praises the incredible merits of ChatGPT, we turn on the radio, and even France Culture goes with its "ChatGPT: the new challenges of AI".

ChatGPT soon on the front page of Closer?

So, no, don't worry, this article won't talk about ChatGPT as such (that will be the subject of a future post), but rather about what's behind it, what makes you dizzy when you look into the subject, and what must automatically make us, the actors of the digital world and bearers of the revolutions associated with it, ask ourselves quite a few questions... I'm talking about the gargantuan appetite in terms of calculations and therefore in terms of the energy consumed by these models that allow for such feats.

Because yes ChatGPT consumes a lot of energy ... and it consumes a lot of energy at several stages of its life.

To begin with, like all AI models, its training is greedy. Very greedy, very very greedy even.

Indeed, ChatGPT is built on a model of the GPT3 family, the latest Large Language Model (LLM) from OpenAI. There have been a few studies on the subject of the consumption of this model, notably by the University of Copenhagen, which estimated that a GPT3 training session requires some 190,000 kWh ... that is the average consumption of 126 households for 1 year in Denmark. This is the average consumption of 126 households for one year in Denmark. We would expect nothing less to train the 175 billion parameters that make up the GPT3, with hundreds of billions of texts... (I will not knowingly talk about carbon footprint, because the passage from electricity consumption to CO2 depends on many parameters, particularly with regard to the energy source used, but you will have understood that the idea of carbon footprint obviously arises)

This unit figure is all the more important when you know that there was obviously not just one drive for this model to arrive at the results we know...

And it is even more important when you take a step back and realise that GPT3 is far from being the only one in its category ... Google, Facebook, NVidia, Microsoft, ... have similar models in the pipeline and we have already seen versions with updates potentially ready to jump or in any case certainly consuming a lot of energy because they are in the middle of a training phase.

Limiting ourselves to a 'Western' vision of the digital giants would not allow us to have an overall view and we don't have to look far before coming across Wu Dao, a Chinese project with a whopping 1.75 trillion parameters ... that's just 10 times more parameters than GPT3, in all simplicity.

But why this excess? Well, simply because one of the first basic adages of AI in terms of design is absolutely not "More with less", but rather unfortunately something more like "The more, the better" ...

For the language, everything really changed in 2018, with the release of BERT, which democratised the use of the "attention" mechanism, with an architecture that effectively suggested a new race to gigantism. It then got off to a slow start as it contained 'only' 110 million parameters... ridiculous!

The year 2019 will see no less than 4 model size records broken (Open AI: GPT2 / 1.5 billion, Nvidia: Megatron LM / 8.3 billion, Google: T5 / 11 billion and Microsoft: Turing- NLG /17.2 billion)

In 2020, OpenAI seems to put everyone in agreement with GPT3 and its 175 billion ...

Surprisingly enough, there is little mention of Google's GShard, which uses 600 billion parameters, and even less of Wu Dao, mentioned above, with 1,750 billion parameters developed by the Beijing Academy of Artificial Intelligence, or Baidu's RecSYS model, which is thought to contain 10,000 billion in its largest version (!!). So the race is not over...?

And the field of language analysis is not the only one to follow this credo, nor the only field to use AI: images, sounds, and of course tabular data are also part of the game with tens of thousands of models, perhaps not at the level of the gigantic LLMs, but ... there is strength in numbers in terms of power consumption!

The University of Denmark estimates that AI-related energy costs have increased by a factor of 300,000 between 2012 and 2018! Keeping in mind that this figure does not even take into account the impact of LLMs, since in 2018, we had barely entered the world of "ogre models" with the arrival of BERT.

So are we, in the end, just idle spectators in the face of this disproportion?

No, because we are also all actors, at least at the first level, i.e. by simply being the end user.

Because yes, AI consumes a lot of energy to be trained, but all these models also require infrastructures proportionally gigantic to their size to be used!

Coming back to ChatGPT, I don't have the answer to the question of the energy cost of an inference (i.e. the act of making a request to the model and getting a response), but the 45GB of VRAM required to run an instance of the model once trained, indicates that the latter will not be mechanically trivial.

Especially if we put it in perspective of the craze of use that it has had since its opening to the general public on November 30: 1 million "users" after one week of opening, many closures in January related to infrastructure upgrades to meet demand, an estimated 10 million requests per day, by, to date, 96 million estimated users per month, and a prediction of more than a billion users by the end of 2023 ... All this does not necessarily smell very good at the kWh level ...

I won't go into the usefulness of the queries made on ChatGPT today or even its usefulness as such, knowing that this question of usefulness can (must?) also be asked about the myriad of services based on AI models that are flourishing online ??, but it is important to remember that each user is therefore individually responsible for the uses he makes of these particularly energy-consuming services ...

And we, as digital specialist/consultant, IT solution integrator or IT researcher, we who daily advise on usage, look for new approaches or ways to improve existing models or implement AI projects, are the frontline actors on these energy issues. But ...

Who has already integrated eco-design steps for their AI projects?

Who has a chapter dedicated to assessing the carbon footprint of AI services in their consultancy offer?

Who is directing at least part of their research activities towards AI frugality issues??

And all this at the risk of presenting project or service costs that are significantly higher than those of the competition?

Because, in the end, it is a question of being concerned about the frugality of the AI projects that we implement, and this in a systematic way. In the second part of this article, I propose to review the main families of actions that allow us to move towards reducing energy consumption and make us a real player in this area of concern.

And to conclude this first part, I wanted to come back to Eliza, the first chatbot in history. A program that simulated a conversation with a Rogerian psychotherapist. There was no AI in this application and yet the numbers around the project were already staggering, just think: 2 years of programming started in 1964, hundreds of lines of code, a specific programming language and above all 128Kb of memory needed to run this titan (we're talking about KB, not MB and even less GB ...).

So yes, these figures make you smile today, but at the time, they were really staggering, to the point where the project was widely criticised and not taken seriously because of the disproportionate infrastructure needed to run it, which, it was thought, would mechanically kill the possibilities of its use.

It's all a question of the times... In which case, aren't we in the same situation as in 1964, considering today that hundreds or even now thousands of billions of parameters for an AI is totally disproportionate?

Two major differences though: Eliza never had more than a few thousand users and intrinsically it needed immensely fewer resources to run...

Oh yes, and I wanted to tell you, this article was absolutely not created using ChatGPT ??, but I'd be delighted if the latter came to scrape it up to feed on it, maybe it'll allow him to inject a bit more frugality issues into his future words.

jean Paul Muller , Global Practice Manager AI Inetum