Why AI is so expensive: understanding the true costs behind the AI boom

Artificial intelligence has become a significant revenue driver for many tech companies, but it's also a massive investment — and often a financial strain. The increasing costs of developing AI, especially generative models, are not a surprise to those in the industry. As Microsoft, Google, and Meta ramp up their AI efforts, they've also been spending billions to build the necessary infrastructure, from chips to data centers.

But why exactly is AI so expensive? Let’s break it down.

At the heart of AI’s rising costs is the push for larger, more complex models. AI systems like OpenAI’s ChatGPT rely on large language models (LLMs), which are trained using vast amounts of data — from books to online articles. The larger and more sophisticated these models become, the more expensive they are to train.

The cost to train current AI models can reach $100 million, and upcoming models could soar to $1 billion or more. By 2025, experts estimate that the costs could rise even higher, possibly hitting $10 billion.

But why such staggering numbers? Training LLMs requires procuring more data and computing power. This process takes time, money, and infrastructure — and companies are betting that bigger models will lead to more sophisticated AI systems that outperform humans in a growing number of tasks.

The power of GPUs

A major factor driving AI costs is the need for specialized hardware. Unlike the central processing units (CPUs) found in most computers, AI models are trained using graphics processing units (GPUs), which can handle vast amounts of data quickly. But these chips are not only in short supply, they’re also exceedingly expensive.

While companies can avoid buying their own chips by renting from cloud providers like Amazon, this comes with a steep price tag as well. For instance, a group of Nvidia H100 chips (The gold standard for training AI models) can cost nearly $100 per hour to rent.

Data centers: the backbone of AI

Building and maintaining AI systems require more than just cutting-edge chips. The vast infrastructure needed to house these chips is also incredibly costly. Tech giants are racing to construct new data centers, which are custom-built to support the growing demand for AI services.

In 2024, companies are expected to spend nearly $294 billion on constructing data centers, up from $193 billion in 2020.

These facilities are getting bigger, too, with the average data center now spanning over 412,000 square feet — a nearly fivefold increase since 2010.

Cheaper alternatives?

While the AI boom has led to skyrocketing costs, some companies are exploring more cost-effective options. Microsoft has introduced smaller, less resource-intensive AI models that could serve certain customers. These smaller models may not be as powerful as the larger ones but could still deliver significant value without breaking the bank.

As AI continues to evolve, the costs of development will remain high. But for many companies, the potential benefits of advanced AI systems outweigh the financial burden. Whether through larger models or more efficient alternatives, the race to build the future of AI will be costly — but also full of opportunity.