How AI Datacenters Work

Good Morning,

Get the full inside scoop on key AI topics for less than $2 a week with a premium subscription to my flagship Newsletter.

• With the EU and the Middle East following in the footsteps of BigTech Capex, the great data center boom of the 2020s is an unprecedented moment in infrastructure rollouts.
• It’s as if the world is preparing for the coming age of AI. The capabilities of our models of today do not represent what AI can become tomorrow, as the decades ahead will show.

Eric Flaningam of the Newsletter Generative Value is becoming one of my favorite reads related to guides to the AI space.

Last time with him we went over the AI Semiconductor Landscape. Today we will be looking into AI Data Centers. The Current State of AI Markets is getting pretty crazy.

Subscribe to Generative Value

In 2025 we’ll see in the area of $400 Billion in capex from BigTech mostly related to AI Infrastructure and datacenters alone, it’s hard to grasp such big numbers. If you add OpenAI, xAI plus the Big Four (header image), the number certainly approaches $400 billion in actuality.

Luckily for us, Eric’s writing walks us through the enormity of the infrastructure understanding seamlessly:

Start Here

1. A Primer on Data Centers
2. Data Industry Primer
3. An Overview of the Semiconductor Industry

In just a few minutes you can grasp major concepts underlying how AI works.

There are few other publications where I feel as much synergy with my own Newsletter as Generative Value. ?? The rabbit hole we are about to enter is a tunnel into AI’s value creation itself.

Share

?? Generative Value ??

Generative Value Sharing learnings about tech & business.

By Eric Flaningam

Let’s get to today’s deep dive:

“As one financier observed at the time, the amount of money required by the burgeoning US electrical system was “bewildering” and “sounded more like astronomical mathematics than totals of round, hard-earned dollars.”” - [A description of the electric grid buildout around 1900] Power Loss, Richard Hirsh 1999, via Construction Physics

We’re currently in the midst of one of the largest computing infrastructure build outs in history.

100+ years ago, we saw a similar buildout of the electric grid (which ironically is a bottleneck for today’s buildout). Throughout the birth of the electric grid, we saw the scaling of power plants (building power plants as large as possible to capture performance improvements), “Astronomical” CapEx investments, and the plummeting cost of electricity.

Today, we’re seeing the scaling of data centers, “Astronomical” CapEx from the hyperscalers, and the plummeting cost of AI compute:

This will be an introductory piece breaking down the AI data center: what exactly it means, who supplies the components into the data center, and where opportunities may lie.

This piece will be specifically focused on the infrastructure required to build AI-specific data centers; for an introductory piece on data centers, I recommend reading this piece I published a few months back.

• For less than $2 a week, consider getting all-access to this Newsletter.

Subscribe to All

1. An Overview of the AI Data Center

The term “data center” doesn’t do justice to the sheer scale of these “AI Factories” (as Jensen affectionately refers to them). The largest data centers cost billions of dollars across land, electrical & cooling equipment, construction, GPUs, and other computing infrastructure.

That’s before the energy costs. The new largest hyperscaler data centers can consume up to 1 GW of energy capacity. For reference, New York City consumes about 5.5 GW. So, for every five of these mega data centers, we’re adding an equivalent NYC to the grid.

We can broadly break down the data center value chain into a few categories: the initial construction to develop a data center, the industrial equipment to support the data center, the computing infrastructure or “Kit” in a data center, and the energy required to power the data center. Additionally, we have companies that own or lease data centers to provide end services to consumers.

We can visualize the value chain here:

I won’t touch on every company exposed to the data center. There are financiers, real estate developers, construction firms, and a host of other companies contributing to this buildout. As Morgan Housel says, “I’m likely to agree with anyone who points out what I’ve missed.”

Before diving deeper, we should look at the history of data centers (which is especially relevant to the energy crunch we’re seeing today, specifically in the Northern Virginia region).

2. An Abridged History of Data Centers

Data centers, in large part, have followed the rise of computers and the internet. I’ll briefly discuss the history of these trends and how we got here.

Pre-Internet Computing

The earliest versions of computing look similar to data centers today: a centralized computer targeted at solving a compute-intensive and critical task.

We have two early examples of this:

1. The Colossus - The computer built by Alan Turing to break the Enigma machine. (Note: Turing is also considered the father of Artificial Intelligence and Computer Science. He proposed the Turing Test as a means of testing if AI is real, which ChatGPT passed last year).
2. The ENIAC - The computer designed in WW2 by the US Military but not completed until 1946. The Colossus was built before the ENIAC, but the ENIAC is often considered the first computer because of the confidential nature of the Colossus.

Both were housed in what could be considered the “first data centers.”

To read the entire piece go here.