Data Centers in Motion – Part 1: Cooling

Editorial Note: Nvidia’s Blackwell GPUs have pushed data centers to the brink, driving the need for advanced cooling solutions to manage a staggering 70% increase in power consumption over the H100s. While securing sufficient amounts of power presents its own set of challenges (to be explored in the next article), data center designers are quickly realizing that cost-effective cooling technologies are in short supply. The rapid increase in demand for training foundation LLMs and processing AI inference has only worsened the issue, with innovations in computing outpacing advancements in cooling. To bridge this gap, companies are exploring unconventional solutions—from launching data centers into space to submerging them underwater. The time has come to navigate the complex landscape of today’s Cooling Conundrum.

Friend: “I’m thinking of reselling H100s.”

Me: “Aren’t you focused on designing specialized LLMs?”

Friend: “Yes, but I want to learn about limitations in data center design and deployment.”

Me: “Hmm…what about non-compute related limitations? I’ve started focusing on opportunities in cooling and energy.”

Friend: “Maybe down the road, but can you share what you find? I heard YC just invested in an underwater data center company.”

Me: “Of course.”

In 7th grade, I thought about designing (and eventually built) my own desktop computer. Hundreds of hours of research went into choosing the CPU with the perfect combination of cores and clock speed, the amount of memory for my GPU, the speed of RAM, and the balance between SSD, hard drive, and M.2 storage. Then came decisions about the motherboard, power supply, chassis—and, most importantly, cooling.

Unlike the other components, cooling offered the most flexibility and took the longest to decide.

Do you use air cooling or liquid cooling?

Air cool your GPU but liquid cool your CPU? Or vise versa?

Use an all-in-one liquid cooling systems or make a custom solution?

In the end, I took a cost-effective route by opting for an all-in-one off the shelve cooling solution for my CPU. Even with my desktop computer now complete, I’ve continued to be fascinated by the transformative potential of cooling in computing.

Cooling is for more than just keeping hardware running—it can unlock performance beyond “what you paid for,” extend system longevity, and reduce operational costs. While these benefits aren’t as impactful in my small-scale desktop build, they become critical when applied to large scale data centers. Let’s quickly break down the implications of cooling solutions on data centers to set a baseline:

Each of these cooling impacts exist on a spectrum and the designer of a data center needs to carefully consider:

How many years is the data center designed to last?

Does a high cost of energy require a more energy efficient solution?

Is the data center for storage or computation?

These, along with many more questions, play a role in choosing the best cooling solution for a given data center, but for this article we are simply focusing on technological advances in cooling solutions (which will hopefully lead to optimizations across all five categories).

Timeline of Data Center Cooling Development

Breaking down the history of data center cooling:

The evolution of data center cooling has undergone significant changes since the 1990s, marked by key technological advancements that have transformed how facilities manage heat and energy consumption. In the 1990s,?air-based cooling systems, including CRAC (Computer Room Air Conditioning) and raised floors, were the industry standard, offering reliable solutions but becoming less efficient as data centers scaled up. The early 2000s saw a shift with the?re-emergence of liquid cooling, specifically?direct-to-chip cooling, which targeted heat directly at the source for better efficiency in high-performance environments.

By the 2010s, innovations such as?hot and cold aisle containment?and?free-air cooling?emerged, improving airflow and reducing energy use. A major breakthrough came in 2014 with the introduction of?immersion cooling, where servers are submerged in non-conductive liquids to enhance heat absorption, making it especially useful for high-density computing.

Since then, more sophisticated solutions have emerged, such as?AI-powered cooling?systems in 2017, which optimize cooling efficiency by making real-time adjustments based on system data. In 2020,?chip-level liquid cooling?further refined temperature management by applying liquid coolants to individual components, enhancing cooling precision for modern, high-power hardware. The most recent innovation,?two-phase liquid cooling, introduced in 2023, leverages phase-change technology for even greater performance by evaporating and condensing coolants to manage heat.        

Today, the push for greater efficiency and sustainability continues to drive the adoption of?liquid-based?and?hybrid cooling systems, combining air and liquid cooling to balance cost and efficiency. Meanwhile,?free cooling?techniques, such as economizers and evaporative cooling, have gained traction in favorable climates, and technologies like?geothermal cooling?and?waste heat recovery?represent the cutting edge of sustainable cooling methods.

Data center Cooling Technology

We've now touched on numerous types of cooling technology, but let’s concisely break down each technique and how they compare:

State of Data Center Cooling Solutions

To map the legacy and emerging landscape for data center cooling, I assembled a list of +45 companies focused on solving thermal dissipation in data centers. Here’s the Airtable link for reference: DATA CENTER COOLING SOLUTIONS

Air cooling?continues to be a widespread method in data centers, primarily through Computer Room Air Conditioning (CRAC) systems, hot aisle/cold aisle containment, and hybrid approaches. Companies such as?Airedale International,?Daikin Applied,?Munters, and?Trane Technologies?have been long-time providers of air-based solutions, focusing on optimizing airflow and reducing energy costs. These solutions have been widely adopted in traditional data centers, where efficient airflow management is critical to keeping equipment cool and operational without significant energy waste.

In contrast,?liquid cooling?solutions are becoming more mainstream, particularly for high-density computing environments where air cooling proves insufficient. Companies like Submer,?Iceotope Technologies,and?GRC (Green Revolution Cooling) are at the forefront of immersion cooling, where servers are submerged in dielectric fluid to efficiently dissipate heat. Liquid cooling allows for higher computational performance without the risk of overheating, which is especially important for applications in AI and high-performance computing (HPC). In addition, companies like?CoolIT Systems?and?Allied Control?are deploying direct-to-chip liquid cooling, where liquid targets specific heat-generating components, providing a more targeted cooling solution.

Phase-change cooling, another innovative approach, involves using coolants that vaporize upon absorbing heat and condense to release it, offering highly efficient cooling. These solutions, developed by LiquidStack and Accelsius are beginning to see deployment in advanced data center applications. Through leveraging the ocean’s natural cooling properties,?NetworkOcean?aims to eliminate freshwater usage and drastically reduce energy consumption, positioning itself as a pioneer in this futuristic cooling approach. Similarly,?Lumen Orbit?is exploring advanced cooling mechanisms for edge computing and space environments, where efficient thermal management is crucial.

Solving the Cooling Conundrum

The cooling challenges in data centers are not new. Microsoft has long recognized the thermal load issues within its facilities, experimenting with innovative solutions such as the 2018 Project Natick, which submerged a data center 117 feet underwater, and leveraging Sweden’s low-temperature climate for natural cooling in 2021 [1,2]. Yet, neither of these approaches has been deployed at scale.

While Microsoft's early innovations were groundbreaking, they missed the AI revolution by failing to commercialize these technologies and underestimating the immense thermal demands of next-generation compute. This oversight has opened the door for companies like OceanNetworks, LiquidStack, and Accelsius, whose submerged and phase-change cooling solutions are poised to handle the increasing thermal requirements of hyper-dense modern data centers.

We are also witnessing a shift in leadership among cooling providers. Legacy air-cooling solutions, such as those from Airedale International—serving the data center industry for over 50 years—will struggle to meet the 70% rise in thermal output driven by Nvidia’s new Blackwell GPUs in compute-focused data centers[3]. While companies like Airedale will continue to support cloud storage-oriented facilities, they will face a shrinking market as data centers grow in scale and density.

The Next Generation of Data Centers

The future of data centers is no longer about simply keeping hardware cool—it’s about doing so with ever-increasing efficiency, sustainability, and adaptability. The rise of AI workloads and high-density compute environments has fundamentally altered the cooling landscape, forcing the industry to rethink traditional approaches.

In the rapidly evolving field of compute, success will not be defined solely by who can build the most advanced AI chips or expand server farms the fastest, but by who can integrate the most effective cooling systems into their infrastructure. The companies that master the cooling conundrum will be the ones that lead us into the future of achieving accessible and affordable compute.

Data Center Cooling Companies (45+ Companies)

https://airtable.com/appKBE21AZnu9NbmF/shrTDYqGR7NZ0MMvw/tblfWb6tKBQIxNx8X

Sources

[1]?????????? “Microsoft finds underwater datacenters are reliable, practical and use energy sustainably,” Source. Accessed: Oct. 15, 2024. [Online]. Available: https://news.microsoft.com/source/features/sustainability/project-natick-underwater-datacenter/

[2]?????????? “How Microsoft’s new datacenter region in Sweden incorporates the company’s sustainability commitments,” Source EMEA. Accessed: Oct. 15, 2024. [Online]. Available: https://news.microsoft.com/source/emea/features/how-microsofts-new-datacenter-region-in-sweden-incorporates-the-companys-sustainability-commitments/

[3]?????????? T. Mann, “Nvidia turns up the AI heat with 1,200W Blackwell GPUs.” Accessed: Oct. 16, 2024. [Online]. Available: https://www.theregister.com/2024/03/18/nvidia_turns_up_the_ai/

?????????? R. Miller, “Google Developing New Climate-Conscious Cooling Tech to Save Water,” Data Center Frontier. Accessed: Oct. 16, 2024. [Online]. Available: https://www.datacenterfrontier.com/cooling/article/33001080/google-developing-new-climate-conscious-cooling-tech-to-save-water

?????????? “A beginner’s guide to data center cooling systems.” Accessed: Oct. 15, 2024. [Online]. Available: https://www.vertiv.com/en-us/about/news-and-insights/articles/educational-articles/a-beginners-guide-to-data-center-cooling-systems/

?????????? “What Are the Best Data Center Cooling Solutions?,” Data Center Resources | Aisle Containment & NOC Manufacturer. Accessed: Oct. 16, 2024. [Online]. Available: https://datacenterresources.com/articles/what-are-the-best-data-center-cooling-solutions/

?????????? O. Johnson, “Data Center 50: The Coolest Data Center Companies Of 2023 | CRN.” Accessed: Oct. 16, 2024. [Online]. Available: https://www.crn.com/news/data-center/data-center-50-the-coolest-data-center-companies-of-2023

?????????? “Premier 22 Companies in Data Center Cooling Solutions.” Accessed: Oct. 16, 2024. [Online]. Available: https://www.inven.ai/company-lists/top-22-data-center-cooling-solutions-companies