Reaching Toward Sustainable AI with Immersion-Born GPU Servers

As you’ve all been around the data center industry for more than a minute, you know that startups, enterprises, multinational corporations, and governments are aggressively deploying intensive data center infrastructure for AI and other demanding workloads. To do that, they need unprecedented data center capacity. Growth is off the charts: colocation volume doubled in the past four years and analysts predict hyperscale capacity to grow by 3x over the next six years.??

But growth comes with a downside…or to be honest, several downsides.??

Understanding the Challenges?

Of course, the industry wants to capitalize on the demand but it hasn’t devised a way to deliver sustainable growth. Even the most sophisticated hyperscalers recently abandoned their net zero commitments due to the challenges of building and operating green data centers.?

Whether these data centers are used for ML, inferencing, or HPC, we have a core problem. Facts are facts: today’s conventional data centers for AI aren’t environmentally friendly. Anyone who is building or operating these data centers experiences:?

• Skyrocketing Power Consumption: AI and HPC workloads are inherently power-hungry, driving up energy consumption and operational costs. New builds frequently consume 100MW, an almost unimaginable size a decade ago.?

• Water Usage: Increased cooling demands can lead to higher water consumption, putting pressure on already scarce and in-demand water supplies.?

• Carbon Footprint: The energy consumption and water usage associated with traditional cooling methods contribute significantly to the carbon footprint of data centers. Data center emissions are expected to keep rising, surpassing those of the airline industry.?

So we’re stuck. We must build more data centers to support intense computational demands at scale, but those data centers demand more power, utilize too much water, and come with a spiraling carbon footprint.?

Fortunately, we’re at the cusp of a change in how we architect and run data centers. More and more data center operators are willing to consider new ways to cool their data centers because they’re beginning to see how air cooling is at the heart of the industry’s sustainability problem.?

?Solving the Cooling Conundrum?

Though air cooling has been a reliable solution for decades, it’s not well suited to the hot and dense infrastructure used for HPC and AI because air is not a highly efficient heat transfer medium. It’s susceptible to problems like airflow obstructions and hot/cold air mixing.?

To make matters worse, rack density for computationally intensive tasks exceeds 100kW, putting unprecedented pressure on air cooling systems built to support 15-20kW per rack. This disconnect comes with compute performance degradation, decreased hardware reliability, and increased maintenance costs. In many cases, air cooling simply can’t support the latest chipset and server designs.?

In the average data center, air cooling systems account for around 40% of a data center’s total energy consumption, so it’s obvious that adding AI will drive up energy consumption and carbon footprints.??

If we stick with air, we’re stuck in a no-win scenario.?

So what can we do??

?The Immersion Advantage?

Simply put, we have to stop using air. It’s time to cool our data center infrastructure with liquid. The simplest and most efficient way to do that is to immerse data center hardware in a non-conductive dielectric fluid, so that every component can dissipate heat into the fluid, and pumps can carry the heat away. We call that immersion cooling, and is far more efficient at absorbing and dissipating heat than air cooling. Immersion cooling reduces power consumption by 50%, cuts water consumption to zero, can handle higher heat loads more effectively, eliminates fans, improves component lifespan, and protects against physical damage.?

Outside of these advantages, it’s an eco-friendly approach that can deliver overall data center power utilization efficiency (PUE) as low as 1.02. If deployed at scale, it would substantially reduce the overall carbon footprint of data centers everywhere.?

But it delivers the best results with the best servers.??

Choosing the Immersion-Born GPU Server?

As you know, AI, HPC, and other demanding workloads require specialized servers that can include the latest CPUs, leading-edge GPUs, and high-performance interconnects. Not all servers are ideal for AI infrastructures. To make matters even more complicated, not all servers are ideal for immersion cooling. Some are simply not built with the right materials, while others aren’t designed from the ground up for maximum efficiency in immersion environments. It’s important to evaluate your distinct server requirements to make sure you’ll end up with all the advantages of immersion cooling?Hypertec is here to help.?

Not only do we have the only portfolio of immersion-born servers (built from the ground up for immersion), we’ve announced ?the Trident iG series; new immersion-born GPU servers for the most challenging requirements, including large language model training, deep learning, inference, and even applications like financial modeling, drug discovery, and climate modeling. Offering dual-socket CPU architecture and up to six GPUs, they have been designed from the ground up to take full advantage of the immersion environment with a set of distinctive technologies and a differentiated design.?

How did we design our Trident iG series??

We rethought how servers should be designed to maximize impact and value.?

Conventional servers are laid out for airflow, drawing air into the front of the server and out the back. This GPU server is engineered with an advanced immersion thermal management design, utilizing a tiered cooling structure within the immersion tank.??

• The hottest Tier 1 components, like GPUs, are located at the lowest part of the tank where coolant is most effective.?

• Tier 2, the middle of the server, includes for moderate-heat components like CPUs.???

• Tier 3 components with lower thermal output, such as NICs and other peripherals, are located at the top of the server.?

This thermal design ensures consistent temperatures across critical components and optimizes overall server performance.???

A new standard in density, TCO, and usability, the TRIDENT iG immersion-born GPU servers are ready to disrupt traditional data center infrastructure by maximizing GPU density while lowering total cost of ownership (TCO) and improving serviceability. With a streamlined design and high usability, organizations can scale their AI and HPC capabilities more efficiently than ever, making the TRIDENT iG series, the preferred choice for those seeking to drive AI innovations while reducing their environmental footprint.??

Supporting a Greener Future?

Hypertec, as a market leader in powering AI sustainably, has, for years, helped data center designers and operators move forward faster toward a sustainable future. However, sustainability isn’t our only goal. By making data centers more resource-efficient, we’re able to help them cut their costs, improve their reliability, and deliver competitive advantages like superior density and performance that air-cooled data center environments can’t match. As our VP of Engineering, Darcy Letemplier, puts it:??

“Our iG immersion-born GPU systems are built for intense AI and HPC workloads, combining efficient cooling, power redundancy, and scalability to deliver the most optimal performance for any type of workload. Through innovative and strategic partnerships, we continue to address challenges in power, thermals, and density, pushing immersion technology forward”?

?

?

This post is also available in: FR