An Introduction to Server Racks and What needs to be considered when transfer to AI rack design

This article provides an introduction to server racks, focusing on their purpose, types, standards, the advantages of the OCP rack standard, and future challenges. Busbar and chassis are not included.?

?

The Purpose of Server Racks

Racks provide a standardized and organized way to house and manage IT equipment, including servers, network switches, and other devices. They optimize space utilization and resource management within data centers, server rooms, and network closets.

Racks offer numerous benefits, including:

1. Space Efficiency: Racks consolidate multiple devices within a defined area.
2. Organization: Racks offer a structured way to arrange and manage equipment, enhancing the overall organization and efficiency of the data center.
3. Cable Management: They facilitate neat cable routing and organization, reducing clutter and improving airflow.
4. Accessibility: Racks provide easy access to the front and back of equipment for maintenance and troubleshooting.
5. Cooling: Particularly rack/enclosures, promote efficient airflow management and cooling, essential for protecting temperature-sensitive equipment.
6. Security: Enclosed racks with lockable doors and side panels offer physical security to protect valuable IT assets.

Three primary types of server racks:

1. Open Frame Racks: These racks consist of vertical mounting rails and a supporting framework, but lack side panels or doors. They are suitable for applications that don’t require strict airflow control or physical security, such as network wiring closets or distribution frames, due to their open design and ample space for cable management.
2. Rack Enclosures: Also known as rack cabinets or enclosures, these racks feature removable front and rear doors, side panels, and four adjustable vertical mounting rails. The doors, often ventilated, encourage airflow from front to back for cooling. Rack enclosures are well-suited for heavier equipment, high-density data centers, and server rooms requiring equipment security. This type of rack is the key agenda of this article.?
3. Wall-mount Racks: These racks are designed to be mounted on a wall to conserve floor space. They can be open-frame or enclosed and are generally smaller, supporting less weight than floor-standing racks. Casters can be added to make them floor-standing.

A basic rack consists of two or four (Sometimes even six) vertical mounting rails and a supporting framework, typically made of steel (Cold rolled plate), to hold heavy equipment. The rails have square or round holes (See the picture below) for mounting equipment with screws, and in the case of square holes, cage nuts are used. Some equipment includes horizontal rails or shelves for extra support. Rack dimensions, including rail width, hole spacing, and cabinet sizes, are standardized to ensure compatibility. Most IT equipment follows the 19-inch width standard, defined by the Electronics Industry Alliance (EIA) and maintained by the Electronic Components Industry Association (ECIA) under EIA-310-D/E, similar to international standards like IEC-60297-3-100 and DIN 41494 as well as SSI specification (Server system infrastructure). Nonetheless, EIA standards are not mandated by any regulatory authority, so compliance is voluntary. Manufacturers can choose whether to follow these guidelines when building cabinets. However, most manufacturers opt to adhere to EIA standards to ensure compatibility with established market practices. 19-inch racks dominate IT applications.

Rack specifications

The height of a rack is measured in "rack units" (U). One rack unit (1U) equals 1.75 inches. The height of a rack is denoted by the number of rack units it accommodates, for instance, a 42U rack provides 42 rack units of space for equipment. When selecting a rack, ensure it has enough rack units for your current and future equipment needs. AWS is reducing the usage of 42U and moving to 48U.?

Vertical hole spacing definition:?

Horizontal spacing:

Front panel width:

Depth:

Load rate: This indicates how much weight the rack can safely support. It’s essential to also ensure that the facility’s floor can handle the combined full-loaded weight of the rack system (The rack and the equipment installed inside). Racks typically provide two types of load ratings: a stationary/static load rating for when the rack is fixed in place, and a rolling/dynamic load rating for when the rack is moved on its casters.?

The Open Rack standard, developed by the Open Compute Project (OCP), introduces a new approach to rack design specifically optimized for data centers. It addresses the limitations of traditional rack designs like the EIA-310-D specification, which was not originally intended for data centers and their specific requirements. The OCP promotes a “grid to gates” design philosophy, encompassing a holistic approach that considers the interdependence of all components within the data center ecosystem, from the power grid to the chips on motherboards.?

Advantages of Open Rack

The Open Rack standard offers several key advantages over traditional rack designs:

1. Improved Airflow and Thermal Management: The Open Rack uses OpenU (OU) as the unit of measurement, slightly taller than the traditional rack unit, which enhances airflow for more effective cooling.
2. Modular and Flexible Design: It allows for flexible density by supporting IT chassis ranging in height from 0.5 OU to 12 OU.
3. Efficient Power Distribution: The Open Rack implements a cableless power distribution system. Servers plug directly into bus bars at the rear of the rack, eliminating the need for individual power supplies. This simplifies cabling and enhances power distribution efficiency.
4. Extended Component Life Cycle: The standard is designed to maximize the product life cycle of compute components. Instead of replacing entire servers, components are replaced based on their individual life cycles, which can be up to 10 years. This reduces waste and lowers the total cost of ownership (TCO).

Compatibility with Existing Infrastructure

The Open Rack, while introducing new features, maintains compatibility with existing data center infrastructure:

1. 24-inch Column Width: Conforms to the standard floor tile pitch in most data centers.
2. Adaptability for 19-inch Equipment: While having a 21-inch wide equipment bay, the Open Rack can accommodate existing 19-inch equipment.
3. Enhanced Space Efficiency: The Open Rack design provides 87.5% space efficiency, surpassing the 73% efficiency of traditional 19-inch racks.

The OCP Rack Standard is designed for continuous evolution to meet the changing demands of data centers and large-scale computing. For example, the standard has been updated to include two IT Gear latch depths (645mm and 789.6mm) and two depths of IT Gear (~800mm and ~660mm).

The brief introduction of the sibling of the rack, chassis:

1. Height: The height of the server chassis is typically measured in "U" units, where 1U equals 1.75 inches or 44.45mm. To ensure easy stacking and disassembly, the actual height of a 1U chassis is often designed to be slightly less than 44mm. This principle applies across different U sizes, with chassis heights generally kept slightly below the standard U measurement. This allows for small gaps between the units in the rack, facilitating easier mounting and dismounting without compromising fit.
2. Width: A standard 19" server cabinet features support shelves made of sheet metal or aluminum at the four interior corners to support and secure the chassis. Due to these shelves, the actual internal width available for the chassis is around 450mm (approximately 17.7 inches), rather than the full 19 inches. When designing the chassis, it’s important to consider ease of mounting, so the width is typically kept below 450mm, usually not exceeding 440mm. This leaves about a 5mm space on each side for easier installation.

If sliding rails are to be used, which are commonly found on chassis up to 4U in height, the width must be further reduced to accommodate them. Slide rails are typically 10 to 20mm thick, so with rails installed on both sides, the chassis width must be narrower to fit properly, ensuring smooth disassembly and maintenance.

1. Depth: When designing the depth of the chassis, it’s important to account for space in the cabinet for heat dissipation, wiring, and other mechanisms. Since there is limited space on the sides, the front and rear areas are used for this purpose. Typically, there should be at least 75mm of clearance at both the front and rear (including the space for the mounting frame). Therefore, for a cabinet with a depth of 600mm, it’s ideal to design the chassis to be no more than 450mm in depth to allow sufficient room for these considerations.

The brief introduction of the second sibling of rack, Busbar:?

The bus bars at the rear of the rack transmit power from the rack-level power sub-system to the IT gear housed in the equipment bay. This design allows the equipment to connect directly to the power source, eliminating technicians needing to disconnect power cords at the back of the rack before servicing the equipment.

Rack options: Thermal-related options are doors and side panels. Maintenance-related options are roof for cable management, casters (wheels on the bottom of the rack) or levelers (Fixed), and tool-less mounting. Security-related options are locks and customized security latches. An enhancement option can be the seismic kit.?

Following shares some of the option questions while I led the discussion of providing the customized rack for the common platform (Including the new rack used by AI server):

1. Power distribution and management: AI rack may require supporting around 120-500KW. Equip racks with more power distribution units (PDU) are expected to accommodate higher power requirements. Busbars as direct connections from power supply to servers may have higher requirements in safety.?Power shelves typically mounted in the rear or lower sections of the rack, distribute DC power to all equipment. The design must ensure there is enough space for the power shelf while accommodating liquid cooling components such as coolant distribution units (CDUs), tubing, and heat exchangers.
2. Cooling and airflow/liquid flow: Traditional front-to-back airflow is designed for old racks. However, AI server requires higher efficiency to get rid of heat which means the coolant distribution unit (CDU) and pump could be inside or near the rack. Some of the liquid cooling system uses Direct-to-chip cooling which provides and cold plate directly on the device such as GPU and CPU and circulation of cool liquid brings down the temperature of the cold plate and reduces the heat on the device. The design of the new rack needs to support a new cooling system.?In addition to providing more spaces, there is a saying, that the bigger means the better. 46U to 52U shall provide enough room for GPU, networking equipment, liquid cooling equipment, power distribution, and CMA. Also, the depth of rack shall be around 1200mm (48 inches) to allow space for air flow and RDHX usage.
3. Weight consideration: The increased weight of GPU and cooling system including pipes, cold plate, and coolant add significant weight. Some may design a radiator, pump, and/or heat exchanger inside the rack. AI rack challenges the specification of traditional full payload weight, which is 3000 lbs (1360.8 kgs) stationary / 2250 lbs (1020.6 kgs) rolling load capacity. The new rack needs to handle the additional load without compromising the stability. In addition, we may consider the option of a strengthened frame to increase the old rack stability and balance the cost and robustness of the new design. A moveable cross brace/stiffener could be one of the workable options. ?
4. Leaking prevention and management: When an unlikely accident happens, the new rack may include a containment tray or special channel to manage leaks and provide space for leak detection sensors.?
5. Rack maintenance and servicing: To simplify maintenance, racks with liquid cooling systems often use quick-disconnect fittings on coolant lines. These fittings allow for easier replacement of components without draining the entire cooling loop. Rack designs should ensure these fittings are accessible and secure. The rack may be designed to support modular liquid cooling systems that can be serviced without shutting down the entire rack. This includes allowing for easy removal and replacement of cooling components like pumps or tubing.?Also, to improve serviceability, tool-less rack sliding rails (Left and right rails interchangeable) should be used for all the slots. ?

Generally, we are moving to AI era and the new design of the rack can't be limited to its silo as we did traditionally but needs to consider all other peripherals including, chassis, PCBA, busbar, PSU, PDU, thermal solution, cable/pipe management, etc. Free to share your thoughts with me!?