Testing and Commissioning in Data Center Facilities

ASHRAE (American Society of Heating, Refrigerating and Air-Conditioning Engineers) has been publishing technical articles and providing training programs on the subject of Commissioning since 1982. This topic has been particularly important for mechanical systems, which are distributed and operated with integrated control. Commissioning is a process that helps overcome quality problems in such systems.

ASHRAE sets out Commissioning as a quality-oriented process to develop the delivery of a project. According to its definition: “The process focuses on verifying and documenting that the facility and all of its systems and assemblies are planned, designed, installed, tested, operated, and maintained to meet the Owner’s Project Requirements.”

Data Center facilities have higher expectations from the commissioning process because they cannot simply switch off their powering and cooling systems like regular buildings. While regular buildings can be fully switched off during maintenance and repair, data centers cannot afford such a luxury due to the risk of downtime. Even partial shutdowns can result in the loss of data and operation, which is why data center facilities cannot take such risks.

The testing and commissioning process in data center projects primarily aims to achieve the following targets:

• The data center infrastructure utilizes multiple units to ensure redundancy, structural compatibility, lower operating costs, and optimum investment. To design the operation capability of these units, complicated electromechanical systems, and distributed topologies are utilized. However, these systems are not guaranteed to operate ideally under practical conditions since they are designed based on ideal scenarios and not tested under integrated systems. This is why trial and error tests and optimization are required during the Testing and Commissioning process before production to ensure that the units function harmoniously with the design and structural parameters at the site. Holistically performing these verifications, particularly under varying IT loads and different modes and conditions, guarantees the elimination of future problems.
• The data center infrastructure comprises around 15 to 20 closely interconnected systems, especially in the disciplines of power, cooling, security systems, and automation. To ensure the uninterrupted operation of the data center, it is essential to isolate and recommission each system's equipment and distribution components during maintenance and other operational interventions. This allows for a thorough evaluation of the performance of every equipment, device, and distribution element, which is critical for maintaining high business continuity. A poorly designed or ignored device that costs only one hundred dollars can jeopardize an investment worth millions of dollars.
• The core staff of the operation team has the opportunity to test all high-risk maneuvers, such as full load, failure, and maintenance scenarios, with dummy loads. During these phases, the initial operation procedures and guides are practically created to eliminate risks and ensure high business continuity in operations throughout the data center lifecycle. Once critical IT equipment is positioned at the data center and the actual IT service is initiated, the team is expected to be competent enough to perform all maneuvers, back up all systems with no downtime, and do so with self-confidence.

For each data center, the Testing and Commissioning process should be started at the plan and design stage of the data center. After a data center investor decides to build a data center, generally the first two agreements are vital for guaranteeing return on investment. The first one is the “Consulting Firm that will set out the requirements and conceptual design of the data center” and the second one is the “Testing and Commissioning Authority” which will guarantee that the investment is realized in the most suitable way for the business objectives.

During the implementation phase, data center testing and commissioning processes differ from those in the regular building industry. Most Data Center service providers have their unique approach. In any case, it is coordinated in levels according to the project plan.

The coordination of testing and commissioning is usually defined in five levels. I put forward that a sixth stage should be included at the start of these stages. The projects in which I have provided consultancy services proved that this sixth stage added high value. Although these stages can be managed with various contents and timing depending on the site conditions, its essentials can be summarized as follows:

• Level Initial: During the testing and commissioning stage, the authority responsible for this process reviews the critical facility projects based on the calculations and drawings provided. They ensure that the contractor companies working in electrical, mechanical, security, and automation disciplines are aligned in their approach and specify their mutual expectations and coordinated operation principles. If necessary, they clarify the details with the product suppliers.
• Level One: Before installation, any equipment that can limit the performance and capacity of the facility, such as generators, chillers, UPS, and any customized devices developed specifically for the project, like electrical, DDC, MCC panels, etc., are thoroughly tested in a factory environment. This stage of testing ensures that the equipment is compatible with other disciplines and that any issues are identified, developed, and approved before installation.
• Level Two: It is important to install the required electrical and mechanical distribution elements such as cabling, piping, and channeling according to the plan, design, and standards before placing and starting up the capacity equipment. The compliance and quality must be tested at each production stage. The installation production tests play a crucial role in ensuring the availability of the required capacity, functional performance, and perfect control of the systems being installed.
• Level Three: During this stage, the capacity equipments are activated in a proper order. The power, signal, water, and gas cycle connections are commissioned progressively from the distribution components. The service provider who starts up each piece of equipment will first check the operating conditions. Then, they will test and demonstrate to the testing and commissioning team that all the functions and communications of the device are operational.
• Level Four: The white spaces in the data center that will house critical IT hardware are now filled with dummy loads that mimic their power and cooling behavior. The building management system, energy management system (SCADA), and environmental monitoring systems are operational, and the interface screens and alarm management system are ready for use by the operations team in the control room. The systems are tested partially and thoroughly in a suitable order until approximately 15 to 20 integrated systems that make up the data center infrastructure are connected. Any issues are identified and resolved.
• Level Five: The entire data center is now fully operational and is expected to run smoothly without any disruptions during its lifespan. The BMS and EMS (SCADA) systems can be managed end-to-end with high reliability. All potential failure, maintenance, and operational scenarios have been tested with dummy loads and addressed accordingly. Tests particularly under varying IT loads and different modes and conditions are performed and recorded for future reference. Meanwhile, the companies responsible for implementation and product suppliers will transfer the knowledge, insight, and responsibility to the operation staff at the end of this stage.

If any facility certification is required, the necessary control tests can be performed under the responsibility of the operation staff after this stage.

The performance of a data center beyond Level Five is mainly determined by the behavior of the team managing it. It takes time for the operational team to establish an organizational culture and standardize their activities. The data center is expected to meet business objectives as it transitions towards Sustainable Operation, which is an independent operational model that strives for excellence regardless of the people involved.

Data Center Cooling Optimization Series

• What are the Advantages of Using DC Aisle Containment Systems? Do they improve Energy Efficiency?
• Improving the Efficiency of DC Cooling Systems? (1)
• Improving the Efficiency of DC Cooling Systems? (2)
• How Confident are you in the Thermal Stability of your Data Center?