Cooling Systems in Data Centers

In part 2 of our blog series “Why Mechanical Engineering is Crucial to Data Centers”, we’ll take a look at the various cooling systems that Mechanical Engineers (MEs) consider when designing data centers.

Initial Considerations

To prevent overheating and keep the IT equipment running, MEs must work with the customer to make decisions on the types of cooling systems that will be optimal for the project. When beginning the design process, the ME will start with some key initial considerations:

1. Geographical location of the facility
2. The type of IT equipment being installed
3. Customer preferences

The geographical location of the facility will greatly impact system choices. A colder climate might lend itself to facilities that can use more outside air or ‘Free Cooling’. A hotter climate may require other methods to reduce temperatures, such as mechanical compressors or the use of water to reduce air temperatures through evaporation.

The type of IT equipment being installed may dictate what cooling systems apply best. Most traditional servers and storage devices can use air to remove heat. At lower heat densities, Perimeter CRAC (Computer Room Air Conditioners) or CRAH (Computer Room Air Handlers) units may work well.

As the heat density increases, row containment and in-row air cooling units are often good add-ons to maintain the required server inlet temperatures.

Newer IT technologies such as equipment utilizing GPUs (used in AI systems) generally require constantly flowing direct fluid contact (direct-to-chip) to remove their much-higher heat output. Air systems can typical only handle approximately 20% of the AI systems total heat dissipation. The HVAC equipment required for direct to chip cooling is newer technology and multiple options are currently available depending on the total AI load and density configuration (i.e. how many kW per rack).

Without Mechanical Engineers, AI couldn’t generate fun photos like this one.

Customer preferences can have a big impact on the design selections for data center cooling systems. A customer may have an existing chilled water system or no water at all. ‘Staying the Course’ or trying something different could be important to them. They might have corporate Green Energy programs that would suggest the use of systems with lower energy costs and/or less impact on the environment. A customer may have financial restrictions that would lead us to look at systems with less expensive initial capital costs. A customer may also have a strict deadline and need to complete a project within a tight time frame. Any of these factors could have a significant impact on the selected design.

All factors mentioned above would be taken into consideration when a mechanical engineer begins their cooling system selection process. This phase of design generally requires significant data collection, interviews with the customer(s) and analysis of options before the engineer can determine the best options to present.

Types of Cooling Systems

There are two main types of cooling systems in a data center, air-based cooling and liquid-based cooling.

Air-Based Cooling

Air-based cooling involves using fans to blow cool air into the data center. This can be accomplished by using:

1. Computer Room Air Conditioning (CRAC) Units:
   1. Most CRAC units work much like traditional residential air conditioners. They remove heat from the air by using fans for air movement across coils, and use compressors to effect evaporation and condensation of the refrigerant. The heat collected in the refrigerant is either removed by being routed directly to heat exchangers in exterior condensers or removed by transfer to a glycol/water solution within the indoor unit which is, in turn, routed to exterior dry cooler(s).
2. Computer Room Air Handler (CRAH) Units:
   1. CRAH units reject heat to chilled water instead of refrigerant. Cold water in a facility water system (FWS) loop, typically containing a certain percentage of glycol and cooled by a separate chiller, is routed through the coils in the indoor unit. Fans then blow the air across the coils and circulate the air throughout the data center.
3. Air Handling Units (AHUs):
   1. AHUs are not typically found in the white space but in dedicated adjacent galleries. They are usually much larger than perimeter ACs. Often, they are ‘one piece’ configurations, not requiring remote heat rejection. Data center air handlers will cycle outside air (Free Cooling) whenever conditions are acceptable. When ambient temperatures are too high, they can harness the evaporation of water to reduce temperatures. These units are often deployed in hyperscale data centers.
4. In-Row Cooling
   1. These units are placed directly between the IT racks. They are also usually deployed in rows with containment, allowing enhanced segregation of the supply and return air streams. This segregation and their close proximity to the heat sources can yield higher cooling capacities and more targeted cooling.

Air-based cooling strategies are common for rack densities up to 50kW. With careful design and the appropriate equipment selections, levels as high as 100kW can be attained.

Liquid-Based Cooling

Liquid-based cooling systems can be more efficient that air-cooling, and allow for cooling of much higher densities of IT loads, as liquid can transfer significantly more heat than air. Once rack densities reach 50kW or more, liquid-based cooling systems start to become necessary to effectively cool the IT equipment. Current commercially available liquid cooling systems can handle rack densities of more than 100kW.

Many new cooling products lines are in the pipelines as the AI chip companies develop even more dense GPUs. Estimated rack densities in 2026 will likely reach as high as 200kW per rack.

1. Rear Door Heat Exchangers
   1. Rear door heat exchangers are installed on the rear of IT cabinets to reduce or remove the temperatures being exhausted from the IT equipment. They utilize chilled water or water/glycol solution loops, and act a lot like a CRAH unit. The exhaust from the IT equipment in the rack enters the rear door and travels across its coils, reducing the exhaust temperatures and in turn, the load on the room cooling systems. They are available in passive (no fans) and active (with fans) configurations. These systems can assist are typically good for cooling up to 50kW per rack, with some providers claiming up to 200kW per rack.
2. Direct-to-Chip Liquid Cooling
   1. Direct-to-chip (DTC) liquid cooling utilizes a cold plate, placed directly on the chip, with a cooling fluid pumped through the cold plate to chill the chip. A Cooling Distribution Unit (CDU) precisely controls the liquid flow through the TCS (technology cooling system) loop. This allows for very high density cooling applications such as GPU deployments, with rack densities exceeding 100kW.

1. Immersion Cooling
   1. Immersion cooling involves placing all of the IT equipment into a vat of dielectric (i.e., non-conductive) fluid. This setup also allows for very high density cooling applications.

Additionally, there are two types of liquid cooling when talking about DTC or Immersion, single-phase and two-phase. In a single-phase system, the cooling liquid remains as a liquid during the cooling process (i.e., a single phase), while in a two-phase system, the cooling liquid reaches its boiling point and changes phase to a gas (i.e., two phases).

Heat Rejection

Regardless of the type of cooling system specified, ultimately the heat needs to be rejected from the building. This can be done through:

1. Condensers (mechanical cooling)
   1. Condensers are typically paired with refrigerant based CRACs. Their fans and heat exchangers allow rejection of the heat. Their cooling capacity depends heavily on outdoor air temperatures. However, they use no water, making them a good solution in applications where water usage is an issue. Their first cost is attractive, and they are stand-alone systems, which can enhance redundancy efforts in the design. They do have distance limitations (between the CRAC and the condenser).
2. Dry coolers (mechanical cooling)
   1. Dry coolers are typically paired with water/glycol solution based CRACs. Their fans and heat exchangers allow rejection of the heat. Their cooling capacity depends heavily on outdoor air temperatures. These units can be installed in a stand-alone configuration, or the dry coolers can be on a common loop, with multiple dry coolers rejecting heat from multiple CRACs. Their distance limitations are less severe than impact (as compared to refrigerant based systems).
3. Evaporative cooling (typical in data center air handlers)
   1. Evaporative cooling systems can be very efficient by using the natural process of water evaporation to absorb heat. They can be ‘direct’ or ‘indirect’ systems. The first allows direct contact with the air, with evaporation reducing the dry bulb temperature, and simultaneously increasing humidity. This air is channeled into the white space. Indirect evaporative units use heat exchangers between the evaporatively-cooled air and the data center air. This allows better control of the data center humidity.

1. Free cooling (economization)
   1. Free cooling, also known as economization, utilizes outside air to cool the data center.
      1. In data center air handlers: AHUs can utilize outside air to cool the data center. When the air temperature is cool enough, it can be routed directly into the data center.
      2. In chilled water (FWS) or water/glycol systems: The heat rejection equipment for closed loop FWS or mechanical cooling systems can be outfitted with devices that allow economization. This reduces their reliance on mechanical augmentation to control data center temperatures.
2. Pumped refrigerant
   1. Similar in function to economizers mentioned above, pumped refrigerant systems make use of cool outside air temperatures to chill the refrigerant in a closed loop system. This reduces their reliance on mechanical augmentation to control data center temperatures.