Two-phase and Single-phase Liquid Immersion Cooling

May 18, 2022 by GIGABYTE

Two-phase and Single-phase Liquid Immersion Cooling

The number of challenges currently faced in the deployment of data centers is rapidly increasing due to the effect of next generation IT applications, such as the rise of GPU parallel computing applications, continuous services, and demand for real-time data synchronization. Due to the rising temperatures of key components in IT equipment, operators are searching for new feasible, energy-efficient cooling options. Hence, GIGABYTE looked beyond conventional air cooling and began researching liquid cooling at an early stage, proposing two-phase and single-phase liquid immersion cooling products. These two types of products seem to be the same but have entirely different operating models. Neither of the products have an absolute advantage over the other. It all depends on the user's investment plan and performance requirements. GIGABYTE aims to offer the widest selection of data center products and cooling options to meet the needs of consumers for diverse applications.

Comparison of Cooling Systems

	Two-phase Liquid Immersion Cooling	Single-phase Liquid Immersion Cooling	Air-cooled
Structural Complexity	★★★★☆	★★★☆☆	★★★★★
Installation Cost	★★★★☆	★★★★☆	★★★★★
Cooling Efficiency	★★★★★	★★★★★	★★★☆☆
Operating Expenses	★★★★☆	★★★☆☆	★★★★★
Cooling Capacity	★★★★★	★★★★☆	★★★☆☆
High-density Performance	★★★★★	★★★★★	★★★☆☆
Reliability	★★★★☆	★★★★★	★★☆☆☆
Deployment Flexibility	★★★★☆	★★★★★	★★☆☆☆

How the Two Cooling Technologies Work

Two-phase and single-phase liquid immersion cooling both use a non-conductive liquid, which has better thermal conductivity than air, for cooling. Besides better heat capacity provided by the phases, the liquid immersion cooling method uses liquid to directly come in contact with the heat source for better heat dissipation.

Furthermore, both of these two cooling structures do not need:

Cooling fins and fans in the IT equipment
Machine room refrigerated air conditioning system (CRAC & CRAH), raised access floor

Two-phase Liquid Immersion Cooling System

In a two-phase liquid immersion cooling system, IT equipment (server) is installed in a specially designed sealed tank that uses a non-conductive liquid medium. Low boiling point liquids are chosen, so the heat generated by servers can easily cause a phase change where the liquid surrounding the components boils and generates vapor, which in turn goes through a phase change returning it to a liquid state while removing the heat. This is why it is called "two-phase" liquid immersion cooling. After changing phases, as the bubbles rise up through the liquid the vapor is condensed on the sealed tank's condenser coils, where it changes from a gaseous phase back to a liquid phase and returns the fluid to the immersion bath. The sealed tank maintains the phase change process through environmental controls, and the heat exchange process continues.

Single-phase liquid immersion cooling system

In a single-phase liquid immersion cooling system, the server is also installed in a tank with a non-conductive liquid medium, which is usually a hydrocarbon-based liquid similar to mineral oil. Heat generated by servers is conducted through the direct contact between the medium and components. Unlike two-phase liquid immersion cooling, this type of system requires a pump in the coolant distribution unit (CDU) to suck the medium into the cooling device for cooling, and goes a step further by using an additional heat rejection equipment to complete the heat exchange cycle. Due to the high boiling point of this type of liquid, the liquid is not volatile, so the liquid tank of a single-phase liquid immersion cooling system does not require a strict sealed design and environmental controls.

System Complexity and Installation Cost

The greatest difference in cost of two-phase and single-phase liquid immersion cooling systems is closely related to the medium and the environmental control system for a liquid tank. 3M is currently the leading brand and main supplier of non-conductive mediums for two-phase liquid immersion cooling in the global market, and its heat conducting liquid is prepared using fluorocarbons. The medium used in single-phase liquid immersion cooling is usually prepared using hydrocarbons. The design of liquid tanks is divided into sealed and unsealed tanks depending on the heat dissipation method.

Two-phase Liquid Immersion Cooling System

The design of two-phase liquid immersion cooling systems is relatively complex and has a higher installation cost. Even though it does not require an additional pump for heat convection, it requires a very tightly sealed design and environmental controls to maintain the system's vaporation and condensation processes. After the server is immersed in an engineered fluid, the cooling medium is not lost because the tank is sealed, but some will be lost during equipment maintenance because the tank must be opened, and medium evaporation will cause operating costs to increase. The Lawrence Berkeley National Laboratory once conducted a study (https://datacenters.lbl.gov/sites/default/files/ImmersionCooling2016.pdf) and found that the cost of refilling evaporated medium offsets the energy saved by two-phase liquid immersion cooling. Hence, the greatest challenge for two-phase liquid immersion cooling systems is designing a sealed system that is convenient to maintain, in order to reduce the loss of medium from evaporation.

Single-phase Liquid Immersion Cooling System

A single-phase liquid immersion cooling system needs a pump to be installed to push in a lower temperature liquid into the system while, at the same time, the warm liquid is removed from the system. Due to the low volatility of the medium, the tank's structure has a relatively simple design. A single-phase liquid immersion cooling system usually uses a hydrocarbon-based liquid, which is transparent, odorless, and non-toxic, so it is also extensively used in cosmetics. Furthermore, the chemical has been around a long time, so there are no concerns regarding its Global Warming Potential (GWP).

Cooling Efficiency and Operating Expenses

In terms of cooling efficiency, two-phase and single-phase liquid immersion cooling both have far better performance than air-cooled data centers. Two-phase liquid immersion cooling can be connected to an external air cooling or water cooling system for different external environments. In some currently running data centers, two-phase immersion cooling systems have been shown to achieve a Power Usage Effectiveness (PUE) score of 1.02. When single-phase liquid immersion cooling systems are connected to an external cooling system the PUE score can reach as low as 1.03. In comparison, air cooling systems have a PUE between 1.4 and 1.7, which is still efficient, but immersion cooling has shown there is room for improvements in efficiency.

Two-phase Liquid Immersion Cooling System

The process of phase changes in a two-phase liquid immersion cooling system automatically creates recirculation of the liquid. Besides the IT equipment, it does not require an additional pump, and the cooling requirements of the data center do not need to be as powerful at cooling compared to air-cooled servers. The only challenge is the high volatility of the head conducting liquid, which makes it necessary to replenish the medium that has evaporated after equipment maintenance. Designing a tightly sealed system that is convenient to maintain and optimizing & designing the maintenance process are the main factors that will affect operating expenses.

Single-phase Liquid Immersion Cooling System

Single-phase liquid immersion cooling systems are relatively cheap to install due to the structure and consumables used. The heat conducting liquid is not volatile, so it is easy to control operating costs. Some liquids can be used for as long as 15 years. The additional pump that recirculates the liquid used in single-phase liquid immersion cooling systems also consumes power, and it is necessary to periodically test the quality of the coolant to ensure that its key characteristics do not deteriorate over time.

Power Usage Effectiveness Comparison

Cooling Capacity and High-density Performance

Due to the progressive trend of next generation applications, such as AI, IoT, and edge computing, the performance and power consumption of core server parts has continued to increase, and reports on the use of liquid immersion cooling and its cooling capacity are constantly being published. The design of core parts, such as the immersion tanks and the device for environmental control devices, has gradually matured and is developing towards a modularized design.

Two-phase Liquid Immersion Cooling System

Examples of applications that have been published show that two-phase liquid immersion cooling systems are able to support extremely high density compute modules. Using the volume of a standard rack for estimation, a two-phase liquid immersion cooling system can support the operations of equipment using up to 250 kW of power. Hence, the Open Compute Plan (OCP) and operators of large data centers, such as public cloud providers, focus on the development of two-phase liquid immersion cooling structures and standardized products.

Single-phase Liquid Immersion Cooling System

When comparing the same volume of a standard rack for calculation, a single-phase liquid immersion cooling system can handle up to 200 kW. As a matter of fact, when the liquid cooling system is connected to the building's cooling tower for heat exchange, it will gain approximately 100 kW of heat exchange performance.

KW Per Rack Potential

Reliability and Deployment Flexibility

Whether it may be two-phase or single-phase liquid immersion cooling, both were developed in response to ICT developments to increase the reliability of IT devices while having flexibility in deployment. One of the reasons is that liquid immersion cooling does not require the support of complex infrastructure that is needed by conventional air cooling equipment. It reduces the amount of electrical equipment that needs to be integrated, and eliminates the use of parts that may cause instability, such as fans. When comparing the reliability data of liquid immersion cooling with conventional air-cooled data centers, there is a significant increase in mean time between failures (MTBF) for immersion cooling.

Two-phase Liquid Immersion Cooling System

The flow of liquid in two-phase liquid immersion cooling systems relies on phase changes at points where heat is generated by the server. The system does not require any fans and the environment is controlled in a stable sealed system. There is no vibration from fans to cause fatigue in parts, so there is no need for additional maintenance of electrical equipment. There is also no need to consider the infrastructure where the system will be deployed. The GIGABYTE Server Management (GSM) software package, which supports Redfish API, can be used to provide convenient machine room management for hybrid IT and Software Defined Data Center, making two-phase liquid immersion cooling systems the best option for deploying edge computing stations in remote areas.

Single-phase Liquid Immersion Cooling System

Single-phase and two-phase liquid immersion cooling systems are similar in that IT equipment is installed in a tank with a non-conductive liquid medium. The heat conducting liquid directly comes in contact with heat generating points of the server, and the pump in the medium cooling device pumps the heat conducting liquid into the cooling device for heat exchange. The system can also use the building's cold-water supply system to improve the efficiency of heat dissipation. Even though single-phase liquid immersion cooling systems have an extra pump, the main purpose is to make the medium flow to help remove heat from components just like how high-speed fans do in an all air-cooled server. Hence, it provides a computing environment that is far more stable and reliable than an all air-cooled system.

Conclusion

Liquid immersion cooling has a solid theoretical foundation that demonstrates that it can help systems achieve high performance and do so while running cool. Two-phase and single-phase liquid immersion cooling systems each have their own advantages. The cost of these two types of data centers is relatively high and a high level of customization is often needed to meet the many different application requirements. However, GIGABYTE has gained the ability to provide servers suitable for liquid immersion cooling through leading experiments and projects, and also conducted experiments on environmental controls and worked with different equipment integration partners to find ways to lower the cost. GIGABYTE is able to assist the operation and development for companies, government, education, and telecom operators, allowing customers to easily face the different challenges in future operations and deployment.

Two-phase and Single-phase Liquid Immersion Cooling