Dense AI systems put more heat into less space. Liquid cooling moves that heat closer to the source, but it also changes the rack, the building and the maintenance plan.
Artificial-intelligence servers are concentrating more computing power into each rack than conventional data centres were designed to cool.
Air remains effective for many systems. At very high rack densities, moving enough air becomes difficult, noisy and energy intensive. Liquid cooling moves heat away closer to the chips and can support far denser hardware.
That is why liquid cooling is moving from a specialist high-performance-computing technique towards mainstream AI infrastructure.
AI changed the amount of heat in one place
A traditional server rack might contain many relatively modest systems. An AI rack can contain tightly connected accelerators, CPUs, memory, networking and power equipment operating continuously near high utilisation. The total heat is not the only problem. It is concentrated in a small physical volume. Open Compute Project work identifies increased chip power, system density and operating efficiency as drivers for liquid cooling. Its current cooling work covers cold plates, coolant distribution units, immersion, rear-door heat exchangers and heat reuse.
The building and computer can no longer be designed separately.
Why liquid removes heat more effectively
Cooling transfers heat from the electronic components into another medium and moves it elsewhere. Air has several advantages:
- it is already present
- leaks are not a concern
- systems are familiar
- and ordinary servers can be serviced easily.
Its weakness is low heat capacity and density. Large fans and volumes of conditioned air are needed to move substantial heat. Liquid can carry far more heat through a much smaller flow. The liquid does not normally touch exposed electronics in a direct-to-chip system. It passes through sealed cold plates attached to the hottest components. That brings the cooling path closer to where the heat is produced.
Direct-to-chip cooling
Direct-to-chip is the most visible AI cooling approach. A cold plate sits on components such as the GPU or CPU. Coolant flows through channels inside it and absorbs heat. The heated liquid travels to a coolant distribution unit, or CDU. The CDU controls flow, pressure and heat exchange between the technology-cooling loop and the facility’s wider cooling system. A simplified path is:
- chip produces heat
- cold plate absorbs it
- coolant carries it out of the server
- CDU transfers it into the facility loop
- the facility rejects or reuses the heat.
Fans may still cool components not connected to cold plates. “Liquid-cooled” does not necessarily mean an entirely fanless rack.
Advantages
- high heat removal close to the source
- greater rack density
- reduced dependence on moving huge air volumes
- potential for warm-water operation
- and easier heat recovery.
Challenges
- pumps, pipes, connectors and monitoring
- leak prevention and detection
- facility changes
- coolant compatibility
- maintenance training
- and service procedures for liquid-connected hardware.
The cooling system becomes part of the server’s availability design.
Immersion cooling
Immersion cooling places hardware in a dielectric fluid that does not conduct electricity in the way ordinary water does.
Heat transfers directly from components into the surrounding fluid.
There are two broad approaches.
Single-phase immersion
The fluid remains liquid and circulates through a heat exchanger.
Two-phase immersion
The fluid boils at a controlled temperature, then condenses and returns to the tank. Immersion can cool a large portion of the system and remove the need for many server fans. It also changes almost everything about installation and servicing. Technicians work with tanks rather than conventional racks. Components, cables, labels and materials need to be compatible with the fluid. Hardware replacement and contamination control require different processes. Immersion is powerful, but it is not automatically the simplest option.
Warm-water cooling sounds wrong, but can save energy
Cooling liquid does not need to feel cold to a person. The relevant question is whether it remains cool enough to absorb heat from the chips. Some current AI platforms are designed around relatively warm supply temperatures. NVIDIA, for example, describes warm-water direct liquid cooling at up to 45°C for its Vera Rubin rack-scale platform. Warmer cooling water can reduce or eliminate the need for compressor-based chilling in suitable climates because outside air can help reject the heat.
It also makes the waste heat more useful. A lukewarm air stream is difficult to reuse. A controlled warm-water loop can contribute to:
- building heating
- district heat networks
- domestic hot water preheating
- or industrial processes.
Heat reuse still depends on location, temperature, infrastructure and a nearby customer for the heat.
Does liquid cooling save water?
Not automatically. A liquid-cooled server loop can be closed, meaning the same coolant circulates repeatedly. The facility may still use evaporative systems elsewhere to reject heat. Another site may use dry coolers and little operational water while consuming more electricity under certain conditions. The correct comparison includes: cooling-loop design, local climate, water source, heat-rejection system, electricity and heat reuse. “Liquid cooling” and “high water consumption” are not synonyms. Neither is liquid cooling automatically water-free.
Why air cooling will not disappear
Most computing hardware does not need rack-scale liquid infrastructure. Air cooling remains attractive for: lower-density racks, existing facilities, general storage and networking, edge sites and systems where simplicity matters more than maximum density. Hybrid designs are likely to remain common. GPUs and CPUs may use cold plates, while lower-power components and the room still use air. Rear-door heat exchangers can capture heat from air-cooled racks without redesigning every server. The move is not from all air to all liquid. It is towards using liquid where heat density makes air inefficient or impractical.
The operational risks are real
Liquid cooling creates new failure modes: leaks, pump failure, blocked flow, corrosion, incompatible materials, trapped air, condensation in poorly controlled systems and mistakes during servicing. Good designs use: monitored quick-disconnect fittings, leak detection, redundant pumps, pressure control, compatible coolant and materials and clear maintenance processes. The standard is not “liquid never leaks”. It is that the system detects and controls failures without damaging expensive equipment or interrupting the wider service.
It changes the used-hardware market too
Liquid-cooled datacentre hardware can be less straightforward to redeploy. A used accelerator may depend on: a specific cold plate, manifold, rack, coolant specification, power architecture and high-speed interconnect. A conventional PCIe card can often move between compatible servers. A rack-scale liquid-cooled module may be inseparable from its platform economics. This means model identification and system context become more important in resale.
The valuable object may be the complete tray or rack configuration, not the accelerator alone.
The deeper change
Liquid cooling is not becoming popular because water cooling looks advanced. It is becoming necessary because AI systems are placing more power and heat into less space. The cooling system now affects:
- how much compute fits in a building
- where a datacentre can be located
- how efficiently it operates
- how hardware is serviced
- and whether waste heat can be reused.
The next generation of AI performance depends on more than chips. It depends on whether the building can power and cool them. GPUsed handles enterprise, workstation and datacentre hardware, including specialist AI accelerators and GPUs.