Data Centers in the AI Era: How to Cool the Scorching Heat of GPUs

In the AI era, tens of thousands of GPUs run non-stop, putting data centers under the dual burden of power scarcity and heat generation. KakaoBank's infrastructure team visited overseas data centers to benchmark how to introduce liquid cooling (water cooling) into existing infrastructure.

Traditional Air Cooling uses CRAC/CRAH to circulate cold air and cool server heat, but as power density per rack rises, its limitations become clear. Once density exceeds 10kW per rack, hotspots emerge, and expanding cooling equipment increases both space and cost, ultimately worsening PUE (Power Usage Effectiveness). Typical air-cooled data centers often have a PUE exceeding 1.5.

In contrast, Liquid Cooling, particularly the D2C (Direct-to-Chip) method, delivers coolant directly to heat sources like GPUs and CPUs, removing heat far more efficiently. Because liquid has a greater heat capacity than air, it can lower PUE to the 1.1–1.3 range and handle ultra-high-density rack environments exceeding 100kW.

Cooling technology has evolved to move progressively closer to the chip—from the Room level to the Row level, and finally right above the Chip. This includes room-level CRAC/CRAH, In-Row Cooling placed between racks, RDHx with heat exchangers attached to rack rear doors, D2C with cold plates pressed against chips, and Immersion Cooling, which submerges entire servers in dielectric fluid. The core principle throughout is reducing the distance to the heat source.

AI data centers require more than just swapping cooling equipment. Power systems must be redesigned starting from transformers and switchgear, water introduces new risks like leaks and WUE (Water Usage Effectiveness) management, and reliability must be strengthened beyond the conventional 2N to N+M or triple-redundancy levels. Ultimately, AI infrastructure isn't about standardized design—it requires custom design tailored to workloads and environments, along with integrated operational capabilities that consider power, cooling, and IT together.