Breaking Through the AI Computing Power Bottleneck: Liquid Cooling and HBM Reshape Data Centers

Overview

This report focuses on two pivotal transformations in data center infrastructure driven by the AI wave: the technological evolution and market surge of high-bandwidth memory (HBM), and the imminent cooling‑technology revolution—shifting from air cooling to liquid cooling—spurred by soaring power densities. The report highlights that demand from generative AI and large language models is not only fueling a cyclical recovery in the storage market but also fundamentally reshaping hardware architectures, rendering HBM and advanced packaging increasingly scarce while compelling data centers to adopt liquid‑cooling solutions to address the thermal challenges posed by kilowatt‑class chips.

Core views

Demand Side: AI is the primary driver behind the memory market’s recovery. After hitting a trough in 2023, the traditional DRAM market is expected to stage a robust rebound in 2024–2025, largely fueled by the soaring demand for memory capacity and bandwidth from AI servers. AI servers require significantly more DRAM bits than conventional servers, and the share of HBM in this segment is rapidly expanding. Although HBM currently accounts for only a small fraction of total DRAM shipments, its premium price tag makes it the fastest‑growing sub‑segment. Technology Side: HBM achieves ultra‑high bandwidth through vertical stacking (TSV technology), delivering performance far superior to GDDR6 and DDR5. From HBM1 to the forthcoming HBM4, the number of stacked layers continues to increase—from four to sixteen or even more—while custom logic substrates are being introduced. Meanwhile, data center cooling is approaching physical limits; when a single chip consumes over 1,000 W, conventional air cooling can no longer effectively dissipate hotspots. Liquid cooling—both cold plate and immersion types—has emerged as the inevitable choice due to its high thermal capacity and superior heat‑transfer efficiency. Cold plate systems are relatively mature and well suited for retrofitting, whereas immersion cooling offers exceptionally high efficiency but entails more complex maintenance. As a result, PUE metrics could potentially drop from above 1.5 to around 1.1. Supply Side: HBM production relies on advanced processes such as TSV etching and micro‑bump bonding, with yield and capacity serving as key competitive differentiators. SK Hynix, benefiting from early strategic positioning and an exclusive supply agreement with NVIDIA, currently commands the largest market share. Samsung and Micron are working hard to close the gap. In the packaging stage, TSMC’s CoWoS capacity remains severely constrained, and the resulting supply‑demand imbalance has prompted continuous capacity expansion, making it one of the major bottlenecks limiting AI GPU shipments. Costs and Ecosystem: While liquid‑cooling systems entail higher upfront capital expenditures (CapEx) compared to air cooling, their significantly lower fan power consumption and improved cooling efficiency lead to substantial reductions in operating expenses (OpEx), ultimately delivering a more favorable total cost of ownership (TCO) over the long term. Regulatory policies—such as China’s PUE restrictions and the EU’s Energy Efficiency Directive—are further accelerating this transition. Nevertheless, short‑term challenges persist, including the lack of industry standards, supply‑chain disruptions stemming from 3M’s withdrawal from PFAS‑based fluid production, and a growing skills gap among technical personnel.

Analysis framework

The report employs a comprehensive framework that integrates “technological evolution, supply‑demand analysis, and industry‑chain mapping.” First, by comparing the technical specifications of HBM and conventional memory—such as bandwidth and power consumption—and contrasting the thermodynamic characteristics of liquid cooling versus air cooling, the report establishes the inevitability of technological substitution. Second, it leverages market‑size forecast charts to illustrate the explosive demand trajectory driven by AI, while incorporating vendor‑share data to assess the competitive landscape. Finally, by tracing the entire value chain—from chip design and manufacturing to packaging and cooling infrastructure—the report identifies the segments poised to benefit—such as HBM manufacturers, advanced packaging facilities, and liquid‑cooling component suppliers—and pinpoints potential bottlenecks, including CoWoS production capacity and the supply of dielectric fluids.

Methodology notes

• Industry/ Sector Analysis FrameworkSupply-and-Demand Framework

  The explosive growth in AI computing demand has led to shortages in HBM and advanced packaging capacity.

  The research report analyzes the gap between the incremental demand driven by AI servers and the existing semiconductor manufacturing and packaging capacity, explaining why HBM and CoWoS have become industry bottlenecks. Based on this analysis, it infers the bargaining power and expansion incentives of the relevant manufacturers.

• Industry/ Sector Analysis FrameworkTechnology Life Cycle

  The paradigm shift in cooling technology from air cooling to liquid cooling

  The research report notes that when chip power consumption exceeds the physical limit (exceeding 1,000 W), conventional air-cooling technology becomes ineffective, and liquid cooling technology enters a phase of rapid market penetration. This aligns with the characteristic of a technology’s S-curve—transitioning from the introduction phase to the growth phase—thereby underscoring the high growth potential of the liquid-cooling market.

• Company Fundamentals and Financial FrameworkFree cash flow analysis

  TCO (Total Cost of Ownership) Analysis

  The research report compares the CapEx (capital expenditures) and OpEx (operating expenses) of liquid cooling and air cooling, demonstrating that although liquid cooling entails higher upfront investment, its substantial long-term energy‑saving benefits make it economically viable for large‑scale adoption. This constitutes the core financial rationale for evaluating infrastructure investment decisions.

Asset mapping & comparison

Structured mapping from thesis to named assets (strengths, weaknesses, peers, risks).

• SK Hynix

  HBM market leader and a major supplier to NVIDIA.

  Strengths

  Technologically advanced, with the largest market share and an exclusive supply relationship.

  Comparison

  Leading Samsung and Micron

  Risks

  Competitors’ capacity ramp-up could erode market share.

• Taiwan Semiconductor Manufacturing Company (TSMC)

  A core provider of advanced packaging (CoWoS)

  Strengths

  Monopolistic production capacity and high technological barriers

  Weaknesses

  Capacity bottlenecks are constraining downstream shipments.

  Comparison

  It holds an absolute leading position in the advanced packaging sector.

  Risks

  Production expansion has lagged behind demand growth.

• Vertiv / Schneider Electric

  Data center cooling infrastructure provider

  Strengths

  Providing a comprehensive liquid cooling solution.

  Comparison

  Industry leader, benefiting from the rising penetration of liquid cooling.

  Risks

  The slow pace of standardization is hindering large-scale deployment.

• 3M

  Dielectric fluid suppliers (subject to regulatory impacts)

  Strengths

  Existing technological expertise

  Weaknesses

  Withdrawal from PFAS production poses risks of supply disruptions.

  Comparison

  Facing competition from alternative suppliers such as Shell and ExxonMobil.

  Risks

  Stricter environmental regulations have led to a contraction in business operations.

Key data

• AI Chip Power Consumption Threshold>1000WBeyond this power threshold, conventional air cooling struggles to dissipate heat effectively, necessitating a transition to liquid cooling.
• Data Center PUE Target1.03-1.1Liquid cooling technology can reduce the PUE from the conventional level of over 1.5 to this range.
• Rack Power Density Trends<5kW → >100kWAI-intensive computing is driving an exponential increase in the power demand per server rack.
• HBM market growthExponential growthFrom 2023 to 2026, HBM’s value share in the DRAM market will expand rapidly.

Impact & implications

For the semiconductor industry, HBM and advanced packaging will become the primary focus of capital expenditure in the coming years, and companies that possess TSV technology and CoWoS production capacity are poised to generate substantial excess returns. For data center operators, liquid cooling is no longer an optional solution but a mandatory requirement; they must proactively plan for infrastructure upgrades or the construction of new liquid-cooling data centers. Investors should shift their focus from purely GPU chips to the entire “computing power – storage – cooling” closed-loop supply chain, with particular attention to companies that hold monopolistic or leading positions in HBM manufacturing, advanced packaging equipment, and liquid-cooling components.

Risks

• 3M’s withdrawal from PFAS production could lead to a supply shortage of two-phase immersion cooling fluids.
• The liquid-cooling industry lacks unified connector and fluid standards, which hinders large-scale deployment.
• Data center technicians lack the specialized skills required to handle liquid-cooling systems.
• Fluid leakage can damage hardware, although modern designs have significantly reduced this risk.

What to watch

• TSMC’s CoWoS Capacity Expansion Progress and Utilization Rate
• The formulation of the HBM4 technical standard and its mass-production timeline
• R&D Progress in Non-PFAS Dielectric Fluids and the Implementation of Substitution Solutions
• The further tightening of policies by governments worldwide on data center PUE limits