The HBM Base Die Will Evolve from an Interface Layer into an AI System Platform for Control, Expansion, and Near-Memory Computing

Overview

The report discusses how the role of the HBM base die is changing amid AI-driven computing demand. Its core view is that bandwidth, power, area, and capacity constraints will drive a transition from standard HBM to custom HBM using advanced logic processes, following a three-stage evolution of “xPU area reclamation—functional expansion—3D integration.”

Core views

The report first distinguishes between the core die and the base die in an HBM stack: the core die contains DRAM cells, core logic, and TSVs, while the base die provides the PHY-to-TSV path for communication with the compute die, as well as core-die test functions such as DA, MBIST, and IEEE 1500. As higher bandwidth becomes the primary driver of successive HBM generations, the conventional architecture faces two direct bottlenecks: first, limitations on TSV count and pitch in the base die and core-die stack; and second, limitations on the I/O count and speed of the base-die PHY. The report therefore argues that rapid bandwidth scaling requires fundamental changes to the base die. Process evolution provides the foundation for this transformation. The roadmap presented in the report shows the base die moving from 2*nm to 1*nm between HBM2 and HBM3E, while HBM4 and HBM4E shift to a 4nm logic process. Over the same period, the xPU SoC evolves from 16n/12nm in 2016 to 3nm for HBM4 and below 3nm for HBM4E. Samsung uses D1c DRAM and a 4nm logic process in HBM4. The report emphasizes that although energy efficiency per unit continues to improve, total MPGA power consumption continues to rise. Therefore, beginning with HBM4, using advanced logic processes for the base die is critical to containing power growth. It also reduces effective area and narrows the process gap between the base die and the xPU SoC. This is defined as the beginning of true integration between DRAM and advanced logic. On this basis, the report distinguishes standard HBM from custom HBM. The base die in standard HBM primarily provides basic data and test paths, with most of its area passively used for routing. Custom HBM, by contrast, shares a standard core-die stack while using advanced logic processes to customize the base die and give it SoC-like functionality. The motivation is that conventional methods of scaling xPUs are approaching physical limits: process scaling is slowing, monolithic dies are approaching the reticle-size limit, and interposer dimensions in multi-chiplet solutions are also nearing their limits. Because the base die already uses an advanced logic process and has substantial available area, the report proposes migrating some xPU functions into the base die. The first stage focuses on reclaiming xPU area and optimizing the interface. The conventional HBM PHY occupies the largest functional region on the base die, and from HBM2 to standard HBM4, its footprint and channel length continued to increase to support higher bandwidth. Custom HBM replaces the conventional HBM PHY with a D2D interface implemented using an advanced logic process. A smaller interface footprint and shorter channels can reduce pJ/b, improve energy efficiency, and free valuable silicon area for xPU expansion. The report identifies HBM4 through HBM5 as an inflection period in which advanced logic processes significantly reduce PHY and D2D area. However, this area reduction also increases power density and creates thermal hot spots. The comparison in the report shows that sHBM4E has an I/O speed of 14Gbps and a power density of 0.5W/mm², while sHBM5 has an I/O speed exceeding 28Gbps, approximately 2 times that of sHBM4E, and a power density exceeding 2.0W/mm². To address this issue, the report proposes an HPB thermal-path block based on Samsung's cHBM4 experience. When HPB covers more than 50% of the PHY region, peak temperature can be reduced by more than 35%. The first stage also includes migrating the memory controller from the xPU SoC to the custom HBM base die. The report considers the memory controller the primary migration candidate because this arrangement can reclaim xPU area while keeping the thermal impact of the added controller manageable. Conventional HBM memory controllers are currently being ported to cHBM. Once the controller is positioned closer to memory, unused base-die space can also be used for SRAM repair resources that provide fine-grained address decoding and redirection for failed cells and share repair capacity across channels. Compared with the limited and inflexible row and column repair resources in the core die, the base-die SRAM solution provides greater capacity and higher configuration flexibility. The second stage uses the base-die area that remains available after memory-controller migration to add functions. The first category is RAS and testing: cHBM can natively integrate temperature, voltage, process, and aging sensors to provide real-time, high-speed telemetry. On-chip self-test mechanisms such as OD-ATE and PGEN can expand test coverage and improve manufacturing yield. The second category is capacity expansion. The report notes that AI model context windows are growing at 30 times per year, significantly increasing KV-cache requirements, while the storage and retrieval of long-term memory remain major bottlenecks for next-generation AI models. AI SoCs therefore urgently require greater memory capacity in addition to bandwidth. The base die can use its outer edge to connect directly to external memory and integrate dedicated PHYs and controllers. The report argues that this approach can deliver significantly higher bandwidth and lower latency than conventional PCIe expansion. The second stage can also migrate some xPU computation to processing units in the base die. Because data is processed closer to memory, D2D bandwidth requirements, data movement, power consumption, and thermal burden can be reduced. In advanced HBM solutions within 2.5D systems, base-die processing units can also significantly reduce data movement across the interposer, thereby improving system-level energy efficiency. However, this approach is constrained by the base die's limited silicon area and by the increase in local thermal density caused by processing units. Functional scale must therefore balance computing benefits, area, and heat dissipation. The third stage is the true 3D integration represented by zHBM. The report argues that AI memory architectures are rapidly moving toward tight coupling, with the key objective in AGI and inference scenarios being to increase bandwidth and TPS under strict power constraints. zHBM vertically integrates the xPU with the core-die stack and eliminates the 2.5D interposer. Distributed I/O shortens in-stack data paths while removing two-dimensional interfaces such as conventional HBM PHYs or D2D interfaces. The primary benefit is a substantial reduction in I/O power consumption: removing data alignment and DQ I/O from SERDES can eliminate unnecessary power overhead and convert the saved power and thermal headroom into greater system computing performance. The report uses “1 GPU plus 4 HBM4E” and “1 GPU plus 4 zHBM” as the basis for a system-architecture comparison but does not provide specific quantitative results. Implementing zHBM still depends on critical process and co-design capabilities. Wafer-to-wafer bonding and hybrid copper bonding must provide ultra-high I/O density. The process shown in the report includes wafer-level bonding of 4 core-die layers with an xPU or intermediate layer. Meanwhile, the SoC and DRAM must be co-architected using a unified design and verification flow. Overall, the report views the advanced-logic base die as the pivotal element of the evolution: the first stage reclaims xPU area through a compact D2D PHY and controller migration; the second stage adds telemetry, testing, capacity expansion, and processing units; and the third stage uses zHBM to eliminate two-dimensional and 2.5D interfaces, achieving direct vertical integration of xPU logic with the DRAM stack and maximum energy efficiency.

Analysis framework

The report begins with the existing division of responsibilities between the HBM core die and base die and identifies scaling bottlenecks involving TSVs, PHYs, power consumption, and package dimensions. It then uses generational changes in processes and interfaces from HBM2 through HBM4E to demonstrate the necessity of adopting advanced logic processes. Next, under the constraints of area, channel length, energy per bit, power density, and thermal hot spots, it sequentially evaluates D2D interfaces, HPB, memory-controller migration, SRAM repair, RAS and testing, capacity expansion, near-memory computing, and 3D integration, ultimately forming a three-stage architectural roadmap.

Methodology notes

• (Out-of-Vocabulary Method)

  Three-Stage Architectural Evolution Roadmap

  The report divides base-die evolution into three stages—xPU area reclamation, additional functional expansion, and true 3D integration—to explain the implementation sequence and dependencies among the technologies.

• (Out-of-Vocabulary Method)

  Comparison of HBM Generational Parameters

  The report compares the years and processes of HBM2 through HBM4E, as well as the I/O speeds and power densities of sHBM4E and sHBM5, to identify the inflection created by advanced logic processes and the resulting new thermal constraints.

• (Out-of-Vocabulary Method)

  System-Level Trade-Off among Power, Area, Bandwidth, and Heat Dissipation

  Rather than examining only the memory chip itself, the report simultaneously evaluates how shorter interfaces, functional migration, and vertical integration affect xPU area, data movement, I/O power, local thermal density, and headroom for system performance.

Key data

• HBM Process Evolution Timeline2016 HBM2; 2019 HBM2E; 2021 HBM3; 2023 HBM3E; 2026 HBM4; 2027 HBM4EHBM generations and years presented in the report
• HBM4/4E Base-Die Process4nmThe base-die process previously evolved from 2*nm to 1*nm between HBM2 and HBM3E
• HBM4 Core-Die ProcessD1c/1*nmThe report states that Samsung uses D1c DRAM and a 4nm logic base die in HBM4
• xPU SoC Process Evolution16n/12nm in 2016; 8n/4nm in 2019; 4nm in 2021—2023; 3nm in 2026; <3nm in 2027Corresponding to the HBM2 through HBM4E roadmap
• sHBM4E I/O Speed14GbpsBaseline for the thermal-density comparison with sHBM5
• sHBM5 I/O Speed>28GbpsApproximately 2 times that of sHBM4E
• PHY Power Density0.5W/mm² for sHBM4E; >2.0W/mm² for sHBM5Thermal hot-spot risk rises as the PHY shrinks and accelerates
• HPB Peak Temperature Reduction>35%When PHY coverage is >50%
• AI Context-Window Growth Rate30 times/yearUsed in the report to illustrate the rapid growth in KV-cache and memory-capacity requirements
• zHBM System Comparison Configuration1 GPU + 4 HBM4E, compared with 1 GPU + 4 zHBMThe report does not provide specific quantitative results for this comparison

Impact & implications

The report argues that the dimensions of HBM competition will expand beyond bandwidth alone to include interface energy efficiency, xPU area utilization, RAS, testing, capacity expansion, and near-memory computing. Advanced-logic base dies allow more system functions to be deployed closer to DRAM, while zHBM further broadens the scope of optimization from an individual memory device to the joint architectural level of the xPU, DRAM, and packaging. Accordingly, heat dissipation, ultra-high-density bonding, and unified design and verification capabilities will become critical constraints on realizing these benefits.

Risks

• Physical limitations involving TSV count and pitch, PHY I/O count and speed, and reticle and interposer dimensions may constrain further bandwidth scaling in conventional HBM.
• Reducing PHY area increases power density and creates thermal hot spots; the report shows that sHBM5 PHY power density exceeds 2.0W/mm².
• Deploying processing units in the base die is subject to the dual constraints of limited available silicon area and increased local thermal density.
• The implementation of zHBM depends on wafer-to-wafer bonding, hybrid copper bonding, and a unified SoC—DRAM design and verification flow.

What to watch

• Monitor whether HPB can achieve its target of more than 50% PHY coverage and sustain a peak temperature reduction exceeding 35% at HBM5 I/O speeds above 28Gbps.
• Monitor progress in porting conventional HBM memory controllers to the cHBM base die.
• Monitor the practical integration of base-die external memory expansion, on-chip RAS sensors, self-test, and processing units.
• Monitor how WoW, hybrid copper bonding, and a unified SoC—DRAM design and verification flow support zHBM.