AMD’s Competitive Boundaries Are Expanding from Individual Chips to Packaging, Racks, and Entire AI Clusters

Overview

The report examines the evolution of AMD’s data-center and AI architecture. Its core conclusion is that AMD’s strategy is no longer confined to competition in CPU and GPU chips. Instead, it uses heterogeneous integration to reduce the cost of moving data while progressively expanding the scope of system design to packages, nodes, racks, and clusters. Accordingly, the report argues that AMD’s competitiveness should be assessed not by core count or GPU FLOPS, but by useful compute, bandwidth, energy efficiency, and coordination efficiency at the rack and AI-factory levels.

Core views

First, the report views custom ASICs as one of AMD’s important future opportunities. Large cloud service providers are adopting both general-purpose GPUs and internally developed ASICs optimized for specific AI workloads. The two architectures do not necessarily replace each other and may coexist over the long term. AMD’s potential advantage lies not merely in designing an ASIC, but in combining customers’ proprietary accelerator IP with its CPUs, I/O, memory controllers, die-to-die interconnects, and packaging technologies. AMD already has a modular foundation comprising CPU compute dies, GPU compute dies, I/O dies, HBM, Infinity Fabric, and chiplet packaging. Custom ASICs can therefore be understood as a natural extension of its CPU/GPU heterogeneous integration capabilities to third-party accelerator IP. Placing compute units closer together can reduce SerDes power consumption, signal loss, and latency associated with conventional PCIe and PCB connections while increasing bandwidth density and lowering energy consumption per bit. Second, EPYC’s core advantage is shifting from “more cores” to its chiplet architecture. The report summarizes its structure as “Compute Die → Chiplet → I/O Die → Package → Server Platform”: compute dies provide the main processing capacity, while the I/O die integrates memory controllers, PCIe/CXL, Fabric connections, and external I/O. Compared with large monolithic CPUs, chiplets offer three major advantages: smaller dies reduce the risk that a single defect causes the entire chip to be discarded, improving wafer utilization and manufacturing economics; compute and I/O functions can use different process nodes to optimize performance, cost, and yield separately; and the same compute die can serve multiple server markets through different core counts, TDPs, and memory configurations. Consequently, the next stage of EPYC competition is closer to “Compute Density × Memory Bandwidth × I/O Bandwidth × Packaging Efficiency × Power Efficiency,” rather than merely core count. As core density increases, memory and I/O will become new EPYC bottlenecks. If memory bandwidth does not expand in parallel, the marginal benefit of adding more cores will decline, and upgrades will therefore focus more heavily on the I/O die. This die must simultaneously support DDR memory interfaces, PCIe, CXL, chiplet interconnects, accelerator connections, and network I/O, transforming it from an auxiliary component into a critical part of server CPU architecture. In AI servers, the CPU must also continuously exchange data with GPUs, NICs, DPUs, storage, CXL memory, and other accelerators. The report therefore recommends expanding the CPU performance framework from “Core Count × IPC × Frequency” to “Compute × Memory × I/O × Interconnect ÷ Power.” The relationship between CPUs and GPUs is also shifting from a host–accelerator hierarchy toward platform-level coordination. Large AI models generate workloads involving data preprocessing, memory management, KV Cache, networking, storage access, and distributed computing, making CPUs an important component of AI infrastructure once again. AMD’s future priority is therefore not simply to increase the number of GPUs, but to improve coordination among CPUs, GPUs, and memory. The architectural evolution moves from “CPU → PCIe → GPU” to “CPU ↔ High-Speed Fabric ↔ GPU ↔ HBM,” ultimately forming a complete “CPU + GPU + HBM + NIC + Switch + Optical I/O” system, which is also one of the foundational principles of Helios. Helios is significant because it moves AMD from being a component supplier toward becoming a participant in rack-scale computing platforms. Historically, AMD primarily supplied EPYC CPUs and Instinct GPUs, with server OEMs and ODMs responsible for system integration. Rack-scale architecture, however, requires AMD to participate in the co-design of CPUs, GPUs, HBM, interconnects, switches, NICs, power delivery, liquid cooling, mechanical structures, and software. The basic design unit of AI hardware is expanding progressively from “Die → Package → Board → Tray → Rack → Cluster,” with the GPU becoming a compute unit within a larger-scale “AI factory.” Because more powerful accelerators also consume more power, system performance is increasingly constrained by memory bandwidth, network bandwidth, power delivery, cooling, and interconnect efficiency. AMD’s competitiveness must therefore be measured more at the rack level than solely by GPU FLOPS. The scaling approach for Instinct GPUs is changing as well. Semiconductor process-node shrinkage alone is no longer sufficient to sustain AI performance growth, and the architecture is shifting toward “Multiple Compute Dies + I/O Dies + HBM + Advanced Packaging.” AMD can separate different functions across multiple dies and reconnect them through high-speed die-to-die interfaces to create larger accelerators. This approach reduces the yield risk associated with extremely large monolithic dies and allows different functions to use different process nodes, but at the cost of significantly greater packaging complexity. Advanced packaging is evolving from a back-end manufacturing step into part of the GPU architecture itself. Final shippable yield must account simultaneously for “KGD Yield × Bonding Yield × Interconnect Yield × HBM Yield × Packaging Yield × Final Test Yield”; a problem at any stage will reduce the final number of deliverable accelerators. Within this system, SoIC and CoWoS address different dimensions of scaling. SoIC mainly enables Z-axis vertical integration through 3D stacking and hybrid bonding, shortening the distance between dies, increasing interconnect and bandwidth density, and reducing communication power consumption. CoWoS primarily enables X-Y planar scaling through interposers, RDL, and advanced substrates, integrating GPU compute dies, I/O dies, and multiple HBM stacks into a large package. Together, they allow accelerators to progressively overcome the physical limits of a single reticle, a single die, and conventional packaging. The competitive focus in advanced packaging is also shifting from nominal capacity to “Good Package Output.” Large AI accelerators simultaneously contain multiple logic dies, multiple HBM stacks, interposer or RDL structures, advanced substrates, and numerous micro-bump or hybrid-bonding interfaces. A defect in any critical die or interface can significantly lower package yield and cause substantial economic losses. The report therefore locates competitiveness in the coordinated capabilities of wafer fabrication, advanced packaging, 3D integration, and testing, rather than merely comparing CoWoS capacity. Suppliers capable of coordinating these processes will become increasingly important in the AI accelerator value chain. The actual performance of next-generation Instinct products also cannot be explained simply by how many times faster a single GPU is than its predecessor. AI training and inference depend on complete systems, and the report focuses on the combined effects of compute performance, HBM bandwidth, Scale-Up bandwidth, cluster utilization, Scale-Out networking, and software efficiency. Even if a single GPU’s compute performance increases substantially, insufficient inter-GPU communication bandwidth will still constrain cluster performance. The future competitive scope of Instinct will therefore extend from GPU silicon to the Helios platform, while the product form will evolve from a single GPU to a GPU cluster and ultimately into an AI factory. As GPU clusters expand, networking becomes the next major bottleneck. Higher-speed SerDes exacerbate the insertion loss, power consumption, and signal-integrity pressures faced by PCB traces, connectors, package routing, and Retimers. The conventional link “Switch ASIC → Package → PCB → Connector → Optical Module” contains a relatively long high-speed electrical channel. CPO moves the optical engine close to the switch ASIC, creating a structure closer to “Switch ASIC → Optical Engine → Fiber.” This shortens the electrical path while reducing insertion loss, SerDes power consumption, Retimer requirements, and PCB signal-integrity pressure. The report emphasizes that CPO does not simply move the optical module into the package; it redesigns the I/O architecture of AI clusters. Taken together, EPYC, Instinct, SoIC, CoWoS, Helios, and CPO all address the same issue—data movement. CPU chiplets handle data transfer between compute dies, HBM connects compute and memory, SoIC handles vertical die interconnects, CoWoS handles package-level transfer between GPUs and HBM, high-speed Fabric supports CPU–GPU and GPU–GPU communication, Helios coordinates rack-scale computing, and CPO addresses inter-rack and cluster-scale transmission. The AI computing bottleneck is gradually shifting from insufficient compute to an inability to move data efficiently enough to keep computing resources fully utilized. Accordingly, AMD’s scaling roadmap is summarized as “Transistor Scaling → Chiplet Scaling → Package Scaling → Rack Scaling → Cluster Scaling”: EPYC handles CPU scaling, Instinct handles accelerator scaling, SoIC and CoWoS handle package scaling, Helios handles rack scaling, and networking and CPO extend the architecture to the cluster level. AMD’s competition with NVIDIA, Intel, and cloud service providers’ internally developed ASICs should no longer be reduced to CPU versus CPU or GPU versus GPU. Instead, the comparison should focus on who can deploy more useful AI compute within a rack or an entire data center while reducing power consumption, latency, and data-movement costs. The report ultimately concludes that AMD’s most important long-term technology asset is the heterogeneous computing platform formed by “EPYC + Instinct + Chiplets + HBM + Advanced Packaging + Networking + Helios + CPO,” whose objective is shifting from “More Compute per Chip” to “More Useful Compute per AI Factory.”

Analysis framework

The report advances its analysis hierarchically from chips to clusters. It first discusses how custom ASICs could extend AMD’s heterogeneous integration capabilities, then examines the EPYC chiplet architecture, memory and I/O bottlenecks, and CPU–GPU coordination. It subsequently broadens the scope to Helios rack design, Instinct’s multi-die architecture, SoIC/CoWoS, and composite package yield. Finally, it analyzes system-level performance, network bottlenecks, and CPO, unifying the technology roadmaps at each level through the concept of “data-movement cost.” When assessing competitiveness, the report consistently replaces point metrics with system metrics—for example, shifting from core count to compute, memory, I/O, interconnects, and power consumption; from packaging capacity to good package output; and from single-GPU performance to cluster utilization and useful compute at the AI-factory level.

Methodology notes

• Competition and Strategy FrameworkMoat / competitive advantage

  Heterogeneous Computing Integration Capabilities

  The report does not limit AMD’s advantages to its ability to design individual CPUs, GPUs, or ASICs. Instead, it attributes them to the company’s long-term platform capability to integrate compute dies, I/O, HBM, interconnects, packaging, racks, and networking.

• Industry Analysis Framework

  Layered Bottleneck Analysis from Chips to Packages, Racks, and Clusters

  The report identifies memory, I/O, packaging yield, interconnect, power delivery, cooling, and network constraints across successive layers of AI systems to explain why performance competition is expanding from individual chips to the entire AI factory.

• Industry Analysis Framework

  Composite Package Yield Breakdown

  The report treats the final yield of shippable accelerators as the combined result of KGD, bonding, interconnect, HBM, packaging, and final-test yields, illustrating why nominal packaging capacity cannot be equated with actual good-unit output.

• Industry Analysis FrameworkSubstitution Effect Analysis

  Coexistence of General-Purpose GPUs and Custom ASICs

  The report believes that the general-purpose GPUs and internally developed ASICs adopted by cloud service providers do not necessarily replace each other and can coexist within large data centers for different workloads.

Asset mapping & comparison

Structured mapping from thesis to named assets (strengths, weaknesses, peers, risks).

• Advanced Micro Devices, Inc.（AMD）

  The report positions AMD as an entity evolving from a CPU and GPU supplier into a provider of rack-scale and cluster-scale heterogeneous AI computing platforms.

  Strengths

  AMD has established a modular foundation encompassing EPYC, Instinct, CPU/GPU compute dies, I/O dies, HBM, Infinity Fabric, and chiplet packaging, and has experience integrating different computing and interconnect technologies into a platform.

  Weaknesses

  Expansion to racks and clusters will significantly increase the complexity of coordinating advanced packaging, interconnects, power delivery, liquid cooling, mechanical structures, and software. The performance of an individual chip is no longer sufficient to determine overall competitiveness.

  Comparison

  Compared with NVIDIA, Intel, and cloud service providers’ internally developed ASICs, the report argues that the competitive benchmark will no longer be CPU versus CPU or GPU versus GPU, but useful AI compute, energy consumption, latency, and data-movement costs per rack and within the data center.

  Risks

  Yield issues at any packaging stage, insufficient inter-GPU communication bandwidth, lagging memory and I/O scaling, and inadequate rack-scale hardware-software coordination could all constrain final system performance.

Key data

• EPYC Composite Competitive MetricCompute Density × Memory Bandwidth × I/O Bandwidth × Packaging Efficiency × Power EfficiencyThe report argues that EPYC competition should no longer be measured solely by core count.
• Change in the Server CPU Performance FrameworkCore Count × IPC × Frequency → Compute × Memory × I/O × Interconnect ÷ PowerThis reflects the increasing dependence of AI server performance on system I/O throughput and energy efficiency.
• Evolution of the CPU and GPU Platform ArchitectureCPU → PCIe → GPU；CPU ↔ High-Speed Fabric ↔ GPU ↔ HBM；CPU + GPU + HBM + NIC + Switch + Optical I/OThe architecture is shifting from a traditional host–accelerator connection toward platform-level coordination.
• AI Hardware Design UnitDie → Package → Board → Tray → Rack → ClusterHelios expands AMD’s design boundary to the rack level.
• Composite Advanced Packaging YieldKGD Yield × Bonding Yield × Interconnect Yield × HBM Yield × Packaging Yield × Final Test YieldA failure at any stage may reduce the final number of shippable accelerators.
• Division of Roles Between SoIC and CoWoSSoIC = Vertical Integration；CoWoS = Horizontal IntegrationSoIC handles Z-axis vertical scaling, while CoWoS handles X-Y planar scaling.
• Change in the CPO LinkSwitch ASIC → Package → PCB → Connector → Optical Module；shifting to Switch ASIC → Optical Engine → FiberThe objective is to shorten high-speed electrical paths and reduce loss and power consumption.
• AMD Scaling RoadmapTransistor Scaling → Chiplet Scaling → Package Scaling → Rack Scaling → Cluster ScalingThe competitive scope is progressively expanding from transistors and chiplets to packages, racks, and clusters.
• Long-Term Heterogeneous Computing PlatformEPYC + Instinct + Chiplets + HBM + Advanced Packaging + Networking + Helios + CPOThe report believes this integrated platform represents AMD’s long-term technology assets better than any individual CPU or GPU.
• Technology Strategy Objective“More Compute per Chip” → “More Useful Compute per AI Factory”The focus of evaluation is shifting from single-chip compute performance to useful compute at the AI-factory level.

Impact & implications

The report’s main implication is that the competitive evaluation framework for AMD and the AI accelerator value chain must become system-oriented: CPUs should be assessed in conjunction with memory, I/O, interconnects, and power consumption; GPUs should be assessed based on HBM, Scale-Up, Scale-Out, software, and cluster utilization; and advanced packaging should be assessed based on good-unit output rather than nominal capacity. As the design boundary expands, AMD must evolve from a chip-component supplier into a rack-scale platform participant capable of coordinating compute, networking, power delivery, cooling, and software. The importance of coordination among wafer fabrication, 3D integration, advanced packaging, testing, and optical interconnects will also increase.

Risks

• Multi-die GPUs reduce the yield risk associated with extremely large monolithic dies but significantly increase packaging and integration complexity.
• Problems at any stage involving KGD, bonding, interconnects, HBM, packaging, or final testing could reduce final good package output and cause economic losses.
• If memory bandwidth and I/O capabilities do not improve in step with CPU core density, the marginal performance gains from adding cores may decline.
• Even if single-GPU compute performance improves significantly, insufficient inter-GPU communication bandwidth will continue to constrain cluster performance and computing-resource utilization.
• Higher-speed SerDes will increase insertion loss, power consumption, Retimer requirements, and PCB signal-integrity pressure.
• Rack-scale systems are increasingly constrained by power delivery, liquid cooling, network bandwidth, software efficiency, and interconnect coordination.

What to watch

• Monitor whether subsequent EPYC upgrades simultaneously improve memory bandwidth, I/O dies, PCIe/CXL, and accelerator connectivity.
• Monitor whether AMD participates in cloud service providers’ custom ASIC projects and whether it can integrate customers’ accelerator IP with its own CPU, I/O, memory, and packaging technologies.
• Monitor Helios’s progress in rack-scale coordination across CPUs, GPUs, HBM, switches, NICs, power delivery, liquid cooling, and software.
• Monitor advanced packaging Good Package Output and the composite yield of KGD, bonding, interconnects, HBM, packaging, and final testing.
• Monitor next-generation Instinct’s HBM bandwidth, Scale-Up bandwidth, Scale-Out networking, cluster utilization, and software efficiency rather than focusing solely on single-GPU performance.
• Monitor whether CPO can shorten high-speed electrical paths and reduce insertion loss, SerDes power consumption, Retimer requirements, and signal-integrity pressure.