Beyond Cloud AI, Semiconductor Back-end Processes and Advanced Packaging Enter the Era of System Optimization

Overview

This report summarizes Jefferies' seminar views on semiconductor back-end process and packaging trends. Its core judgment is that although cloud AI remains the main axis of AI systems, it is not the whole answer. As GPU computing power continues to improve, interconnects, I/O power consumption, packaging density, system-level architecture, and edge AI deployment are becoming new bottlenecks and opportunities. The report emphasizes that back-end processes have evolved from traditional assembly steps into key sources of value for system performance, power consumption, cost, and commercialization capabilities.

Core views

First, AI systems need to be redefined as communications systems, with chiplets, advanced packaging, and CPO serving as important paths to address data-movement bottlenecks. Second, the true significance of CPO lies in changing the location and role of I/O, with the focus on reducing external connection power consumption rather than merely increasing computing speed. Third, edge AI and cloud AI will form a division of labor: cloud AI will handle training and integration, while edge AI will handle inference and immediate decision-making. Fourth, HBM is suitable for accelerating AI, but in edge scenarios it is constrained by power consumption, packaging costs, and scale; DDRx 3D stacking may be better suited. Fifth, hybrid bonding is not merely a simple connection technology following microbumps, but a key technology that extends BEOL interconnects beyond chip boundaries and restructures the boundaries of system design. Sixth, Japanese companies have strengths in materials and components, but need to shift from selling individual materials to providing “process + evaluation + design” solutions, while strengthening overseas back-end process co-creation platforms.

Analysis framework

The report adopts an analysis framework based on expert seminar minutes. It follows Professor Fumihiro Inoue's technical views on AI systems, advanced packaging, CPO, chiplets, edge AI, substrate materials, and hybrid bonding, while combining industry cases involving NVIDIA, TSMC, Qualcomm, Huawei, Resonac, and others to examine the impact of technological evolution on materials, packaging, equipment, design, and business models.

Methodology notes

• Technology trend analysisAI system bottleneck migration framework

  Shift from computing bottlenecks to communications, interconnect, and power-consumption bottlenecks

  The report argues that once GPU computing performance continues to improve, data movement between GPUs, between GPUs and HBM, and between racks becomes a constraint, making back-end packaging and interconnect technologies key areas of competition in AI systems.

• Value-chain analysisarchitecture × package

  Collaborative design of architecture and packaging

  Advanced packaging is not merely a manufacturing step; it requires synchronized optimization of system architecture, optoelectronic and thermal design, volume-production packaging, and business models.

• Technology roadmap analysisScale-up × Scale-out

  Vertical integration and planar expansion of chiplets

  The report breaks down the value of chiplets into Scale-up and Scale-out, noting that the functions and structures required for edge AI place greater emphasis on Scale-up, while Japanese back-end R&D is currently overly focused on the Scale-out and panel-level packaging boom.

Asset mapping & comparison

Structured mapping from thesis to named assets (strengths, weaknesses, peers, risks).

• NVIDIA

  Leader in the cloud AI ecosystem and GPU platforms

  Strengths

  Builds a software ecosystem based on CUDA, offers a platform rather than a single GPU, and has established a combination of customer and investor trust with TSMC.

  Weaknesses

  Will continue to face interconnect and power-consumption bottlenecks in the next phase.

  Comparison

  Compared with pure-play chip suppliers, NVIDIA's advantage lies in its design and software ecosystem platform.

  Risks

  If data movement, I/O power consumption, and rack interconnect issues cannot continue to improve, system scaling efficiency may be constrained.

• TSMC

  Key producer of advanced processes, advanced packaging, and CPO

  Strengths

  Holds an overwhelming position in advanced-process and advanced-packaging supply capabilities, while COUPE and SoIC bonding demonstrate high-precision connection capabilities.

  Weaknesses

  Solutions such as CoWoS that rely on silicon interposers face potential substitution risks from cost and scalability concerns and technological advances on the substrate side.

  Comparison

  Compared with open chiplet models, TSMC, Intel, and Samsung tend to favor vertically integrated models.

  Risks

  If improvements in organic RDL, glass-core substrates, and substrate-side interconnect density reduce the necessity of silicon interposers, parts of the existing value allocation may be reshaped.

• Japanese materials and components companies

  Suppliers of advanced packaging materials and components

  Strengths

  Possess world-class core technologies in materials and components.

  Weaknesses

  Lack sufficient capabilities in systems, packaging design, volume-production assembly, synchronized optoelectronic and thermal optimization, and the commercial concept of “how to integrate and sell.”

  Comparison

  CPO globally is approaching volume production, while Japanese companies have long treated CPO as a future research theme and have become involved relatively late from design, assembly, and business perspectives.

  Risks

  If they remain focused on selling individual materials, they may miss system-level value creation in the architecture × package phase.

• Resonac

  Promoter of back-end process co-creation platforms

  Strengths

  Through US-JOINT and JOINT3, promotes US back-end process co-creation, a panel-level organic interposer prototype line, and shared pilot lines.

  Weaknesses

  Needs to convert alliance and prototyping capabilities into volume production, supply-chain, and customer design-in capabilities.

  Comparison

  Compared with relying solely on domestic Japanese capabilities, overseas co-creation platforms are closer to the advanced packaging ecosystem.

  Risks

  If coordination among participating companies is insufficient or volume-production barriers are not overcome, the value of the alliances may remain at the prototype stage.

• Qualcomm

  Explorer of AI inference and alternative memory-packaging solutions

  Strengths

  HBC Gen1 achieves high effective bandwidth without using HBM bandwidth devices or adopting TSMC CoWoS.

  Weaknesses

  The report does not provide complete evidence on volume-production yield, cost, or ecosystem compatibility.

  Comparison

  Compared with the HBM + interposer route, Qualcomm's solution seeks to reduce the additional power consumption caused by physical bridging.

  Risks

  If supply, testing, and cost issues cannot be resolved, the solution may encounter the volume-production barrier in chiplet commercialization.

• Huawei

  Proponent of the Beyond Moore and CMOS2.0 directions

  Strengths

  The Tau Scaling Law focuses on shortening signal transmission time, reflecting an overall optimization approach spanning devices, wiring, chips, and systems.

  Weaknesses

  Constrained by limited access to EUV equipment, it needs to improve performance through system-level and vertically integrated approaches.

  Comparison

  Compared with traditional More Moore transistor scaling, Huawei's direction places greater emphasis on system latency and ecosystem-level optimization.

  Risks

  Technical validation, manufacturing constraints, and ecosystem coordination remain key uncertainties.

Key data

• AI semiconductor market sizeUS$236bn in 2025, expected to reach US$371bn in 2028The report cites a VLSI forecast; cloud AI is estimated at approximately US$160bn and edge AI at approximately US$80bn in 2025, rising to approximately US$250bn and US$120bn, respectively, in 2028.
• TSMC COUPE technology effectsMore than 170% speed improvement at the same power consumption, with power consumption reduced by 40%The report also mentions low chip-to-chip interface impedance, at least 16× higher bonding density, an 85% reduction in parasitic capacitance, and additional optical-path insertion loss for COUPE optimized to nearly 0 dB.
• Qualcomm HBC Gen1 bandwidth133TB/s effective memory bandwidth per cardThe report states that this solution is used in Qualcomm's AI250 AI inference chip, does not use TSMC CoWoS, and provides 18× the bandwidth of AI200 LPDDR5.
• Resonac US-JOINT10 Japanese and US companies participatingA co-creation framework for back-end process technology established in Silicon Valley.
• Resonac JOINT327 companies participating, targeting construction of a 515×510mm panel-level organic interposer prototype lineThis corresponds to the back-end process trend of shifting from wafers toward panels and larger areas.

Impact & implications

From an investment perspective, the importance of advanced packaging, CPO, hybrid bonding, fine-pitch substrates, glass-core substrates, organic RDL, advanced materials, and back-end process co-creation platforms is increasing. Companies that traditionally provide only individual materials or components need to upgrade into suppliers capable of providing collaborative solutions covering processes, evaluation, and design. For Japanese companies, their advantages in materials and components remain valuable, but if they cannot close gaps in system packaging, volume-production assembly, integrated optoelectronic and thermal design, and commercial integration, they may fall behind during the global commercialization of advanced packaging.

Risks

• Even after functional validation, CPO, chiplets, and hybrid bonding may remain stuck at the prototype stage because of testing, supply, yield, responsibility-boundary, and cost issues.
• Silicon interposers face structural redundancy, scalability, and cost risks, while advances on the substrate side may dilute their value.
• If Japanese companies cannot close gaps in system design, volume-production packaging, and commercial integration, their materials advantages may be difficult to convert into leadership in advanced packaging.
• Growth in edge AI demand does not imply a decline in cloud AI demand. Misjudging the division of labor between the two could lead to an imbalanced allocation of investment themes.
• The report is an industry technology thematic report and does not provide explicit individual stock earnings forecasts, ratings, or target prices; investment conclusions require further validation against company fundamentals.

What to watch

• The timeline for CPO to move from a research theme to volume production, as well as actual product adoption by NVIDIA, TSMC, Broadcom, Marvell, Ayar Labs, and others.
• The power-consumption, yield, and cost performance of TSMC COUPE, SoIC bonding, and other high-density optoelectronic integration solutions.
• The pace of edge AI deployment in industrial, automotive, robotics, and infrastructure monitoring applications, and the suitability of DDRx 3D stacking relative to HBM.
• Whether organic RDL, glass-core substrates, bridges, and advanced RDL on substrate can reduce reliance on silicon interposers and CoWoS-type architectures.
• Whether Resonac US-JOINT and JOINT3 can progress from co-creation and prototype lines to customer qualification, volume supply, and revenue contribution.
• Whether validation related to Huawei's Tau Scaling Law, CMOS2.0, and hybrid bonding can establish a replicable path to system-level performance improvements.