AI data center demand remains strong, B200 GPU pricing stays firm, and memory price inflation is starting to cool
AI summary card
AI data center demand remains strong, B200 GPU pricing stays firm, and memory price inflation is starting to cool
J.P.Morgan's Data Center Watch shows that in July, LLM token volume and spending continued to post strong year-over-year growth, B200 GPU rental prices accelerated month over month, while DRAM and NAND contract price increases are expected to slow significantly in 3Q26.
- Under the OpenRouter metric, July token volume grew 18% month over month and about 21x year over year, while overall spending grew 7% month over month and about 11x year over year.
- Closed-source models still dominated spending, accounting for about 80% of July spending; open-weight model spending share rose to 20%, with both price and volume growing rapidly.
- The average B200 GPU rental price in July was about $5.72/GPU-hour, up 7.3% month over month, with the B200/H100 price ratio rebounding to about 2.13x.
- DRAM DDR5 16Gb spot prices rose to $50.83 in July, up 17% month over month and 890% year over year; NAND 1Tb spot prices edged down 0.5% month over month but were still up 417% year over year.
- TrendForce expects 3Q26 DRAM and NAND contract prices to rise 16% and 13% quarter over quarter, respectively, a clear slowdown from 56% and 58% in 2Q26.
Report interpretation
Overview
This report is J.P.Morgan's review of high-frequency indicators related to AI data centers, focusing on LLM token usage and pricing, non-hyperscaler GPU rental pricing, and DRAM/NAND memory prices. The overall conclusion is that AI infrastructure demand remains healthy: LLM usage and spending continue to expand, and next-generation B200 GPU pricing is stronger; meanwhile, memory spot prices remain elevated, but the pace of contract price increases is expected to slow materially in 3Q26.
Core views
The core view includes three points. First, LLM usage growth slowed in July versus June, but remained at a very high multiple year over year, indicating that AI inference demand is still expanding. Second, GPU rental prices are showing generational divergence, with B200 month-over-month gains accelerating while H100 posted its first month-over-month decline, possibly reflecting a shift in demand toward newer-generation GPUs. Third, DRAM and NAND price inflation remains strong, but due to a high base, weakening consumer demand, CSP long-term agreements, and inventory effects following pull-ins in the first half, contract price increases in 3Q26 are expected to cool sharply.
Analysis framework
The report uses a monthly data-tracking framework that combines weekly usage and pricing samples for about 300 LLMs on OpenRouter, the Bloomberg GPU rental price index, Bloomberg memory spot data, and TrendForce contract price forecasts to assess AI infrastructure demand, compute pricing, and changes in memory costs.
Methodology notes
Tracks monthly token usage, pricing, and spending for about 300 LLMs through sampling on the OpenRouter platform.
This metric is skewed toward developers, startups, and agentic-coding traffic, and does not include direct first-party API traffic from OpenAI, Anthropic, and others, or hyperscaler-hosted endpoints; it is therefore useful for observing marginal trends, but does not represent complete market demand.
Uses the Bloomberg index to observe non-hyperscaler GPU rental pricing.
The report compares monthly average rental prices and intergenerational price ratios for A100, H100, and B200 to assess relative supply-demand strength across GPU generations.
Combines Bloomberg spot prices with TrendForce contract price forecasts to track cost inflation in DRAM and NAND.
Spot prices reflect short-term market tightness, while contract prices better capture the effects of CSP long-term agreements, inventory, and quarterly procurement cadence on memory costs.
Asset mapping & comparison
Structured mapping from thesis to named assets (strengths, weaknesses, peers, risks).
- AI infrastructure supply chainBenefits from expansion in LLM usage, token spending, and GPU rental prices.
- Strengths
- Demand remains strongly higher year over year, B200 pricing is firm, and AI inference usage is expanding.
- Weaknesses
- July month-over-month growth slowed versus June, and the OpenRouter sample does not cover the full set of first-party APIs and hyperscaler endpoints.
- Comparison
- Compared with June, month-over-month growth in token volume and spending slowed; compared with the same period last year, growth remains at a high multiple.
- Risks
- Further slowdown in LLM usage growth, intensified model pricing competition, and changes in cloud vendor capex cadence.
- Nvidia GPU rental and high-end GPU supply-demandB200 rental pricing is stronger than H100 and A100, reflecting demand tilting toward newer-generation GPUs.
- Strengths
- B200 rental prices rose 7.3% month over month in July, and the B200/H100 price ratio rebounded.
- Weaknesses
- H100 prices declined month over month, while A100 gains were modest; older-generation GPU pricing momentum is weaker than B200.
- Comparison
- B200 pricing is about 2.13x H100, while H100 is about 1.63x A100.
- Risks
- Greater supply of new GPUs, cooling rental demand, and divergence between non-hyperscaler markets and large cloud-provider demand.
- DRAMAI server and data center memory demand supports pricing, but contract price increases are entering a slower phase.
- Strengths
- DDR5 16Gb spot prices rose 890% year over year, and contract prices are still expected to rise quarter over quarter in 3Q26.
- Weaknesses
- 3Q26E contract price growth is expected to slow to 16%, well below the prior two quarters.
- Comparison
- DRAM contract prices rose 56% q/q in 2Q26 and 96% in 1Q26, slowing to 16% in 3Q26E.
- Risks
- Weak consumer electronics demand, elevated buyer inventories, and slower follow-on procurement after first-half pull-ins.
- NANDStorage prices remain far above year-ago levels, but spot prices have already posted consecutive slight month-over-month declines.
- Strengths
- NAND 1Tb spot prices rose 417% year over year, and 3Q26E contract prices are still expected to rise 13% quarter over quarter.
- Weaknesses
- July spot prices fell 0.5% month over month, and contract price growth has slowed significantly from earlier periods.
- Comparison
- NAND contract prices rose 58% q/q in 2Q26 and 88% in 1Q26, slowing to 13% in 3Q26E.
- Risks
- Rising inventories, weak consumer demand, a high price base, and order declines after advance pull-ins.
Key data
- July token volume growth+18% m/m, about 21x y/yUnder the OpenRouter metric, this slowed from +70% m/m in June, but year-over-year growth remained strong.
- July average token price-3% m/m, +2% y/yYear-over-year pricing shifted from a prior decline to slight growth, while open-weight model VWAP rose 36% month over month.
- July total token spending+7% m/m, about 11x y/yClosed-source models accounted for about 80% of spending, while open-weight model share rose to 20%.
- Average B200 rental price$5.72/GPU-hour, +7.3% m/mB200 posted the strongest month-over-month increase among the three Nvidia GPU categories in July.
- Average H100 rental price$2.68/GPU-hour, -1.1% m/mThis ended seven consecutive months of month-over-month increases, with the H100/A100 price ratio falling to about 1.63x.
- Average A100 rental price$1.66/GPU-hour, +0.9% m/mThis marked the sixth consecutive month of month-over-month gains, though the pace slowed clearly from June and May.
- DRAM DDR5 16Gb spot price$50.83, +17% m/m, +890% y/yPrices continued to rise in July, indicating DRAM spot remains elevated.
- NAND 1Tb spot price$26.90, -0.5% m/m, +417% y/yThis marked the fourth consecutive month of slight month-over-month declines, but prices were still sharply higher year over year.
- 3Q26E DRAM contract price+16% q/qA significant slowdown from +56% q/q in 2Q26 and +96% q/q in 1Q26.
- 3Q26E NAND contract price+13% q/qA significant slowdown from +58% q/q in 2Q26 and +88% q/q in 1Q26.
Impact & implications
From an investment perspective, resilience in LLM token spending and B200 rental pricing supports the view that AI infrastructure demand remains in an expansion cycle, benefiting high-end GPUs, AI servers, networking equipment, and related hardware supply chains; however, the cooling in memory contract price increases suggests DRAM/NAND pricing elasticity may be shifting from rapid upside to a more moderate recovery, so related assets should be monitored for inventory, long-term contract, and consumer-demand changes.
Risks
- OpenRouter data is skewed toward developers, startups, and agentic-coding traffic, and cannot represent the full LLM API market.
- July month-over-month growth in LLM token volume and spending slowed significantly versus June; if sustained, this could weaken expectations for AI infrastructure demand.
- Month-over-month declines in H100 rental pricing suggest supply-demand conditions for older-generation GPUs may be starting to diverge.
- DRAM and NAND contract price increases are expected to slow significantly in 3Q26, which may reduce upside elasticity in memory pricing.
- Weak consumer demand, high buyer inventories, and first-half advance pull-ins could suppress subsequent memory procurement.
- Forecasts and market data in the research report may be revised as supply-chain conditions, cloud-vendor capex, and the model-competition landscape evolve.
What to watch
- Subsequent monthly changes in OpenRouter token volume, average pricing, and total spending.
- Whether spending share for open-weight models continues rising from 20%, and whether models such as Kimi K3 can continue ranking among top spending contributors.
- Changes in B200/H100 and H100/A100 rental price ratios as indicators of GPU generational demand migration.
- Whether DRAM DDR5 16Gb and NAND 1Tb spot prices continue to diverge.
- Whether TrendForce continues to revise down its 3Q26 and subsequent-quarter DRAM/NAND contract price forecasts.
- The effects of CSP long-term agreements, inventory levels, and consumer electronics demand on memory pricing.