For the better part of two years, the AI hardware story had a single protagonist: the GPU. Training runs consumed clusters of accelerators at scales that made NVIDIA the most valuable company on earth, and every conversation about AI infrastructure circled back to H100s and their successors. In 2026 that story changed. The industry formally entered an inference-dominated cycle, and with it the center of gravity in silicon has shifted away from the training accelerator and toward two categories that spent the last decade as afterthoughts: the CPU and memory/storage.
The shift is structural, not cyclical. It is driven by the mass deployment of agents and intelligent endpoints — scenarios where compute requirements look nothing like a large training run. Understanding it requires rethinking not just which chips matter, but how the entire compute architecture is being rebalanced.
The GPU:CPU Ratio Is Inverting
The cleanest signal of the new regime is the ratio of GPUs to CPUs in AI deployments. In the training era, the industry standard was 8 GPUs to 1 CPU (sometimes 4:1): the accelerator did the heavy parallel work, and the CPU handled only orchestration, scheduling, and system control. CPUs were a supporting cast member, sized to not get in the GPU’s way.
Inference inverts the equation:
- Cloud inference: the ratio moves toward 4:1 or 2:1, as serving workloads demand more orchestration, batching logic, and serial task coordination.
- AI agents and edge intelligence: the ratio approaches 1:1, as agents continuously issue tool calls, manage state across long sessions, and coordinate multi-step task chains that are inherently serial.
The reason is straightforward. Training is massively parallel: millions of matrix operations run simultaneously, and a GPU’s thousands of cores are exactly the right tool. Inference for agents is different. An agent does not perform one giant parallel computation; it performs a long sequence of dependent steps — call a tool, parse the result, decide what to do next, call another tool — where each step depends on the output of the last. This serial, logic-heavy, control-intensive workload is precisely what CPUs were designed for. The GPU remains essential for the model’s token generation, but the CPU becomes the conductor of an increasingly complex orchestration layer.
The result has been a demand surge that caught the industry off guard. Intel’s stock roughly tripled during 2026, driven not by a single product cycle but by the realization that the inference era makes x86 CPUs indispensable again. Domestic CPU makers in China — Hygon, Loongson, Huawei Kunpeng, Phytium — have ridden the same wave, with government, finance, energy, and telecom customers reserving multiple domestic supply slots as a hedge against the capacity crunch.
The Storage Super-Cycle Extends
If CPUs are the surprise winner of the inference cycle, memory and storage are the confirmed, extended winners. The consensus as of late 2025 was that storage shortages would persist until new capacity arrives in 2028, with a price inflection no earlier than the second half of 2027. The acceleration of AgentAI has only widened that gap.
The numbers are stark:
- Terminal device memory is jumping from 16GB to over 100GB.
- Storage per device is moving from 512GB–1TB to tens of terabytes.
- Edge nodes require low-latency, high-reliability storage at multiples of previous demand.
At the high end, HBM (High Bandwidth Memory) is in a structural shortage. HBM3E has only just reached stable supply, and HBM4 is already entering scarcity. The major suppliers — Samsung, SK Hynix, Micron — are locking in 3-to-5-year long-term agreements directly with hyperscalers, bypassing the module makers that historically sat between them. Profit is concentrating upstream: the original equipment manufacturers capture the margin, while module makers see their bargaining power erode as large customers go direct.
The underlying dynamic is simple. Storage supply is rigid. Fab expansion takes 18–24 months, and demand from inference workloads is growing faster than capacity can be added. Unlike the GPU shortage of 2023–2024, which was driven by a single product cycle, the memory shortage is driven by a structural shift in how much memory every AI-powered device needs. That gap is not closing in 2026 or 2027.
Why Agents Break the Old Assumptions
The reason both CPU and storage demand have spiked simultaneously is that agentic workloads break the assumptions that shaped the training-era hardware stack.
-
Agents are stateful, not stateless. A chat completion is a one-shot inference: prompt in, completion out, state discarded. An agent maintains state across hours or days of tool calls, requiring memory to persist between turns and storage to checkpoint progress. Every agent session is, in effect, a long-running process that consumes both compute and memory continuously.
-
Agents are serial, not parallel. The reasoning chain of an agent — plan, act, observe, replan — is fundamentally sequential. Each step depends on the previous one. This rewards the low-latency serial throughput of a CPU rather than the massive parallelism of a GPU.
-
Agents run everywhere. The training era concentrated compute in a handful of hyperscale data centers. The inference era distributes it: in the cloud, at the edge, in vehicles, in industrial controllers, in phones. Distributed inference requires CPUs at every node because every node needs orchestration, and it requires storage at every node because every node needs local state.
The compound effect is that the compute architecture for the agent era is not “more GPUs.” It is “GPUs plus a lot more CPUs plus a lot more memory, deployed everywhere.”
The China Accelerator Market: Non-GPU Overtakes GPU
A parallel data point confirms the direction of travel. IDC reported that in the first half of 2026, total AI accelerator shipments in Chinese data centers reached 2.008 million units, of which 1.016 million were non-GPU accelerators — the first time non-GPU accelerators have exceeded GPU shipments in the market. NVIDIA remains the single largest vendor, but its gap to Huawei’s Ascend line has narrowed to roughly six percentage points of market share.
This matters because it shows the inference cycle is not just a CPU story; it is a silicon diversification story. As inference becomes the dominant workload, the value of specialized accelerators — NPUs, TPUs, custom ASICs, and domestic alternatives — rises relative to general-purpose GPUs. The training-era assumption that one giant GPU cluster serves every workload is being replaced by a heterogeneous architecture where different silicon serves different inference patterns.
For Chinese model providers like DeepSeek, this diversification is both strategic and practical. The Inner Mongolia data center reportedly deploying at least 160,000 Huawei AI accelerators is the concrete expression of a training-stack migration to domestic silicon. Meanwhile, lightweight models that run on consumer GPUs reduce inference dependence on high-end accelerators. The inference cycle rewards exactly this kind of silicon flexibility.
Hardware Value Returns
The broader investment implication is a return of hardware value. The 2020s were dominated by software narratives — SaaS multiples, platform effects, zero-marginal-cost distribution. The AI inference era is reasserting the value of physical infrastructure: CPUs with decades of ecosystem lock-in, memory fabs with 18-month lead times, advanced process nodes booked years in advance. Hardware has certification barriers, cash-flow durability, and anti-cyclical properties that software cannot replicate.
The next two to three years will be defined by supply-demand imbalance across three axes: CPU demand explosion, the storage super-cycle, and advanced-process capacity scarcity. The companies that win will not be those with the best marketing story but those with control over capacity, process nodes, and ecosystem integration. Intel’s IDM model — owning its own fabs — is suddenly a strategic advantage when competitors must queue for third-party foundry capacity. The memory oligopoly — Samsung, SK Hynix, Micron — is capturing multi-year profits through long-term direct agreements.
What This Means for the Model Layer
For model providers, the inference cycle has two consequences. First, inference efficiency is now a product feature. A model that can deliver comparable reasoning with fewer tokens, smaller memory footprint, and lower CPU orchestration overhead has a structural cost advantage — which is exactly why efficient reasoning models like DeepSeek’s V4.1-Flash and DeepThink’s reasoning engines have gained traction. Second, silicon strategy is now model strategy. The choice of training accelerator, the design of the inference runtime, and the optimization for edge deployment are no longer backend concerns; they are determinants of unit economics and therefore of market position.
The training era made the GPU king. The inference era is making the whole compute stack matter again — and the companies that control the silicon, the memory, and the efficiency of the models running on top of it are the ones positioning themselves for the next phase of AI.