Preface Executive Briefing

Week of May 9, 2026

The AI Hardware Bottleneck: Memory & Storage

Compute is nothing without Data Delivery

A 'chip' in AI is not just the GPU; it is Compute + Interconnect + Memory. While the world obsesses over GPUs, Memory (HBM, DRAM) and Storage (SSD/NAND) have emerged as the critical bottlenecks for AI Inference. SanDisk and Micron are surging as the real scarcity shifts from compute to data staging and delivery.

Two Phases of AI Hardware

Compute, memory, storage, and interconnect are the parts, but which workload uses them most? Training and Inference have fundamentally different bottlenecks.

Build the Model

The process of building an AI model from massive datasets and specific training methodology. Focuses on model learning and feature selection.

  • Two-way flow: Forward pass + Backpropagation
  • Bottleneck: HBM Capacity & Sync Bandwidth
  • Splits work across many GPUs constantly

Run the Model

The deployment of the trained model as a standalone application or integrated into a larger system. Optimizes for cost per request + latency.

  • One-way flow: Forward pass only
  • Bottleneck: Latency & Per-User Throughput
  • Limiting factor: Faster data delivery via memory

This Week's Core Breakthroughs

The real scarcity in AI infrastructure is shifting from GPUs to Storage and Stage memory as organizations broadly deploy AI inference.

SanDisk Surge & The Memory Bottleneck

Memory and storage are currently the key bottlenecks. Benefitting from global NAND shortages and an inference pivot, SanDisk shares surged nearly 1000% since its spin-off from Western Digital. Micron's shares have also surged 300% due to the "memory wall".

+1,129%

SanDisk stock growth in 6 months

HBM Market Concentration

High Bandwidth Memory (HBM) is the strategic choke point feeding AI accelerators at maximum speed. The Q4 2025 market is highly concentrated, with SK Hynix leading as the HBM leader.

57%

SK Hynix Q4 2025 HBM share

OpenAI Stargate & MRC Protocol

If data moves too slowly across the interconnects, GPUs sit idle. OpenAI launched Stargate and partnered with AMD, Broadcom, Intel, Microsoft, and NVIDIA to deploy MRC (Multipath Reliable Connection), spreading packets to avoid network hot-spots during large-scale AI training.

MRC

New protocol for high-perf GPU networks

The Global Chip Landscape

While the chip market is led by US AI chip makers, alternatives and critical material suppliers are rising.

US Dominance vs Chinese Alternatives

NVIDIA leads by a wide margin with models like the Vera Rubin boasting 50,000 TFLOPS of compute and 3,600 GB/s interconnect speeds. Chinese models like Huawei's Ascend 910C (~1,100 TFLOPS) struggle with interconnect bottlenecks and frequent crashes during large-scale training runs.

  • Chinese Hybrid Strategy: Use NVIDIA H800s for high-stakes training, and Huawei for cost-sensitive inference.
  • DeepSeek: Unveils new models at rock-bottom prices with 'full support' from Huawei chips.
Unexpected Players in Japan

Consumer-facing companies are leveraging unique tech for AI chip materials. Ajinomoto provides ABF thermosetting films for high-performance semiconductor packaging. TOTO supports AI chip production through advanced ceramics and Y2O3 films that provide outstanding plasma resistance in manufacturing equipment.

South Korea's Sovereign AI

A government-backed initiative shortlists national champions for domestic model development, providing direct funding and large-scale GPU capacity.

  • LG AI Research: Front-runner leading in intelligence benchmarks.
  • Upstage: Startup powerhouse developing Solar Open reasoning models.

The Memory Hierarchy Explained

From persistent storage to ultra-fast compute: data moves up the hierarchy to feed the chip.

Layer 3

3. FEED: HBM

High Bandwidth Memory (Volatile)

  • Fastest, closest, and most expensive memory.
  • Ultra-high bandwidth (TB/s range) and very low latency.
  • Purpose-built to feed the compute chip really fast during training/inference.

Layer 2

2. STAGE: DRAM

System Memory (Volatile)

  • Main working memory used to stage active data and programs for processing.
  • Larger capacity (GBs-TBs) but higher latency than HBM.
  • Off-package flexible working memory.

Layer 1

1. STORE: SSD / NAND

Solid State Drive (Persistent)

  • Warehouse for long-term data storage. Retains data without power.
  • Stores models, datasets, logs, and checkpoints.
  • High density, low cost per bit, but slower than DRAM. SanDisk specializes here.

Chip Specs, Components & Korea Sovereign AI

Explore the specific benchmarks and architectural data driving this week's AI hardware narrative.

Chip Specs Comparison

US AI Chip Makers vs Rising Alternatives

MetricHuawei Ascend 910CNVIDIA H20 (Restricted)NVIDIA Vera Rubin
Process NodeSMIC 7nm (DUV)TSMC 5nm (EUV)TSMC 3nm (EUV)
Raw Compute~1,100 TFLOPS296 TFLOPS50,000 TFLOPS
Memory Bandwidth3.2 TB/s4.0 TB/s22 TB/s
Interconnect Speed392 GB/s900 GB/s (NVLink)3,600 GB/s (NVLink v5)

Key Takeaways

Jensen Huang's famous '5-Layer Cake Concept' illustrates that AI isn't just about models; it spans Application, Model, Infrastructure, Chips, and Energy.

01

A "Chip" is a System, Not Just a GPU

In AI, a chip equals Compute + Interconnect + Memory. Focusing solely on the GPU ignores the fundamental physical limits of data delivery, which dictates how fast the entire system can move.

02

Inference Changes the Bottleneck

While training requires massive interconnect speed to keep GPUs busy, continuous inference workloads are heavily bottlenecked by how fast data can be fed to the GPU through memory. Storage scarcity is driving massive growth for SSD/NAND manufacturers like SanDisk.

03

Chinese Alternatives Face Interconnect Hurdles

Despite making headway with hardware like the Huawei Ascend 910C, Chinese chipmakers still struggle with interconnect bandwidth (392 GB/s vs NVIDIA's 3600 GB/s) and stability during large-scale training.

04

Unexpected Winners in the Value Chain

Because the value chain is thin, AI demand impacts surprising sectors. Japanese companies like Ajinomoto (advanced packaging films) and Toto (plasma-resistant ceramics) are emerging as critical pillars in the physical manufacturing process of these chips.

FAQ

SanDisk spun-off from Western Digital in February 2025. There has been a global NAND shortage causing prices to rise heavily. The market is experiencing a massive inference pivot; Inference heavily taxes data storage and staging, putting SSD/NAND manufacturers in prime position.