Micron warns that AI storage walls are intensifying, with HBM accounting for about 17% of the interruptions in Meta Llama 3 training

According to TrendForce, at Hot Chips 2026, Micron emphasized that the progress of AI computing power is faster than storage improvements, and the challenge of the "storage wall" is becoming increasingly prominent. Micron Fellow Raghu Sriramaneni pointed out that the computing power of AI accelerators increases approximately threefold every two years, while the bandwidth of HBM increases by less than double during the same period, and the gap may further widen. The deepening reliance on HBM also brings reliability issues; Micron cited Meta Llama 3 data indicating that HBM failures account for about 17% of unexpected training interruptions. New designs that blur the boundaries between computing and storage may help break through the bottleneck.

The expansion of HBM also brings area and supply pressures: the latest generation of packaging integrates two GPUs with eight 12-layer HBM4 stacks, with storage accounting for about 90% of the semiconductor area, more than eight times the area occupied by GPUs; achieving equivalent capacity with HBM requires about three times the wafers of standard DDR5 DRAM. Thermal management has also become a key constraint, with solutions such as liquid cooling and ultra-thin chips being explored.

Micron is advancing innovations in interconnect, packaging, and cooling, including SerDes and die-to-die PHY optimized for storage, larger size SiP and advanced packaging with glass substrates, as well as liquid cooling and hybrid bonding, and is developing fusion bonding to reduce thermal resistance and enhance data throughput.