SanDisk and SK hynix push High Bandwidth Flash as a new memory layer for AI inference
SanDisk and SK hynix are promoting a new architecture called High Bandwidth Flash, or HBF, that applies HBM-style advanced packaging and die stacking to NAND Flash. The idea is not to replace High Bandwidth Memory, but to insert a new layer into AI server memory hierarchies for large, read-heavy data sets used during inference. According to public material cited in IEEE Spectrum, first-generation HBF is targeting up to 16 stacked NAND chips, as much as 512GB per stack, and read bandwidth of up to 1.6TB/s. SanDisk’s roadmap then points to 2TB/s and 3.2TB/s in later generations. SK hynix executive Hoshik Kim said the approach could deliver bandwidth well beyond standard NVMe storage and help relieve the severe capacity pressure faced by HBM. The concept is aimed at inference rather than training. In training, models require constant reads and writes, which is a poor fit for Flash. In inference, model weights are largely fixed, making static, read-intensive data such as billion-parameter weights and precomputed KV cache more suitable for HBF. The companies have already launched standardization work under the Open Compute Project, though IEEE Spectrum reported that shipping is still at least a year away and broad deployment may take several more years.







