News
DeepSeek Shrinks KV Cache Footprint, Testing AI Memory Demand Elasticity
SUMMARY**
**DeepSeek-V4.1-Flash cuts its reported KV-cache footprint to one-quarter of the prior generation in HBM and one-eighth in SSD. The result lowers memory intensity per long-context session, but cheaper inference and higher concurrency could expand token demand enough to offset the per-unit savings.
A New Architecture Targets the Inference Memory Bottleneck
DeepSeek released V4.1-Flash on September 10, 2026, as a 552-billion-parameter mixture-of-experts model with native text and image input. The company says its Causal Encoder-Decoder architecture activates 8 billion parameters per token during prefill and 16 billion during decoding. The official API supports a one-million-token context window and up to 384,000 output tokens. Model weights are available on Hugging Face under an MIT license.
The memory-relevant change is the global key-value cache. DeepSeek reports 890 bytes per token, approximately one-quarter of V4-Flash and 1/437 of DeepSeek-V1. A direct calculation puts one million tokens of global KV cache is about 0.89 GB. That figure is not total deployment memory: it excludes model weights, activations, temporary buffers, host-memory pools and serving overhead.
The reduction combines three mechanisms. The decoder projects the global KV state from the encoder output rather than storing a separate full representation at each decoder layer. Compressed Sparse Attention 2 shares or reuses KV and sparse-attention indices across selected layers. FP4 main-KV caching compresses the retained state. DeepSeek also uses bounded replay to reconstruct sliding-window KV state, which the company says reduces the persistent cache stored on SSD to roughly one-eighth of V4-Flash.
Lower Bytes per Token Creates Two Opposing Demand Effects
The first-order effect is negative for memory intensity. Long-running agents accumulate context and tool outputs, so the KV cache can limit the number of concurrent sessions supported by a fixed accelerator cluster can support. A fourfold reduction in HBM cache per token should allow more sessions on the same installed HBM capacity. An eightfold reduction in persistent SSD cache similarly lowers storage consumed when prefixes or agent histories are retained for reuse.
This is relevant to HBM suppliers Micron, SK Hynix, and Samsung, and indirectly to enterprise-SSD suppliers, including Samsung, Kioxia, and SanDisk. It does not establish a change in the memory cycle. The architecture affects only one component of inference memory, whereas larger models, more accelerators, multimodal data, retrieval stores, and increasing concurrency can increase system-level DRAM, HBM, and NAND requirements.
The second-order effect may be positive through elasticity. DeepSeek prices V4.1-Flash cache-miss input at $0.15 per million tokens off-peak and $0.30 at peak; output costs $0.60 and $1.20 per million tokens, respectively. Its published concurrency limit is 2,500, versus 500 for V4 Pro. Lower cost and higher serving density can make longer contexts, persistent agents, and more frequent inference economically viable. The decisive variable is therefore total token and session growth relative to the decline in bytes required per token.
The Evidence Is Stronger on Design Than Production Economics
The launch, downloadable weights, API specifications, and current pricing are directly verifiable. The architecture and cache figures, however, remain vendor-reported. Independent reproduction at production concurrency has not yet established whether the claimed memory savings translate proportionally into higher throughput, lower latency, or lower total cluster cost. The 552-billion-parameter backbone also means the model is not lightweight in total weight storage, even though only a fraction of the parameters is active for each token.
DeepSeek’s launch page also contains a stale operational claim. It said that all V4 Pro API requests would route to V4.1-Flash starting September 14. The current pricing page now states that the V4 Pro service will continue with unchanged billing to meet user demand. That reversal does not alter the cache architecture, but it weakens any inference based on an immediate, forced migration of DeepSeek’s installed API workload.
The strongest counter-case is that cache-efficient architectures spread faster than inference volume grows. If competing models adopt similar compression, HBM and enterprise-SSD capacity required per dollar of inference revenue could fall structurally. The worst case for the memory thesis would combine rapid architectural diffusion, slower agent adoption, and weak pricing power across memory suppliers.
MEMORY THESIS IMPACT
Near-term direction is negative for HBM and SSD intensity per long-context session: DeepSeek reports fourfold lower HBM and eightfold lower persistent SSD use for KV cache compared to V4-Flash. The medium-term direction for total memory demand is uncertain because lower API costs, a one-million-token context, and higher concurrency can increase token volume. Confidence is medium-high in the architectural mechanism but medium in its aggregate demand. Micron, SK Hynix, Samsung, Kioxia and SanDisk face an indirect efficiency offset rather than a demonstrated cycle inflection. The thesis would be invalidated if hyperscalers show sustained declines in HBM or SSD intensity per unit of inference revenue while token and agent growth remain too weak to compensate.