By 2024, scaling a language model usually meant scaling its inference bill in lockstep. DeepSeek-V2 breaks that coupling: it carries 236B parameters but fires only 21B per token, so capacity and serving cost stop moving together. The more telling number is the KV cache — the part of attention that quietly dominates long-context memory — shrunk by 93.3%, which is what actually makes 128K context affordable.
Key Findings
- Multi-head Latent Attention (MLA) compresses keys and values into a low-rank latent vector instead of caching them per head, the lever behind the 93.3% KV-cache cut and a 5.76x jump in peak generation throughput.
- DeepSeekMoE uses many fine-grained experts plus shared experts, so the model routes sparsely without the load-balancing waste that plagues coarse MoE designs.
- The combination is what drives the headline economics: 42.5% lower training cost than the dense DeepSeek 67B, while landing among the strongest open-weight models of its release window.
- Trained on 8.1T tokens, then aligned with SFT and RL, so the gains aren't just a pretraining artifact.
Who Should Read This
Great fit if you build or operate LLM serving infrastructure and care about the memory-vs-context tradeoff, or if you're studying attention variants and MoE routing as a research direction — MLA is the contribution worth dissecting. Look elsewhere if you want a hands-on deployment guide; this is the architecture paper, and the practical recipe lives in DeepSeek's later model releases. Note also that the reported economics are relative to DeepSeek's own dense baseline, not a universal benchmark.
