AIAny
Icon for item

DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

Describes DeepSeek-V4.1-Flash, a 552B-parameter multimodal Mixture-of-Experts LLM that supports up to 1M-token contexts and reduces KV cache footprint using Compressed Sparse Attention 2, FP4 KV caching, and SWA Bounded Replay to lower HBM/SSD and prefill costs.

Introduction

Long-horizon agent workloads drive input-heavy model usage where prefill cost and KV cache size become primary deployment bottlenecks. This paper's core insight is that combining an architecture tuned for low-prefill activation with aggressive cross-layer KV reuse and compact numeric formats can cut both in-memory (HBM) and persistent (SSD/host) KV footprints by substantial factors without sacrificing overall performance.

Key Findings
  • Architecture and compute balance: adopting a Causal Encoder–Decoder (CED) layout lets the system activate ~8B parameters per token during prefill and ~16B during decoding — reducing prefill compute for input-heavy agent workloads.
  • KV-cache compression techniques: Compressed Sparse Attention 2 (CSA2) enables cross-layer KV reuse and index sharing; combined with FP4 (E2M1/E4M3-style) KV storage, the global (HBM) KV footprint is reduced to ~890 bytes/token (≈1/4 vs prior generation).
  • Persistent-storage optimization: SWA Bounded Replay reconstructs missing SWA KV states by replaying only a recent window, cutting persistent KV footprint on SSD/host to ~1/8 of the prior model while avoiding full SWA persistence.
  • Scale and scope: DeepSeek-V4.1-Flash is a multimodal MoE backbone with 552B parameters, trained on a 45T-token multimodal corpus (sparse attention at 64K, context extended to 1M at 34T tokens), aiming at agentic and long-context multimodal tasks.
Who it's for and tradeoffs

Great fit if you operate or research long-horizon multimodal agents and need to reduce HBM/SSD KV costs at large scale while keeping high context lengths. It is attractive to infra teams optimizing large-model serving, and to researchers exploring sparse attention, MoE scaling, and numeric-compression tradeoffs. Look elsewhere if you need a small-footprint model for edge devices, if you cannot accept MoE/FP4-related engineering complexity, or if high-precision (non-quantized) KV storage is mandatory; the approach assumes infrastructure capable of managing MoE routing, FP4 compute paths, and replay-based persistent-state reconstruction.

Where it fits

This work sits between advances in long-context sparse-attention and deployment-focused compression: it targets the deployment bottlenecks (prefill compute, HBM bandwidth, SSD capacity) rather than raw benchmark SOTA, offering concrete reductions in KV storage and prefill activation for large-scale agentic workloads.

Method and notable mechanisms
  • CED layout: a 20-layer causal encoder + 20-layer decoder where decoder KV is projected from final encoder states to lower per-token activation during prefill.
  • CSA2: assigns static attention modes (Full/Reindex/Reuse) across layers to share main KV and reuse Top-K sparse indices, plus a hierarchical indexer to bound deeper index costs.
  • FP4 KV caching: low-bit KV storage format with per-channel scaling to compress HBM footprint.
  • SWA Bounded Replay: a deployment technique to avoid persisting full SWA KV by reconstructing recent windows on demand.

The paper emphasizes practical deployment tradeoffs and presents detailed design choices for teams aiming to lower the cost of serving extremely long contexts at scale.

Information

  • Websitearxiv.org
  • AuthorsAnyi Xu, B. Li, Bangcai Lin, Bing Xue, BingCheng Xian, Bingzheng Xu, Bochao Wu, Bowei Zhang, Boyi Deng, C. C. Yu
  • Published date2026/09/17

More Items

Estimates a model's probability of correctness by leveraging its past graded episodes: XConf recalls similar past cases and prompts the model to reflect and restate confidence, improving calibration and discrimination across reasoning, coding, multimodal QA, and agent tasks at much lower generation cost.

Defines recursive self-improvement (RSI) with an evaluation metric (Headroom-Closed Index) and a five-level autonomy roadmap—from executing human-specified improvements to recursively improving improvement mechanisms—while analyzing scenarios, verification challenges, and governance risks.

Investigates long-horizon memorization in language models by evaluating how complementary continual-learning mechanisms (data, function, weight anchors and low-rank allocation/LoRA) compose to reduce catastrophic forgetting across 100 sequential tasks; best composition raises final retention from 1.2% to 34.9%.