Speculative decoding turns one target-model forward pass into several output tokens by letting a small drafter propose multiple tokens that the target then verifies. The practical limit for speedups has been how good and efficient the drafter itself can be. DFlash 2 pushes that limit by drafting every position in a block in one pass and recovering coherence with a lightweight selector and architectural tweaks, yielding a measurable lift in accepted draft length and throughput without changing the verified output distribution.
Key Capabilities
- Parallel block drafting: predicts a full block of token candidates in one forward pass instead of autoregressively producing tokens one-by-one, allowing multiple tokens to be verified per target pass.
- Candidate-path selection: keeps top candidates at each position and traces a coherent path through them so greedy verification remains lossless and sampled outputs preserve the target distribution.
- Architectural fixes for block stability: two-tap dynamic convolutions reduce draft decay toward the end of the block, improving acceptance at longer draft widths.
- Measured gains: across several benchmarks DFlash 2 raises mean acceptance length (mean 4.80 vs MTP 4.28 and DSpark 3.62) and boosts throughput on single-GPU runs (examples: GSM8K concurrency=1 236.1 tok/s vs autoregressive 68.9 tok/s, ~3.43×).
Who it fits and trade-offs
Great fit if you run a transformer-based text generation stack and need higher token throughput or lower latency per completed token on real workloads while keeping the target-model outputs unchanged. It integrates as a drafter in speculative-decoding servers (examples: SGLang, vLLM) and is designed to pair with Qwen3.8-27B as the verified target.
Look elsewhere if you cannot modify your inference path to include a draft+verify loop, if your deployment forbids any extra model in the pipeline, or if memory/quantization constraints force very small verify widths (documentation notes quantized kernels may prefer smaller block sizes). DFlash 2 adds modest draft-model compute and integration complexity in exchange for multi× throughput gains on supported hardware.
