Most speedup attempts for autoregressive LLMs trade quality for parallelism. This paper shows a different path: keep the AR model as the ground-truth distribution and add a small diffusion pathway that drafts multiple tokens in parallel. Because the AR weights remain the final verifier, generation remains statistically identical to standard decoding while Tokens-Per-Forward-pass rises substantially.
Key Findings
- Lossless parallel drafting: A discrete diffusion adapter (implemented as LoRA-like lightweight weights) drafts a block of tokens, then the AR model verifies them in a single forward pass — if accepted, multiple tokens advance at once, yielding true lossless acceleration.
- Diffusion Distillation + Ψ-Spec samplers: The diffusion weights are trained via a short distillation stage that fits into existing pipelines; Ψ-Spec samplers coordinate drafting and verification to preserve the AR distribution exactly while enabling parallel steps.
- Empirical speedups and competitiveness: Uno achieves up to 3× throughput improvement over the baseline AR model (varies by batch size and hardware), shows 2.2–2.5× latency speedups at batch size 1 in reported experiments, and outperforms several diffusion-LLM baselines on reasoning, coding, and agentic tool-use benchmarks.
- Practical RL benefit: Using Uno during RL rollouts can cut end-to-end RL training time (token-generation phase) significantly (reported up to ~40% faster in rollout-dominated workloads) because drafting accelerators remain effective even as AR weights change.
Who it's for and trade-offs
Great fit if you need higher token throughput from existing open autoregressive LLMs without altering their output distribution — e.g., high-concurrency inference services, RL rollout generation, or augmenting pretrained models with minimal extra parameters. Look elsewhere if you cannot modify inference stacks to add the diffusion adapter/Ψ-Spec logic, if hardware/serving constraints prevent multi-token verification patterns, or if you require methods that change the underlying generation distribution (pure approximate speedups may be simpler).
Where it fits
Uno sits between speculative decoding and pure diffusion LLMs: like speculative decoding it seeks parallelism but does not require a separate draft model; unlike pure d-LLMs it preserves the AR distribution exactly. It is most relevant for LLM deployments where maintaining identical output quality is essential while reducing real-world token-generation cost.