A 4B model usually has a fixed compute budget per token; Dory makes that budget adjustable at inference time. Rather than switching to a larger checkpoint, it can repeatedly apply a shared block of layers, spending more computation inside the model and often producing a shorter answer. That makes loop count a practical quality-versus-cost control, not just an architectural detail.
Key Capabilities
- Elastic depth: One checkpoint supports different recurrent-loop counts. NVIDIA recommends two as a starting point; one minimizes compute per token, while four can help on agentic tasks. In NVIDIA’s reported SWE-bench Verified results, the score rises from 50.3 at one loop to 61.8 at four, though results depend on the evaluation setup.
- Long-context reasoning: The model supports up to 256K tokens, and combines sliding-window and full-attention layers. This is useful when a task needs substantial context without moving to a much larger parameter count.
- Flexible memory and formats: Last-loop KV caching can keep cache use near the one-loop footprint even when the model runs multiple loops. BF16 is the reference checkpoint; MXFP8, NVFP4, and a Q5_K_M GGUF export offer lower-memory alternatives.
- Broad post-training: After pretraining, it received supervised fine-tuning and multi-environment reinforcement learning for tasks including code, math, instruction following, and tool use. This supports general reasoning and agent workflows, but does not make it a specialist model for every domain.
Who It’s For
A good fit for teams evaluating compact, English-language models that need adjustable inference effort, long context, or agentic coding performance. Look elsewhere if you need multilingual support, multimodal input, or consistent maximum accuracy without added inference computation. Benchmark figures are reported by NVIDIA under its evaluation harness, so validate them against your own workload.
