A compact parameter count does not necessarily mean a model does little computation per token. This model’s defining trade-off is to reuse its middle Transformer layers three times: it keeps a 26.4B-parameter weight set, while looping raises effective active computation to about 5.4B parameters per token. That makes it an unusual attempt to bring stronger reasoning and agentic coding performance to hardware that can run a roughly 20GB checkpoint locally.
Key Capabilities
- Text and image inputs share a multimodal processing path, with text generation as the output. The one-million-token context window and visual input support suit tasks involving long documents, screenshots, and code-related images.
- Sparse MoE routing selects 8 of 128 experts per token, while repeated middle layers add computation without storing a separate set of weights for each pass. The result is a compute-for-memory trade-off, not a free speedup: inference speed depends on runtime and hardware.
- The model is designed for coding, tool use, and general agent workflows as well as conversation. Its reported evaluations are especially strong on several coding and agent benchmarks, though benchmark results do not guarantee reliability on a particular project.
- Open weights under Apache 2.0 allow local inference and downstream adaptation. The mixed NVFP4/MXFP8 checkpoint is about 20GB, with 32GB or more of system memory or VRAM recommended for supported local setups.
Who It’s For—and the Trade-Offs
A good fit for developers who want an open-weight multimodal model for local coding agents, long-context analysis, or experimentation with looped transformer architectures. Look elsewhere if dependable tool execution, consistently strong non-English output, or factual accuracy is critical without substantial verification: the model card notes weaker performance outside English, hallucinations, repetitive reasoning, and the possibility of destructive agent actions. Run agentic tasks in a sandbox and require confirmation for irreversible changes.
Why the Looping Matters
The middle 12 of 24 layers are reused for three passes, yielding 48 layer applications per token from the 24-layer weight set. This lets the model spend more computation on each token without increasing the stored parameter count in the same way as a deeper, independently weighted network. The trade-off is higher compute per token and less transparent reasoning: internal processing is not fully visible in generated text, so a visible chain of thought should not be treated as a complete account of the model’s computation.