Lower-precision weights do not automatically mean a smaller model fits on ordinary hardware: this checkpoint still occupies about 400 GiB and is designed for multi-GPU Blackwell systems. Its main value is targeted quantization of routed experts while retaining higher precision in other components, rather than reducing every part of the model uniformly.
Key Capabilities
- Routed MoE expert projections use NVFP4 weights and activations, while Engram lookup tables use MXFP4. Attention, vision, shared experts, and other excluded components retain their source precision, limiting the scope of quantization changes.
- The underlying model accepts text and images and supports context lengths up to one million tokens. Its Causal Encoder-Decoder design activates 8B parameters during prefill and 16B during decoding, from a 552B backbone plus conditional memory.
- Reported vLLM evaluations are close to the source checkpoint across six benchmarks: for example, GPQA Diamond scores 92.93% versus 92.77%, while Terminal-Bench 2.1 scores 81.11% versus 81.81%. These results suggest broadly preserved benchmark performance, not identical outputs.
- Deployment is supported through vLLM and SGLang, but vLLM requires MXFP4 Engram support. The published validation used four NVIDIA GB300 GPUs; this is a specialized serving checkpoint, not a lightweight local model.
Who It’s For
A good fit for teams already operating Blackwell-based inference infrastructure that want a pre-quantized checkpoint for agent, chatbot, or RAG workloads. Look elsewhere if you need consumer-GPU deployment, a small download, or a model whose behavior is guaranteed to match the original: Engram conversion is lossy, and the base model can produce biased, toxic, or inaccurate output.
