Why this matters
Most production agent stacks need a model that balances long-context capabilities, throughput, and cost. This release targets that tradeoff: a Mixture-of-Experts hybrid that exposes a small active working set (≈3B active) inside a 30B-parameter topology, plus an NVFP4 quantized checkpoint and external speculative drafters so you can route inexpensive, high-volume work to Lightning while reserving heavier models for planning.
Key Capabilities
- Hybrid MoE architecture (Mamba-2 + MoE + Attention) that yields a 30B total / ~3B active parameter profile — lower active compute for many agent sub-tasks.
- Very long context support (up to 1,000,000 tokens) for retrieval- and document-heavy workflows.
- Post-training NVFP4 quantized checkpoint with selective BF16 preservation for attention-related layers to retain accuracy while reducing memory and bandwidth.
- Speculative decoding ecosystem: DSpark, DFlash and MTP options to improve throughput in low/high-concurrency serving setups.
- Tool-calling and reasoning control baked into chat templates (reasoning ON/OFF flags) and parsers for agent integrations.
- Ready examples and deployment recipes for vLLM, TensorRT-LLM, SGLang and API-serving; tuned patterns for H100/GB200/DGX Spark hardware.
Who it's for — and tradeoffs
Great fit if you need a cost-conscious, long-context workhorse for agent systems: use Lightning as a sub-agent drafter, a local inference option on high-end consumer/GPU boxes, or a mid-tier model in a multi-model routing stack.
Look elsewhere if you need the highest possible single-query accuracy from an ultra-frontier dense model for open-ended research tasks, or if you cannot provision NVIDIA GPUs and the specialized MoE runtimes described here. Running full 1M-token context or multi-GPU expert-parallel setups increases infra complexity and storage for checkpoints despite the reduced active compute.
Where it fits
Positioned as an efficiency-first Nemotron 3.5 variant: more suited than small Nano checkpoints for agentic reasoning and long-context workflows, but cheaper-to-run (in many agent use cases) than very large dense frontier models. Use it as the “workhorse” layer in a cascade: fast speculative drafts and retrieval fusion, with larger models reserved for complex planning or verification.
