AIAny
AI Model2026
Icon for item

NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4

Deployment-optimized hybrid MoE LLM (75B total / 9.3B active) produced via Iterative Puzzle compression and Multi-Token Prediction to double server throughput and raise single-GPU concurrency; designed for multilingual reasoning, long-context generation, and high-volume agentic/chat deployments.

Introduction

Why this matters

Many production teams face a trade-off: keep a very large MoE model for peak accuracy, or compress it for feasible, high-throughput serving. This model targets the middle ground — preserving Nemotron‑3‑Super’s reasoning and long-context strengths while cutting the active inference budget and improving real-world throughput and concurrency.

Key Capabilities
  • Architecture and scale: hybrid Mamba2‑Transformer LatentMoE design; compressed from ~120.7B total / 12.8B active to 75.3B total / 9.3B active parameters.
  • Inference optimizations: Iterative Puzzle compression + knowledge distillation + RL recovery + post-training quantization (NVFP4 for Blackwell, FP8 for Hopper) and continued MTP training to raise speculative decoding efficiency.
  • Performance outcomes: ~2× higher server throughput on a single 8×B200 node at matched user-throughput constraints and sustainable 1M-token single‑H100 concurrency rising from 1 to 8 requests in tested configs.
  • Reasoning & capabilities: retains strong scores across reasoning, coding, multilingual and long-context benchmarks (examples: MMLU-Pro ~82, long-context RULER @1M ~93.2).
  • Operational features: Multi-Token Prediction (MTP) support, up to 1M token context (default HF config: 256k), recommended runtime engines include vLLM and Hugging Face Transformers with trust_remote_code enabled.
Who it's for and trade-offs

Great fit if:

  • You need a high-quality reasoning/chat model that is tuned for interactive, high-concurrency serving and long contexts (RAG, agents, multi-turn technical assistants).
  • Your infra targets NVIDIA GPUs (Blackwell/Hopper/H100) and you can use NVFP4/FP8 optimized checkpoints and vLLM or optimized TF runtimes.

Look elsewhere if:

  • You require a fully open permissive license (this uses OpenMDW‑1.1 which has specific governing terms).
  • Your deployment must avoid vendor-provided toolchains or trust_remote_code usage, or you cannot allocate the GPU memory needed for very long contexts at scale.
Deployment notes
  • Serving examples: vLLM serve commands and Transformers examples are provided; recommended tensor-parallel sizes and MTP speculative token settings are documented in the model card.
  • Hardware: tested on NVIDIA H100/B200 and optimized for Blackwell/Hopper microarchitectures with NVFP4/FP8 variants.
  • Safety & governance: released under OpenMDW‑1.1; NVIDIA provides safety, bias and privacy subcards and recommends V‑model testing and risk audits before commercial use.

Information

  • Websitehuggingface.co
  • OrganizationsNVIDIA, NVIDIA NeMo
  • Published date2026/07/06

Categories

More Items

Hugging Face
AI Model2026

Runs text generation, summarization, and question answering directly on Windows devices as a 2.3B-parameter instruction-tuned model. It supports eight languages and a 32K context window, trading broad world knowledge and deep reasoning for local execution and privacy.

Hugging Face
AI Model2026

Handles text, image, and video understanding with sparse attention designed for contexts up to 1 million tokens. It has 770B total parameters with 49B activated per token, targeting long-document analysis, multimodal reasoning, and extended workflows.

Hugging Face
AI Model2026

Estimates metric depth for every pixel in a photo, video, or live feed, with sharp object boundaries and stable results across frames. Multiple model sizes are available; weights are restricted to non-commercial research and academic development.