AIAny
AI Model2026
Icon for item

Nemotron-Labs-Audex-30B-A3B

Adds discrete audio tokens and an audio encoder to a 30B MoE text LLM so a single model can perform ASR, speech translation, TTS, text-to-audio and speech-to-speech while preserving text reasoning and long-context capabilities; supports thinking/instruct modes and up to 1M-token context.

Introduction

Bringing discrete audio tokens and a dedicated audio encoder into a 30B Mixture-of-Experts text LLM enables unified handling of audio and text tasks without regressing on text intelligence. The design aims to let one model answer audio understanding queries, transcribe and translate speech, synthesize speech, and generate audio while keeping the backbone's reasoning, alignment, and long-context behavior.

Key Capabilities
  • Unified modality support: Extends a Nemotron-Cascade-2 30B MoE backbone with an expanded vocabulary for discrete audio tokens and audio encoders to support audio-in / audio-out workflows (ASR, speech translation, audio understanding, text-to-speech, text-to-audio, speech-to-speech).
  • Two inference pathways: Optimized recipes for vLLM-based streaming/offline inference and Hugging Face / transformers-based runs; recommends vLLM for offline audio QA and provides scripts for both server and batch use.
  • Large context & thinking mode: Supports up to 1,000,000-token context and a special "thinking" mode (internal reasoning delimited by <think> tags) plus an instruct (non-thinking) mode.
  • Practical engineering: Provides conversion and inference scripts (vLLM, Megatron-LM native inference, transformers >=4.53), and recommended sampling settings per audio task to balance fidelity and speed.
Who it's for and tradeoffs

Great fit if you need a single, locally runnable model that covers both advanced text reasoning and a wide range of audio tasks (ASR, translation, TTS, TTA, S2S) and can integrate with vLLM or HF transformers pipelines. Look elsewhere if you require a permissive commercial license (this release uses NVIDIA's OneWay Noncommercial License) or extremely small, low-latency edge deployments—the model is large (30B MoE with ~3B activated params) and optimized for multi-GPU / tensor-parallel setups. The release includes recommended inference setups and benchmark notes, but reproducing exact quality may require the provided XCodec assets and enhancement VAE for highest audio fidelity.

Information

  • Websitehuggingface.co
  • OrganizationsNVIDIA
  • AuthorsZhifeng Kong, Sang-gil Lee, Jaehyeon Kim, Boxin Wang, Zihan Liu, Sungwon Kim, Yang Chen, Arushi Goel, Rajarshi Roy, Wenliang Dai …
  • Published date2026/06/08

More Items

Hugging Face
AI Video2026

Distills MiniMax H3 video generation into 2 or 4 steps, with an option to predict pixels directly and skip VAE decoding. Supports synchronized audio and first- or last-frame conditioning.

Hugging Face
AI Model2026

Generates video with audio from prompts using a MiniMax H3 fine-tune, with an uncensored tuning approach. Optional quantized checkpoints reduce memory demands; the weights are large and require local inference.

Hugging Face
AI Model2026

Uses a LoRA edit to reduce Inkling 2 Mini’s refusal behavior and provides GGUF builds for local use. It accepts text and images; evaluations show refusals drop, while MMLU falls by 1.5 points.