Explores asymmetric compute allocation in looped Mixture-of-Experts models by compressing the first loop visit's sequence and reallocating saved compute to enlarge hidden size, yielding consistent NLL reductions and predicted compute savings across 1.3B–7B configurations.
AI Model2026
Generates text conditioned on multimodal inputs (text, image, audio, video) with a 1M-token context window; an open-weights mixture-of-experts model aimed at long-horizon coding and agentic workflows, with NVFP4 checkpoints to reduce memory footprint.
AI Model2026
Provides open-weight, natively multimodal LLM optimized for long-horizon multi-step reasoning and coding/agentic workflows; 816B total / 42B active parameters, 1M-token context window, available as BF16 and NVFP4 checkpoints under the Muse Model License.