AI Model2026
A sparsely activated Mixture-of-Experts (MoE) causal language model with 16B total parameters and 2.8B active parameters per token, released with end-to-end checkpoints and training recipe; trained on AMD Instinct GPUs and licensed for research use.
Converts image-content removal into a contrastive on-policy self-distillation signal: the EMA teacher produces next-token distributions with and without image content, uses their log-probability differences to sharpen visual-grounded candidates, and distills that full-distribution target into the student—no external teacher or extra inference cost.
AI Model2026
A text-only open-weight MOE code model (35B total, 3B active) fine-tuned with SFT+RL for agentic coding; achieves strong agentic-code benchmarks, supports 262k context and deployment via Transformers/vLLM; vision weights are not included.
Empirically studies how transformer-based native multimodal pre-training scales under fixed compute, deriving compute- and data-allocation power laws and an efficiency frontier for model size, token count, and data mixture; evaluates cross-modal transfer and multimodal in-context learning.
AI Model2026
Delivers image and video understanding plus a built-in event‑gated streaming gate — a unified 4B multimodal foundation model that uses codec-aligned tokenization to cut visual tokens by >75% and yield up to 3.5× wall‑clock inference speedup for streaming and long‑horizon video tasks.
AI Model2026
Generates text from text, image, or audio inputs using a native multimodal, Mixture-of-Experts autoregressive transformer (276B total / 12B active) with up to 1M-token context; targeted at conversational, agentic, coding and multimodal applications.
AI Model2026
Open-weight multimodal Mixture-of-Experts LLM with native vision and a 1,048,576-token context window. 2.8T parameters (104B activated), MXFP4 quantization, released for agentic long-horizon coding, knowledge work, and vision-in-the-loop workflows.
AI Model2026
GGUF-quantized build of Moonshot AI's Kimi K3 for local inference: MXFP4-aware quantization, image-text-to-text pipeline support, native vision and a 1,048,576-token context window. Intended for local GGUF runtimes (vLLM, SGLang, TokenSpeed) with Kimi K3 license constraints.
AI Model2026
A 350M-parameter multilingual bidirectional masked-language encoder with an 8,192-token context window, intended for fine-tuning on classification, token-level tasks, retrieval/reranking and semantic-similarity; optimized for long-context CPU inference and on-device use.
AI Model2026
A 2.6B causal LLM post-trained for agentic workloads and long-context on-device text generation. Key features: 128K context window and vocabulary, function-calling/tool use support, agentic RL/post-training pipeline, and optimized CPU/Apple inference and multiple deployment formats; suited for agents, RAG and long-context extraction.
AI Audio2026
Multilingual neural text-to-speech model (0.6B params) with zero-shot voice cloning and a bundled 44.1 kHz codec. Preview release targets 11 recommended languages and aims to deliver near-SOTA quality in a compact checkpoint suited for voice cloning and multilingual TTS prototypes.
AI Model2026
Provides a 750-billion-parameter multilingual Mixture-of-Experts (MoE) foundation language model optimized for long-context understanding, agentic workflows, and instruction following. Key features include a 262,144-token context window, speculative decoding (MTP/DSpark), 37B active parameters, 10-language support, and an Apache-2.0 license.