Studies looping shared transformer layers in Mixture-of-Experts models under matched budgets and proposes SMELT: loop the middle half twice while matching per-token FLOPs, non-embedding parameters, and KV cache. Shows 6.8–18.0% training-FLOPs savings on the compute-optimal frontier, stronger downstream gains on code and long-context tasks.
Builds high-fidelity image generators by pairing a 6B Diffusion Transformer with a frozen LLaDA2.0-Mini vision-language module, relying on extensive image-only pretraining and mid-training; model weights, training code, and recipes are released.
Trains a world-action model that predicts future visual states to guide zero-shot robotic manipulation; introduces CoAE, SVP, IDM and KASO to pretrain generative and action components from scratch on manipulation data, scaling up to 30,000 hours and improving zero-shot success.
Trains end-to-end driving without human trajectory supervision by decoupling perception and action: DriveVFM distills multiple frozen vision foundation models into a single camera backbone, and DriveRL trains a privileged closed-loop RL teacher whose rollouts supervise a camera-only planner, yielding state-of-the-art closed-loop benchmark results.
AI Model2026
A multimodal, agentic LLM optimized for long‑horizon, visually grounded workflows — capable of operating browsers and terminals and autonomously executing and testing code. Open‑source weights are available and the family ships in mini, Pro and Max variants for different compute/quality tradeoffs.
AI Agent Papers2026
Explores a practical mechanism for recursive self-improvement by post-training LLMs: uses a routing harness to record agent executions and convert traces into curriculum-guided supervised fine-tuning and on-policy distillation data, closing an evaluation-selection-update loop and improving benchmark performance.
Adds a Next Concept Prediction objective that predicts discrete, product-quantized latent concepts spanning multiple tokens alongside next-token prediction, yielding faster pretraining convergence and measurable downstream gains with an 8.9B-parameter latent-space LLM.
AI Audio2026
Ruibin Yuan, Hanfeng Lin +56Multimodal Art Projection (MAP), Hong Kong University of Science and Technology (HKUST)
Generates full songs from lyrics and a style prompt, producing vocals and accompaniment and exporting editable symbolic scores for melody/chord control. Uses symbolic planning with agentic multi-turn editing and runs local 48 kHz inference on a 24GB GPU.
AI Model2026
A multimodal Mixture-of-Experts foundation model with a million-token context window; uses a causal encoder–decoder layout and aggressive KV-cache compression (~890 bytes/token) to limit per-token activation to 8B/16B—designed for long-context, agentic, and multimodal workloads.
An 8B native unified multimodal model that understands, reasons about, and generates high-fidelity visual content without a separate visual encoder or VAE—focused on spatially coherent patch reconstruction, native 4K generation, multi-expert distillation, and improved text rendering and image editing.
AI Model2026
Preview agentic language model for research and engineering workflows that turns research questions into executable, verifiable workflows via tool use and long-context reasoning; built on a 744B-parameter MoE (GLM-5.2) with MIT-licensed BF16 and FP8 checkpoints.
Presents an open 7B dense foundation model for mathematical reasoning and tool-assisted web search, supporting 256K-token context. Key features: hybrid gated sliding-window + global attention for efficient long-context training, FP8 Muon optimizer, progressive 16K→64K→256K curriculum, and open-source weights, data, and recipes.