AI Model2026
Generates and reasons about multimodal physical-world content—text, images, video, audio, and robot/action trajectories—conditioned on combinations of text, image, video and action inputs. The 64B “Super” variant targets Physical AI use cases and supports vLLM‑Omni, Diffusers, and action prediction.
Proposes TrOPD, a method that restricts token-level on-policy distillation to regions where teacher supervision is reliable to stabilize training under teacher–student distribution mismatch. Adds outlier handling (clipping, masking, forward-KL) and off-policy guidance; shows consistent gains on math reasoning, code generation and general benchmarks.
Studies small trainable adapters (PEFT) used as persistent personal models on top of large foundation models, analyzing three scaling axes—Scale Up, Scale Down, Scale Out—and introducing MinT, an infrastructure for adapter identity, provenance, evaluation, and serving.
Omnimodal world model that jointly processes and generates text, images, video, audio, and action trajectories for physical AI. Uses a mixture-of-transformers to combine autoregressive reasoning and diffusion-based multimodal generation; released open-source with checkpoints, datasets and benchmarks for robotics and simulation.
Trains a GPT-style causal Transformer on a 2-billion-frame retargeted motion corpus to enable zero-shot whole-body motion tracking and control. By scaling both data and model capacity, it tracks highly dynamic behaviors while generalizing to unseen motions; accepted to CVPR 2026.
Explores how training recipe — data composition, teacher guidance, and task mixture — shapes few-step distillation for text-to-image generation and instruction-guided image editing; introduces Qwen-Image-Flash and empirical findings that training pipeline organization matters as much as distillation objectives.
AI Model2026
Native multimodal model for image/text/video→text tasks with million‑token context support. Uses a sparse-attention operator to cut long‑context compute and latency, and targets agentic, coding, and long-horizon conversational workloads.
Removes the subspace of frequent, uninformative tokens that LLMs inject into text embeddings via the model's unembedding matrix. EmbedFilter is a lightweight linear transform that refines LLM-derived embeddings to improve zero‑shot semantic retrieval, enable dimensionality reduction, and speed up indexing; code on GitHub.
Analyzes the parameter-space geometry of on-policy distillation (OPD) for LLM training, showing OPD updates affect fewer weights, avoid principal directions, and rapidly lock into a low-dimensional update subspace. Compares OPD with supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR) and studies implications for optimization and objective mixing.
Models visual preference as distributions over rubric scores and introduces Z-Reward, a teacher–student framework that decouples reasoning-heavy judgment (teacher trained with GDSO) from efficient deployment (student via RISD). Demonstrates higher human-preference accuracy and works as a differentiable reward for text-to-image optimization.
AI Agent Papers2026
Lets a single LLM simultaneously act as agent and environment to bootstrap co-evolutional training — using state-prediction process rewards (World-In-Agent) and failure-mode retrieval (Agent-In-World) to reshape training data; reports ~4% average benchmark gain.
AI Model2026
Generates text from interleaved text, image, and short-video inputs using discrete diffusion and block‑autoregressive multi‑canvas sampling; built on a sparse MoE (8/128) Gemma 4 backbone and optimized for low‑latency inference and very long contexts (up to 256K tokens).