Category

Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learning

Analyzes how to build effective training environment distributions for multimodal agents and proposes Ability-aware Environment Selection (AES) and Hierarchical Difficulty Curriculum (HDC) to improve diversity and difficulty scheduling, yielding large relative gains in experiments.

WorldClaw: Agentic 3D Open-World Generation at Scale

Generates large-scale, explorable 3D open-world scenes from open-ended text prompts, producing editable instance-level assets and a consistent global terrain. Uses agentic planning to convert text into region/terrain/asset specifications and a coarse-to-fine pipeline for terrain construction, mesh reconstruction, and render-based refinement.

ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation

Orchestrates reasoning, external tool use, and native image generation under one unified multimodal agent policy via post-training. Introduces RAD-GRPO for agentic reinforcement fine-tuning and releases training data plus the full post-training infrastructure.

Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes

Systematically studies how language and vision interact during unified multimodal pretraining, identifies mechanisms that enable modality synergy versus competition, demonstrates the benefit of early joint training, and derives efficient pretraining recipes validated at scale.

GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?

Evaluates VLMs' ability to form global spatial awareness from long-horizon egocentric video. Introduces GST-Bench: a VQA benchmark with human-verified questions from 6,790 minutes of synthetic video, reveals a large gap (best zero-shot 42.68 vs human 79.08) and provides GST-Train dataset.

Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval

Generates retrieval-centric Chain-of-Thought (RC-CoT) over initially retrieved candidates to improve unified multimodal retrieval via reranking or full-corpus re-retrieval with a dual-mode embedder. Trains an embedder–adviser framework (UniME-R1) using mined hard negatives, supervised learning, and retrieval-oriented reinforcement learning.

SimWAM: A Simple World Action Model for End-to-End Autonomous Driving

Uses video generation only as a training signal to co-train a pretrained video expert and a lightweight action expert, then discards the video branch at inference to produce a low-latency end-to-end driving planner; enhanced with RL for compositional driving rewards.

YOLO-PEFT: Parameter-Efficient Fine-Tuning on YOLO Family

Turns adapter placement for PEFT on YOLO-family real-time detectors into an auditable constraint-planning problem that emits budgeted target-module plans or calibrated refusals; shows planner-selected RS-LoRA improves mAP and cuts peak training memory in evaluated detectors.

AdvFD: Boosting Visual Generation via Adversarial Fr'echet Distance Loss

Post-training distribution-level objective that augments static Fréchet-distance losses with an adversarially learned representation and a real-feature whitening step to stabilize min–max optimization and avoid trivial feature amplification; targets one-step image generator post-training.

AVA-Encoder: Towards Agent-Native Video Representation Learning

Encodes videos into a Film Knowledge Graph and reconstructs them to learn agent-native, editable video representations for agentic reasoning and manipulation. Uses agentic auto-encoding with dual-loop textual-gradient optimization, reports large reconstruction gains, and releases a benchmark and dataset.

StateFlow: Building, Evolving, and Accessing 3D World States for Previsualization

Builds an editable, persistent 3D world state to drive iterative previsualization for film, games, and design — enabling local edits and recombinations instead of one-shot video regeneration. Uses separate stages for state construction, state evolution, and state access, with render-feedback camera refinement.

Alaya-EVOKE: From Linear-Scaling Supervision to Endless World

Externalizes persistent scene state into a camera-indexed world bank and designs a long-horizon teacher whose sparse-attention supervision is distilled into a three-step student, enabling responsive, low-latency interactive long-horizon video generation with bounded denoiser context.