AIAny
Icon for item

Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories

A vision-language-action foundation model trained on 100k+ hours of real-world robot manipulation trajectories to follow natural-language instructions and adapt to downstream tasks with minimal fine-tuning. Uses a two-stage (pre-/post-) training recipe and a scalable auto-labeling pipeline; shows clear scaling benefits and state-of-the-art sim-to-real transfer on standard benchmarks.

Introduction

The most surprising result here is empirical: scaling pre-training to 100k+ hours of real-world manipulation trajectories, combined with automatic natural-language annotation of state transitions, produces a VLA model whose improved pre-training behavior transfers directly into better out-of-the-box real-robot performance in unseen environments. In other words, more real trajectories + language-conditioned action supervision buys practical sim-to-real generalization rather than just higher offline metrics.

Key Findings
  • Large-scale real-world pre-training: the paper trains on over 100k hours of manipulation trajectories collected with UMI devices and uses an auto-labeling pipeline to convert clips into language-conditioned action supervision, providing rich behavior-conditioning at scale — this is presented as the main data-side lever. So what: it removes the dependence on handcrafted instruction labels and enables broad, generalizable action generation.
  • Two-stage recipe (pre-training → post-training): pre-training imbues broad action capabilities; post-training aligns the model to robot embodiments and imperative human instructions. So what: stronger pre-training models yield better zero-shot and few-shot real-robot performance, reducing downstream fine-tuning needs.
  • Empirical scaling and SOTA numbers: Xiaomi-Robotics-1 shows consistent improvements with more data and larger models and establishes new state-of-the-art results on benchmarks — 57.6% success on RoboCasa365 (previous best 46.6%) and an average score of 20.07 on RoboDojo (prior 13.07). So what: concrete, reproducible gains on standard embodied benchmarks indicate practical utility beyond paper claims.
Who it's for and trade-offs

Great fit if you are researching or building real-world robot policies that must follow natural-language instructions and you can access large-scale real trajectory data or want a strong pre-trained policy to fine-tune. Look elsewhere if your constraints are small-data-only or purely simulated datasets (the approach relies on massive real-world trajectories and an auto-labeling pipeline), or if you need guarantees about safety-critical control beyond what imitation-like pre-training provides.

Where it fits

This paper sits at the intersection of embodied AI, multimodal foundation models, and robotics engineering: it emphasizes data scale and language-conditioned action supervision as the primary scaling axis, complementing model-size and post-training alignment. For teams focused on sim-to-real transfer or robot foundation policies, it provides a concrete data+recipe template; for small labs without access to large real trajectory corpora, it signals why shared pre-trained robot policies matter.

Information

  • Websitearxiv.org
  • OrganizationsXiaomi Robotics
  • AuthorsJun Guo, Piaopiao Jin, Jason Li, Peiyan Li, Yingyan Li, Futeng Liu, Wanli Peng, Optimus Qin, Yifei Su, Nan Sun
  • Published date2026/07/16

More Items

Analyzes adversarial weaknesses of World-Action Models (WAMs) via BadWAM, a framework that crafts visual perturbations to decouple a model’s imagined future from its executed actions. Introduces two attack modes—action-only (disruptive) and imagination-preserving (stealthy)—and shows large drops in closed-loop task success (e.g., 96.5%→43.1%).

Turns fragile, implicit search progress into explicit, persistent, shared state for multi-agent information seeking — externalizes progress as Frontier Task, Evidence Graph, Coverage Map and Failure Memory, and uses pipeline-parallel scheduling plus a middleware harness to avoid repeated failed searches and improve utilization and throughput.

Converts completed on-policy trajectories into natural-language 'hindsight skills' and converts the skill-induced action probability shifts into a dense token-level on-policy distillation signal, jointly optimized with outcome-based RL to improve sample efficiency and long-horizon agent behavior.