AIAny
Icon for item

Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories

A vision-language-action foundation model trained on 100k+ hours of real-world robot manipulation trajectories to follow natural-language instructions and adapt to downstream tasks with minimal fine-tuning. Uses a two-stage (pre-/post-) training recipe and a scalable auto-labeling pipeline; shows clear scaling benefits and state-of-the-art sim-to-real transfer on standard benchmarks.

Introduction

The most surprising result here is empirical: scaling pre-training to 100k+ hours of real-world manipulation trajectories, combined with automatic natural-language annotation of state transitions, produces a VLA model whose improved pre-training behavior transfers directly into better out-of-the-box real-robot performance in unseen environments. In other words, more real trajectories + language-conditioned action supervision buys practical sim-to-real generalization rather than just higher offline metrics.

Key Findings
  • Large-scale real-world pre-training: the paper trains on over 100k hours of manipulation trajectories collected with UMI devices and uses an auto-labeling pipeline to convert clips into language-conditioned action supervision, providing rich behavior-conditioning at scale — this is presented as the main data-side lever. So what: it removes the dependence on handcrafted instruction labels and enables broad, generalizable action generation.
  • Two-stage recipe (pre-training → post-training): pre-training imbues broad action capabilities; post-training aligns the model to robot embodiments and imperative human instructions. So what: stronger pre-training models yield better zero-shot and few-shot real-robot performance, reducing downstream fine-tuning needs.
  • Empirical scaling and SOTA numbers: Xiaomi-Robotics-1 shows consistent improvements with more data and larger models and establishes new state-of-the-art results on benchmarks — 57.6% success on RoboCasa365 (previous best 46.6%) and an average score of 20.07 on RoboDojo (prior 13.07). So what: concrete, reproducible gains on standard embodied benchmarks indicate practical utility beyond paper claims.
Who it's for and trade-offs

Great fit if you are researching or building real-world robot policies that must follow natural-language instructions and you can access large-scale real trajectory data or want a strong pre-trained policy to fine-tune. Look elsewhere if your constraints are small-data-only or purely simulated datasets (the approach relies on massive real-world trajectories and an auto-labeling pipeline), or if you need guarantees about safety-critical control beyond what imitation-like pre-training provides.

Where it fits

This paper sits at the intersection of embodied AI, multimodal foundation models, and robotics engineering: it emphasizes data scale and language-conditioned action supervision as the primary scaling axis, complementing model-size and post-training alignment. For teams focused on sim-to-real transfer or robot foundation policies, it provides a concrete data+recipe template; for small labs without access to large real trajectory corpora, it signals why shared pre-trained robot policies matter.

Information

  • Websitearxiv.org
  • OrganizationsXiaomi Robotics
  • AuthorsJun Guo, Piaopiao Jin, Jason Li, Peiyan Li, Yingyan Li, Futeng Liu, Wanli Peng, Optimus Qin, Yifei Su, Nan Sun
  • Published date2026/07/16

More Items

Predicts an LLM agent's final success or failure from partial execution traces and halts runs when outcomes are confident to save per-task compute. Uses LightGBM success/failure classifiers on behavioral, textual, and reference features; cuts 13–26% steps and up to 44% input tokens on benchmarks.

Distills operational know‑how from ML GitHub repositories into compact, verified 'skills' that research agents can load and reuse. Produces a skill format (SKILL.md, references, scripts), the AREX‑Skill Library (5,000+ skills from 1,000 repos), and demonstrates sizable benchmark gains when agents use skills.

Learns generalizable World Action Models for robotic manipulation by scaling causal egocentric video pretraining and grounding learned dynamics with heterogeneous robot trajectories. Key features: a three-stage curriculum (video pretraining, video-action mid-training with a unified action representation, and target-robot specialization) and a Slow–Fast dual-system for 30 Hz real-time action prediction.