AIAny
Icon for item

Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories

A vision-language-action foundation model trained on 100k+ hours of real-world robot manipulation trajectories to follow natural-language instructions and adapt to downstream tasks with minimal fine-tuning. Uses a two-stage (pre-/post-) training recipe and a scalable auto-labeling pipeline; shows clear scaling benefits and state-of-the-art sim-to-real transfer on standard benchmarks.

Introduction

The most surprising result here is empirical: scaling pre-training to 100k+ hours of real-world manipulation trajectories, combined with automatic natural-language annotation of state transitions, produces a VLA model whose improved pre-training behavior transfers directly into better out-of-the-box real-robot performance in unseen environments. In other words, more real trajectories + language-conditioned action supervision buys practical sim-to-real generalization rather than just higher offline metrics.

Key Findings
  • Large-scale real-world pre-training: the paper trains on over 100k hours of manipulation trajectories collected with UMI devices and uses an auto-labeling pipeline to convert clips into language-conditioned action supervision, providing rich behavior-conditioning at scale — this is presented as the main data-side lever. So what: it removes the dependence on handcrafted instruction labels and enables broad, generalizable action generation.
  • Two-stage recipe (pre-training → post-training): pre-training imbues broad action capabilities; post-training aligns the model to robot embodiments and imperative human instructions. So what: stronger pre-training models yield better zero-shot and few-shot real-robot performance, reducing downstream fine-tuning needs.
  • Empirical scaling and SOTA numbers: Xiaomi-Robotics-1 shows consistent improvements with more data and larger models and establishes new state-of-the-art results on benchmarks — 57.6% success on RoboCasa365 (previous best 46.6%) and an average score of 20.07 on RoboDojo (prior 13.07). So what: concrete, reproducible gains on standard embodied benchmarks indicate practical utility beyond paper claims.
Who it's for and trade-offs

Great fit if you are researching or building real-world robot policies that must follow natural-language instructions and you can access large-scale real trajectory data or want a strong pre-trained policy to fine-tune. Look elsewhere if your constraints are small-data-only or purely simulated datasets (the approach relies on massive real-world trajectories and an auto-labeling pipeline), or if you need guarantees about safety-critical control beyond what imitation-like pre-training provides.

Where it fits

This paper sits at the intersection of embodied AI, multimodal foundation models, and robotics engineering: it emphasizes data scale and language-conditioned action supervision as the primary scaling axis, complementing model-size and post-training alignment. For teams focused on sim-to-real transfer or robot foundation policies, it provides a concrete data+recipe template; for small labs without access to large real trajectory corpora, it signals why shared pre-trained robot policies matter.

Information

  • Websitearxiv.org
  • OrganizationsXiaomi Robotics
  • AuthorsJun Guo, Piaopiao Jin, Jason Li, Peiyan Li, Yingyan Li, Futeng Liu, Wanli Peng, Optimus Qin, Yifei Su, Nan Sun …
  • Published date2026/07/16

More Items

Evaluates whether vision–language models can convert 2D observations into precise 3D control by introducing RoTA-Bench, an interactive projection-alignment benchmark that synthesizes silhouette-preserving 3D objects, guarantees solvable initializations, and provides 674 tasks with a stratified 100-task test set.

Answers spatial choice, scalar, and point questions by classifying among discretized candidates using pretrained attention projections from a Qwen3.5-4B backbone instead of autoregressive text generation. Trains on converted spatial QA data, adopts coarse-to-fine refinement and joint multi-stage supervision, and reports higher accuracy with several-fold faster inference than autoregressive baselines.

Benchmark for evaluating agents' ability to deliver working AI systems in realistic customer environments: 95 end-to-end delivery tasks with scripted stakeholders and production-like systems, scored by improvement to the customer's business metric and penalized for stakeholder contact.