Trains end-to-end driving without human trajectory supervision by decoupling perception and action: DriveVFM distills multiple frozen vision foundation models into a single camera backbone, and DriveRL trains a privileged closed-loop RL teacher whose rollouts supervise a camera-only planner, yielding state-of-the-art closed-loop benchmark results.
Lets vision-language models control robots via a compact semantic action interface that maps intent to discrete action units; supports zero-shot use of closed-source VLMs, low-cost fine-tuning of open VLMs, and GUI-based demonstration collection.
Simulation-ready home dataset for embodied AI: CAD-based household scenes with configured physical properties and metadata, plus 1,000 robot trajectory episodes (RGB-D, HDF5/USDZ) for simulation training and evaluation under CC BY-NC-SA 4.0.
Provides 1,274 hours of head-mounted egocentric video paired with seven-point IMU arm tracking (24 Hz orientation; raw accel/gyro/mag on a subset), packaged for embodied-AI and egocentric-vision research. Key features: torso-relative pose via chest reference, separate Parquet IMU repo for efficient joins, CC-BY-4.0 license; heavy class skew and limited contributor diversity are important constraints.
Introduces physical rank consistency (PRC) and ActionPiece, a learned action tokenizer that preserves local physical relationships via joint supervision of representation learning and quantization, improving autoregressive vision-language-action policy performance across multiple benchmarks.
Generates humanoid robot motion references that preserve object contact locations/timing by solving windowed trajectory optimizations against contact targets in the object frame. Releases retargeted trajectories for two Unitree robots across 75 objects (≈13.9k robot–motion pairs); CC BY‑NC‑SA 4.0.
Uses a compact discrete-action interface (translation, rotation, gripper) to let frozen vision–language models control robots in closed-loop, and leverages in-context (one-shot) demonstrations to substantially improve manipulation success without robot-specific training.
Evaluates whether generative world models maintain consistent, controllable, and physically plausible simulated environments under exploration, interaction, and intervention. Introduces a six-level W1–W6 capability taxonomy across three tracks (video, spatial, embodied) with human A/B Arena and automated metrics to measure behavioral correctness.
Provides imagined interaction segments generated by world models for RoboTwin2.0 tasks, stored as fixed-length HDF5 chunks (21 observation frames, 20 actions, rewards and episode flags). Useful for training and evaluating world-model-based policies; currently limited to the RoboTwin2.0 subdataset.
Learns joint predictive visual dynamics and action generation for generalist robot manipulation, translating future-relevant visual representations into actions. Integrates a Mixture-of-Transformers coupling a video expert and action expert, a frozen vision-language model for semantics, 4D distillation, and Causal Imprint; pretrained on a 20K+ hour heterogeneous corpus.
Uses 360° panoramic observations to improve vision-and-language navigation by predicting longer action sequences, confidence-guided execution, and combined semantic–geometric panorama features; yields large SR gains on R2R-CE and RxR-CE Val-Unseen.
Records structural priors with skill-specific policies so a runtime agent can select and compose the version of each skill best suited to new states, improving out-of-distribution and compositional generalization for robot manipulation from few demonstrations.