Open egocentric multimodal dataset for embodied AI and robot learning captured on commodity iPhone Pro: ~200 hours and ~10M RGB frames with LiDAR depth, ARKit 6‑DoF poses, IMU, two‑hand MANO mocap, room meshes, and hierarchical action captions.
Performs fast, high-quality vision–language grounding: given an image plus a natural-language prompt it returns bounding boxes or points for referred objects. Uses Parallel Box Decoding for parallel coordinate prediction (higher throughput) and targets research/non-commercial use.
Trains a GPT-style causal Transformer on a 2-billion-frame retargeted motion corpus to enable zero-shot whole-body motion tracking and control. By scaling both data and model capacity, it tracks highly dynamic behaviors while generalizing to unseen motions; accepted to CVPR 2026.
Simulates egocentric, embodied human–world interactions and enables customizable, self-evolving local scenes by defining anchor views and text-driven evolution. Uses exogenous viewpoints and full-body motion supervision to improve spatial grounding and interaction consistency.
A benchmark that evaluates interactive spatial reasoning for multimodal agents in realistic tasks. It unifies eight heterogeneous simulators under a simulator-agnostic protocol, provides 760 human-annotated tasks with vision-only partial observability, and uses text-based actions plus terminal-state verification to measure task success.
Provides 1000+ hours of high-precision optical motion-capture for humanoid robotics and embodied AI, including full-body skeleton, 20+DoF hands, object 6D, and multi-view video at 120 Hz. Sub-mm spatial accuracy, BVH/CSV/NPZ outputs and Unitree G1 retargets; ideal for imitation learning and sim-to-real, with some raw captures gated by license.
Learns, maintains, and runs unified world models for Physical AI using a cross-embodiment pretraining curriculum and a hybrid linear temporal-attention architecture. Emphasizes long-horizon state persistence, theoretical bounds on error accumulation, and deployment-aware low-latency inference for real-world embodied agents.
Language-conditioned robot policy that reuses a pretrained geometric foundation model and inserts a causal future predictor at an intermediate layer so the same backbone produces future 3D-aware features and action outputs, enabling geometry-aware temporal prediction with minimal architectural change.
Converts large-scale egocentric human videos into robot-format pseudo-action trajectories and introduces ACE-EGO-0, a VLA pretraining framework that unifies camera-space actions, morphology conditioning, and reliability-aware weighting to jointly learn from noisy human and high-quality robot data for improved robotic manipulation transfer.
Provides a harness that lets language models control embodied manipulation via iterative perception–reasoning–action loops, semantic action abstractions, and multimodal observations. Demonstrates distilling capabilities into a 4B open-source model with under 2K simulated trajectories and shows sim-to-real generalization.
Adapts pretrained Vision-Language-Action (VLA) models to new camera poses and robot embodiments from a single demonstration by performing weight-vector arithmetic that injects domain-specific information. Filters noise via subspace alignment of singular components; designed for one-shot adaptation under visual and embodiment shifts.
Provides a portable C++ inference runtime to deploy embodied AI models (vision–language–action and world–action) on heterogeneous robot hardware, enabling latency-first batch-1 closed-loop control. Key features include modular multi-rate layers, fused low-latency inference, and extensible head/IO plugins.