Language-conditioned robot policy that reuses a pretrained geometric foundation model and inserts a causal future predictor at an intermediate layer so the same backbone produces future 3D-aware features and action outputs, enabling geometry-aware temporal prediction with minimal architectural change.
Converts large-scale egocentric human videos into robot-format pseudo-action trajectories and introduces ACE-EGO-0, a VLA pretraining framework that unifies camera-space actions, morphology conditioning, and reliability-aware weighting to jointly learn from noisy human and high-quality robot data for improved robotic manipulation transfer.
Provides a harness that lets language models control embodied manipulation via iterative perception–reasoning–action loops, semantic action abstractions, and multimodal observations. Demonstrates distilling capabilities into a 4B open-source model with under 2K simulated trajectories and shows sim-to-real generalization.
Provides SimReady 3D assets for robotics and industrial simulation. The OpenUSD content is prepared for simulation workflows, helping teams assemble environments for physical AI.
Adapts pretrained Vision-Language-Action (VLA) models to new camera poses and robot embodiments from a single demonstration by performing weight-vector arithmetic that injects domain-specific information. Filters noise via subspace alignment of singular components; designed for one-shot adaptation under visual and embodiment shifts.
Provides a portable C++ inference runtime to deploy embodied AI models (vision–language–action and world–action) on heterogeneous robot hardware, enabling latency-first batch-1 closed-loop control. Key features include modular multi-rate layers, fused low-latency inference, and extensible head/IO plugins.
Provides a systematic benchmark and design roadmap for video-based world models to evaluate robot policies, introducing WMBench and GigaWorld-1 optimized for long-horizon, action-faithful rollouts. Offers controlled comparisons across model families, action encodings, and 324k+ simulated vs real rollouts, with code, models, and datasets released for reproducible evaluation.
Detects when an action-chunked VLA policy drifts from expected visual dynamics and triggers lightweight corrective replanning via a latent-space vision monitor and online gradient guidance; creates an event-driven adaptive action horizon without retraining the backbone.
Evaluates generalist robot manipulation policies across simulation and real-world settings using 42 sim tasks and 18 real tasks; measures generalization, memory, precision, long-horizon execution and open-vocabulary instruction following, and provides a cloud-accessible real-world evaluation system with XPolicyLab integration and a public leaderboard.
Trains cross-platform GUI agents by combining a Uni-GUI cross-platform dataset with platform-conditioned multi-teacher on-policy distillation, enabling a shared policy to adapt to new platforms while retaining platform-specific behaviors; suitable for research on continual GUI agent learning and cross-platform adaptation.
Autoregressively synthesizes long-horizon, playable video worlds conditioned on current state and user actions for real-time interaction. Ships as an open-source, full-stack framework covering data preparation, model architectures, training, inference acceleration, and deployment for interactive generative worlds.
Pretrains a DiT-based Mixture-of-Experts video foundation model for embodied intelligence by augmenting internet videos with robot-centric footage and using a multi-dimensional reward system to prioritize physical realism and task completion while scaling MoE for better capacity vs. inference trade-offs.