Provides a portable C++ inference runtime to deploy embodied AI models (vision–language–action and world–action) on heterogeneous robot hardware, enabling latency-first batch-1 closed-loop control. Key features include modular multi-rate layers, fused low-latency inference, and extensible head/IO plugins.
Provides a systematic benchmark and design roadmap for video-based world models to evaluate robot policies, introducing WMBench and GigaWorld-1 optimized for long-horizon, action-faithful rollouts. Offers controlled comparisons across model families, action encodings, and 324k+ simulated vs real rollouts, with code, models, and datasets released for reproducible evaluation.
Detects when an action-chunked VLA policy drifts from expected visual dynamics and triggers lightweight corrective replanning via a latent-space vision monitor and online gradient guidance; creates an event-driven adaptive action horizon without retraining the backbone.
Evaluates generalist robot manipulation policies across simulation and real-world settings using 42 sim tasks and 18 real tasks; measures generalization, memory, precision, long-horizon execution and open-vocabulary instruction following, and provides a cloud-accessible real-world evaluation system with XPolicyLab integration and a public leaderboard.
Trains cross-platform GUI agents by combining a Uni-GUI cross-platform dataset with platform-conditioned multi-teacher on-policy distillation, enabling a shared policy to adapt to new platforms while retaining platform-specific behaviors; suitable for research on continual GUI agent learning and cross-platform adaptation.
Autoregressively synthesizes long-horizon, playable video worlds conditioned on current state and user actions for real-time interaction. Ships as an open-source, full-stack framework covering data preparation, model architectures, training, inference acceleration, and deployment for interactive generative worlds.
Pretrains a DiT-based Mixture-of-Experts video foundation model for embodied intelligence by augmenting internet videos with robot-centric footage and using a multi-dimensional reward system to prioritize physical realism and task completion while scaling MoE for better capacity vs. inference trade-offs.
Generates videos from text and image+text prompts using a 30B Mixture-of-Experts model tuned for embodied intelligence; includes a refiner and structured prompt rewriter, and supports diffusers/SGLang runtimes with multi-GPU inference.
Reconstructs historical experience into latent memory tokens and weaves short- and long-term latent memories directly into vision-language-action reasoning to improve long-horizon robotic manipulation. Uses a four-part pipeline (curator, seeker, condenser, weaver) so memory participates natively in multimodal action formation.
Evaluates agents inside a structured hospital workflow via a downloadable FastAPI runtime that enforces role-specific tool permissions, evidence-before-treatment discipline, deterministic grading, dense process rewards, and full trajectory logging. Designed for RL, offline policy learning, multi-agent workflow research and process-supervision datasets; not for real patient care.
Provides a deliberative Agent OS layer for robots that handles scene-conditioned planning, context-isolated skill execution, multi-stage verification, persistent multi-modal graph memory, and edge–cloud collaboration. Introduces EmbodiedWorldBench (16 scenes, 200+ tasks) and a failure-driven self-evolution loop; shows improved task success and strong memory benchmark scores.
Unifies high-level visual-language reasoning and low-level control for visual navigation by decoupling cognition and control: a slow vision-language reasoner produces pixel goals with explicit chain-of-thought, and a fast action expert converts those anchors into continuous waypoints for robust urban and indoor navigation.