Grounds language-based robot planning in learned skill value functions, selecting actions that are both useful and feasible. PaLM-SayCan reports 84% planning and 74% execution success across 101 real-world instructions.
Turns language commands into executable robot policies that combine perception, control APIs, loops, and geometry libraries. Hierarchical code generation recursively fills in missing functions to support new behaviors without policy retraining.
Autonomously generates, iteratively debugs, and composes robot control programs into a reusable skill library for discovering and refining manipulation skills across simulation and real robots. Uses a code-as-policy continual learning loop—accumulating validated sensorimotor skills instead of model weights—to tackle long-horizon household and benchmark tasks.
Provides 1,800+ hours of synchronized egocentric multi-view recordings with 3D hand reconstructions, wide‑FOV depth, and hierarchical task/subtask annotations for embodied AI and robot learning. Includes six fisheye views, hand meshes, and per-episode temporal labels across 44k+ episodes.
Generates articulated motion for arbitrary rigged 3D skeletons from text prompts with a single unified model and no per-skeleton retraining. Uses topology-aware diffusion-transformer architecture and a large UniML3D motion dataset to generalize across bipeds, quadrupeds, and other topologies; suitable for text-to-animation workflows and rapid prototyping.
Integrates a pretrained vision–language model with a BEV perception head and a Planning Expert to provide 3D perception, driving VQA and motion planning for autonomous driving while keeping the base VLM architecture unchanged.
Wraps vision–language–action policies into executable skills that are runtime-validated, executed as bounded low-level action chunks, outcome-verified, and logged as structured trajectories. A fixed skill interface enables swapping or adapting low-level VLA policies and provides component-level supervision for training and optional online adaptation.
Trains a world-action model that predicts future visual states to guide zero-shot robotic manipulation; introduces CoAE, SVP, IDM and KASO to pretrain generative and action components from scratch on manipulation data, scaling up to 30,000 hours and improving zero-shot success.
Generates compositional 3D scenes as collections of individual object meshes by conditioning a single-object 3D generative prior on multi-view posed observations. Key features include anchor-aligned canonical frames, multi-view DINOv3 feature lifting with an IBR-style fusion, and LoRA adaptation to complete heavily occluded objects; includes a large UE-MeshyScene benchmark.
Multimodal foundation model for visual understanding, spatial reasoning and multi-step agent tool use — accepts text, multiple images and video at any resolution and supports long contexts (up to 128K tokens). Emphasizes fine-grained 2D/3D relations, affordance reasoning and embodied-AI planning.
Trains end-to-end driving without human trajectory supervision by decoupling perception and action: DriveVFM distills multiple frozen vision foundation models into a single camera backbone, and DriveRL trains a privileged closed-loop RL teacher whose rollouts supervise a camera-only planner, yielding state-of-the-art closed-loop benchmark results.
Trains LVLMs to improve 3D spatial reasoning using a synthetic block-stacking dataset of 15,000 problems that cover 3D→2D projection, viewpoint transforms, and structural combination; includes color-anchored cues and provides code and data.