Analyzes adversarial weaknesses of World-Action Models (WAMs) via BadWAM, a framework that crafts visual perturbations to decouple a model’s imagined future from its executed actions. Introduces two attack modes—action-only (disruptive) and imagination-preserving (stealthy)—and shows large drops in closed-loop task success (e.g., 96.5%→43.1%).
A vision-language-action foundation model trained on 100k+ hours of real-world robot manipulation trajectories to follow natural-language instructions and adapt to downstream tasks with minimal fine-tuning. Uses a two-stage (pre-/post-) training recipe and a scalable auto-labeling pipeline; shows clear scaling benefits and state-of-the-art sim-to-real transfer on standard benchmarks.
Generates robot manipulation actions from visual observations and text instructions using a 1.5B vision-language-action model. Uses streaming context and visual-token compression to cut per-step compute, runs a unified policy across tasks, and is open-sourced on Hugging Face under Apache-2.0.
Predicts eight future [x,y,yaw] waypoints for language-conditioned embodied person-following using fused DINOv3 and SigLIP visual features; trained with quality-driven, DAgger-style self-evolving data and optimized for on-device inference (~5+ FPS, ~180 ms).
Generates and reasons about multimodal physical-world content—text, images, video and action trajectories—conditioned on text, images, video and robot/vehicle action inputs. An edge-sized (4B) Mixture‑of‑Transformers omni-model optimized for single‑GPU inference and Physical AI tasks (image→video, action prediction, robot policy).
Selects a referred target from candidate bounding boxes, then decodes tracking waypoints for single-camera embodied visual tracking. Injects past selected-bbox geometry via sliding-window TVBI tokens and is co-trained on a Refer‑QA dataset; achieves SOTA on EVT‑Bench and demonstrates sim-to-real on legged and humanoid robots.
Provides a unified survey of progress-reward modeling for robotic learning, detailing interfaces, modeling techniques, and evaluation practices. Organizes the literature into three perspectives—interface, model internals, and data/benchmarks—and highlights limitations and open problems. Useful for researchers designing rewards for long-horizon or sparse-reward robotic tasks.
Enables tactile-aware robot manipulation by pretraining a vision–tactile–language–action foundation model and improving offline policies with ALTER. Combines large-scale NeoData visuo-tactile pretraining, a latent tactile pathway for predictive touch signals, and advantage‑conditioned offline RL for contact-rich tasks.
Provides a portable, robot-free UMI capture pipeline and shows that policies post-trained only on this high-fidelity data deploy directly on real robots matching teleoperation baselines. Capture achieves ~3 mm end-effector accuracy, microsecond sync, ultra-wide FOV, and releases 2,000h HiFi-UMI-2K.
Captures synchronized multimodal embodied-human data in real homes — egocentric and multi-view video, metric body/hand/object motion, audio, and tactile signals. Released under a gated non-commercial research license with identifiable participants and strict non-redistribution/privacy constraints.
Directly maps visual observations and language instructions to continuous robot actions, replacing LLM-centric V→L→A pipelines. Uses separate visual and language encoders with lightweight bidirectional interaction and a compact decoder to cut inference cost and VRAM, achieving ~31 ms latency and <1 GB VRAM on an RTX 4090; suited for real-time robotic manipulation under tight compute budgets.
Evaluates whether vision-language models can make actionable decisions for a physical body by decoupling decision-making from low-level motor execution. Introduces HumanCLAW-Bench with 1,218 long-horizon egocentric episodes across 41 indoor scenes and diagnoses a lack of embodied self-awareness in current VLMs.