Evaluates proactive, multimodal agents on 400 bilingual real‑world tasks across five capability axes (Skill Usage, Exploration, Long‑Context Reasoning, Multimodal Understanding, Cross‑Platform Coordination) using live Docker‑based, stepwise closed‑loop evaluation to separate base model skills from framework design.
Provides a deliberative Agent OS layer for robots that handles scene-conditioned planning, context-isolated skill execution, multi-stage verification, persistent multi-modal graph memory, and edge–cloud collaboration. Introduces EmbodiedWorldBench (16 scenes, 200+ tasks) and a failure-driven self-evolution loop; shows improved task success and strong memory benchmark scores.
Unifies high-level visual-language reasoning and low-level control for visual navigation by decoupling cognition and control: a slow vision-language reasoner produces pixel goals with explicit chain-of-thought, and a fast action expert converts those anchors into continuous waypoints for robust urban and indoor navigation.
Acquires repository knowledge via a targeted QA loop before generating patches, decoupling knowledge acquisition from repair. A Questioner and Answerer produce evidence-grounded QA pairs that a Resolver uses to generate fixes; improves Pass@1 on SWE-bench Verified with modest overhead.
Continuously records egocentric visual and audio streams into a lightweight streaming memory that organizes experiences into current, short-term, and long-term tiers and retrieves multimodal evidence to answer queries about past events. Built for on-device use (smartphones/AI glasses) with dynamic retrieval routing.
Converts completed on-policy trajectories into natural-language 'hindsight skills' and converts the skill-induced action probability shifts into a dense token-level on-policy distillation signal, jointly optimized with outcome-based RL to improve sample efficiency and long-horizon agent behavior.
Analyzes adversarial weaknesses of World-Action Models (WAMs) via BadWAM, a framework that crafts visual perturbations to decouple a model’s imagined future from its executed actions. Introduces two attack modes—action-only (disruptive) and imagination-preserving (stealthy)—and shows large drops in closed-loop task success (e.g., 96.5%→43.1%).
Turns fragile, implicit search progress into explicit, persistent, shared state for multi-agent information seeking — externalizes progress as Frontier Task, Evidence Graph, Coverage Map and Failure Memory, and uses pipeline-parallel scheduling plus a middleware harness to avoid repeated failed searches and improve utilization and throughput.
A vision-language-action foundation model trained on 100k+ hours of real-world robot manipulation trajectories to follow natural-language instructions and adapt to downstream tasks with minimal fine-tuning. Uses a two-stage (pre-/post-) training recipe and a scalable auto-labeling pipeline; shows clear scaling benefits and state-of-the-art sim-to-real transfer on standard benchmarks.
Guides an LLM agent to build persistent, editable DAG-based data pipelines via typed, incremental mutations instead of free-form scripts. Combines DataFlow-Skills, a Model Context Protocol exposing live operator registry and pipeline state, and a synchronized Web UI; achieves 93.3% end-to-end pass rate on a 12-task benchmark while cutting cost and latency versus script baselines.
Models long-horizon interactive literary simulation where characters and world co-evolve; introduces an open‑schema framework with a Character Agent and an LLM-based World Model, plus seven trainable tasks and a dataset from 57 books for benchmarking persistent narrative state.
Prunes tool-output lines inside a coding LLM agent by turning the agent's own internal representations into per-line keep-or-prune labels. Implements a small classification head plus a length-aware embedding, saving up to 39% of tokens across benchmarks while preserving task quality.