AIAny
Icon for item

EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

Wraps vision–language–action policies into executable skills that are runtime-validated, executed as bounded low-level action chunks, outcome-verified, and logged as structured trajectories. A fixed skill interface enables swapping or adapting low-level VLA policies and provides component-level supervision for training and optional online adaptation.

Introduction

Most vision–language–action (VLA) models predict actions but don’t by themselves make reliable long-horizon agents. The core insight behind this work is to treat each high-level skill choice as an execution proposal that must pass runtime preflight checks and post-execution verification, transforming black-box action generation into a traceable, trainable closed-loop agent layer.

Key Findings
  • Modular skill interface: skills have typed inputs/outputs and explicit prerequisites, so the runtime can block invalid invocations before physical actions occur — this reduces silent failures and makes trajectories interpretable for supervision.
  • Strong cross-benchmark execution: task-adapted low-level VLA policies in the paper achieve 86.20% average success over 50 RoboTwin 2.0 tasks and 97.40% across four LIBERO suites, demonstrating robust short-horizon execution when integrated into the agent loop.
  • Trajectory-driven training: the system records multimodal context, runtime errors, and verifier outputs so planners, low-level policies, and verifiers can be trained or adapted independently using subtask-level supervision or optional online feedback.
  • Clear limits revealed: the same execution approach yields only 12.5% average success on memory-dependent RMBench tasks, highlighting that skill orchestration improves deployment reliability but does not alone solve long-term memory or complex reasoning deficits.
Who It's For and Trade-offs

Great fit if you need to convert VLA models into inspectable, deployable embodied systems where runtime safety checks, outcome verification, and per-component training matter. It’s especially useful when you want to swap or fine-tune low-level execution policies without changing orchestration logic. Look elsewhere if your primary bottleneck is long-term memory, complex multi-step reasoning beyond short-horizon execution chunks, or when you require end-to-end learned policies with no modular runtime checks.

Where It Fits

EmbodiedSkills sits between high-level planners and robot controllers: it does not replace low-level VLA action models but wraps them with pre-execution validation, bounded execution windows, post-execution verifiers, and structured logging so that embodied agents become safer, debuggable, and easier to train component-wise.

Information

  • Websitearxiv.org
  • OrganizationsAffiliation: College of Computer Science and Technology, Zhejiang University, Affiliation: Nanjing University of Aeronautics and Astronautics, Affiliation: Cornell University, Affiliation: Universal Ubiquitous AI Co., Ltd., Affiliation: Hangzhou DEEP Robotics Technology Co., Ltd., Affiliation: National University of Singapore
  • AuthorsWei Wang, Wenqiao Zhang, Yutong Lin, Yuqian Yuan, Tianwei Lin, Jinhao Mao, Zhenxuan Fan, Mingjian Gao, Yang Dai, Wentong Li
  • Published date2026/09/01

More Items

Studies when and how an assistant should ask clarification questions before converting natural-language problem descriptions into optimization models. Introduces OR-Clarify, a benchmark for pre-formulation clarification, and InterOPT, a two-stage framework that diagnoses formulation-critical gaps and guides targeted questioning and stopping decisions.

Generates synchronized spoken dialogue and explicit full-body co-speech motion (facial expressions, hands, upper- and lower-body) end-to-end from the same hidden states, replacing the speech-then-motion cascade. Trains with a scalable pseudo-labeling pipeline (422,856 ranked pairs) and supports real-time inference (RTF 0.78) while matching teacher motion metrics within ~2%.

Presents two LLM-based search agents (Iris-mini and Iris-pro) trained by alternating supervised fine-tuning and reinforcement learning against live web search. Key features: web-graph-derived multi-hop tasks with entity abstraction, SFT–RL climbing, inference-time context management, and state-of-the-art open-source benchmark results.