AIAny
Icon for item

Omni-IO Skills: Harnessing Your Agent Omni-Native

Provides a plug-and-play harness that makes existing agents omni-native by exposing hierarchical multimodal Skills, a standardized execution interface, dependency-aware orchestration, and a persistent Asset Registry. Represents multi-asset workflows as Declare Execution Graphs to schedule concurrent operations and enable cross-turn reuse across interchangeable execution backends.

Introduction

Most agent research ties new modalities to expensive model updates or brittle ad-hoc tool assemblies. Omni-IO Skills argues an alternative: compose capability at the harness level so a host agent gains broad, evolvable multimodal production abilities without changing its reasoning core.

Key Findings
  • Harness design: 27 hierarchical Skills cover 38 representative tasks across seven artifact modalities (text, image, audio, video, documents, 3D assets, code) and four capability families (understanding, generation, reasoning, retrieval). This modularity lets different execution backends be swapped without changing the host agent.
  • Execution model: Workflows are encoded as Declare Execution Graphs, enabling dependency-aware scheduling that runs independent operations in parallel and registers successful outputs to a persistent Asset Registry for downstream and cross-turn reuse.
  • Empirical gains: On the UniM-90 benchmark, the harness raised input-support rates of two host agents to 100% and substantially improved semantic-quality and strict-structure scores, demonstrating that harness-level composition can yield production-grade multimodal behavior without retraining the core model.
Who it's for and trade-offs

Great fit if you want to extend an existing LLM-based agent to handle diverse media and multi-step asset workflows quickly, or if you need replaceable execution backends and reusable artifacts across turns. Look elsewhere if your priority is improving the model’s internal multimodal reasoning via end-to-end training, or if you cannot accept the added engineering surface (runtime orchestration, asset registry maintenance, and backend adapters) that a harness introduces.

Information

  • Websitearxiv.org
  • OrganizationsNational University of Singapore, University of Oxford
  • AuthorsYanlin Li, Mingyang Hao, Shengqiong Wu, Hao Fei, Mong-Li Lee, Wynne Hsu
  • Published date2026/09/25

Categories

More Items

Tests decision models as actors, trajectory judges, and reinforcement-learning teachers for long-horizon agents. It finds that a model can be a weak actor yet still improve policy learning by evaluating actions after seeing their consequences.

Lets AI agents read task-relevant evidence from external content instead of receiving entire documents by default. Intent tracking, adaptive readers, and an exposure budget reduce token use and latency while retaining a faithful full-view fallback.

Turns GTA V missions into automatically verifiable agent benchmarks and reinforcement-learning environments, with 150 navigation, interaction, and shooting tasks. Results expose persistent difficulty with dynamic control, while training on generated tasks improves some capabilities.