AIAny
Icon for item

Infinite Worlds with Versatile Interactions

Creates an open-ended interactive world simulator with an unbounded interaction horizon via causal pretraining, a distilled real-time runtime that drives 720p@60fps, a wider action/event repertoire, and a pilot–director agent split for behavior planning and environment synthesis.

Introduction

The paper targets a practical gap in interactive world modeling: how to sustain long-running, high-quality interactions while still meeting real-time throughput demands. Its core insight is that combining a causal pretraining regime for horizon-unbounded generation with a distilled real-time runtime and an agentic two-role decomposition (pilot vs director) lets a single system both plan character actions and synthesize evolving environments at interactive framerates.

Key Findings
  • Unbounded interaction horizon: a causal-pretraining approach is used to preserve consistent output quality across arbitrarily long interaction histories, so the simulator can sustain persistent narratives without degeneration.
  • Real-time distilled runtime: a distilled 1.3B model derived from the 14B base enables inference sufficient to drive 720p video at 60 fps, so the system can be used for live streaming and responsive user interactions on modest hardware.
  • Expanded interaction space: the update adds a broader action set (e.g., attacking, archery, spell-casting, shooting) and richer text-driven events, so scenarios can be more diverse and granular than prior LingBot-World versions.
  • Agentic harness (pilot vs director): a pilot agent plans and executes character behaviors while a director agent synthesizes new environmental elements, so control and content generation responsibilities are separated for clearer modularity and emergent coordination.
Who it's for and trade-offs

Great fit if you build interactive simulators, game AI, or research prototypes that need persistent multimodal worlds with live responsiveness; the distilled runtime lowers the bar for deployment while the 14B base supports richer generation. Look elsewhere if you require rigorous benchmarked safety/alignment guarantees, ultra-low compute edge deployment without GPU, or peer-reviewed evaluations across long-term human studies — the architecture trades heavy base-model training and engineering complexity for interactive capability and breadth of actions.

Where it fits

This work sits between large foundation-model research and applied interactive systems: it extends world-modeling efforts by emphasizing runtime distillation and agentic decomposition rather than only raw generation quality, making it a candidate baseline for follow-on research into persistent, multiuser, multimodal simulators.

Information

  • Websitearxiv.org
  • AuthorsZelin Gao, Qiuyu Wang, Jiapeng Zhu, Jingye Chen, Zichen Liu, Qingyan Bai, Jiahao Wang, Yufeng Yuan, Hanlin Wang, Yichong Lu …
  • Published date2026/07/08

More Items

Introduces LoHi, a training-free, single-pass method that mixes dense low-resolution video streams with sparse high-resolution frames to improve long-video vision-language model accuracy under strict token budgets while cutting front-end decoding latency.

Predicts compact 'prospective tokens' that summarize upcoming information needs and uses them to select a small set of past frames for conditioning long-horizon video generation, improving long-range consistency, visual quality, and action alignment while remaining plug-and-play across diverse generators.

Lets general-purpose vision-language models directly command robots via a compact mid-level action interface and asynchronous monitoring, enabling zero-shot manipulation without task-specific policy training; demonstrates strong sim benchmarks and real xArm6 transfer.