AIAny
Icon for item

WorldRuler: Interactive World Modeling under Configurable Rules

Enables interactive world models to follow user-defined rules and preserve them across save-and-restore. An executable 3D rule graph tracks rules, dependencies, and object states, while constraining video generation to show their effects.

Introduction

Interactive world models can generate plausible next frames yet still fail at a more basic promise: a world should continue obeying its rules after an object disappears from view or a session is restored. WorldRuler treats rules and persistent state as part of the world model’s executable structure, rather than relying on the video generator to remember them implicitly.

Key Findings
  • A 3D rule graph stores rule definitions, dependencies, and object states together, so saving and restoring a world also restores the logic governing it.
  • Dynamic object binding links rule-specified entities to visual instances, including objects that have not appeared on screen yet. This helps prevent a rule from being lost simply because its target is temporarily unseen.
  • Attribute-state replacement and counterfactual reconstruction training aim to make rule-driven visual changes accurate while limiting unwanted changes to unrelated objects and motion.
  • The authors build RuleVerse from GTA-V and Habitat interactions, with state and event annotations that include rules taking effect before affected objects become visible.
Who It’s For

A good fit for researchers studying persistent, controllable video worlds, especially where users need to alter game logic rather than merely steer an avatar. It is less suited to applications that only need short, visually plausible clips: the approach introduces explicit state and rule-management machinery, and the reported contribution is a research system and dataset rather than a general-purpose world-generation product.

Information

  • Websitearxiv.org
  • OrganizationsAlaya Lab, Beijing Institute of Technology, Shanghai Innovation Institute, The University of Tokyo, Shenzhen MSU-BIT University
  • AuthorsMingliang Zhai, Chuanhao Li, Jiaming Tan, Zhen Li, Yuwei Wu, Yunde Jia, Zian Meng, Yuanyang Yin, Kaipeng Zhang
  • Published date2026/11/18

More Items

Scales verifiable gameplay data by having a Builder and visual and coding players co-develop web games. Progress-weighted training improves agent performance on both its own benchmark and an external game benchmark.

Generates images in a single forward pass and handles image-to-image tasks and dense prediction with the same model. A shared random state coordinates image codes for global coherence; 512×512 generation takes about 78 ms on an H200.

Generates videos directly in pixel space without a pretrained autoencoder or cascaded refinement stages. It can jointly produce RGB frames and disparity maps, helping generated scenes follow target camera motion.