AIAny
Icon for item

Game2Player: Scaling Gameplay Experience for Visual Agents through Environment–Player Co-Development

Scales verifiable gameplay data by having a Builder and visual and coding players co-develop web games. Progress-weighted training improves agent performance on both its own benchmark and an external game benchmark.

Introduction

Gameplay data is unusually hard to scale: videos omit the actions that produced them, while successful play is difficult to verify. The paper’s key insight is to make environment construction part of data collection: players expose flaws in game interfaces and task definitions as they play, and those environments are repaired before their trajectories are used for training.

Key Findings
  • The framework produces 568 accepted web-game environments and 3,436 tasks, with 15.7K trajectories totaling 307 hours. This provides broader coverage than a small set of hand-authored game tasks.
  • A Builder exposes game state and defines measurable progress and success checks; a screenshot-playing VLM and a state-reading coding player test the same tasks and report problems. Their different observation interfaces catch different failures.
  • Progress-weighted supervised fine-tuning gives greater weight to actions that move a task forward. The 9B model raises average progress on the held-out Player Bench from 14.3% to 31.2%, and on GameWorld from 30.0% to 53.1%.
  • The results suggest that a useful training signal can come from verified progress, not just successful demonstrations. The external benchmark improvement is notable because it shares no games, tasks, or success checks with the collection.
Who It’s For—and the Trade-offs

A good fit for researchers studying visual agents, interactive control, and scalable training data, especially those who need task-level progress and verifiable outcomes. It is less suited to claims about real-time play: the VLM pauses the game while reasoning, and the collection focuses on offline, single-player browser games. Acceptance is not a guarantee of correctness; audits found that some generated tasks were implemented incorrectly.

Information

  • Websitearxiv.org
  • OrganizationsUIUC, UC Santa Barbara, Georgia Tech, Northwestern University, IBM
  • AuthorsXirui Jiang, Yuxi Chen, Rui Yang, Feng Luo, Pengcheng Wang, Bingxuan Li, Zihao Zhao, Yushan Li, Han Wang, Manling Li …
  • Published date2026/11/18

More Items

Enables interactive world models to follow user-defined rules and preserve them across save-and-restore. An executable 3D rule graph tracks rules, dependencies, and object states, while constraining video generation to show their effects.

Finds that agent evolution often stalls not for lack of better prompts or skills, but because noisy evaluations fail to identify real improvements. The study tests seven methods across six models.

Generates images in a single forward pass and handles image-to-image tasks and dense prediction with the same model. A shared random state coordinates image codes for global coherence; 512×512 generation takes about 78 ms on an H200.