AIAny
Icon for item

Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents

Designs and evaluates a foundation GUI agent that performs cross-platform GUI and CLI actions on real devices to complete long-horizon workflows. Emphasizes a unified action space, a large-scale real-device mobile runtime, an AutoResearch-style data flywheel, and online RL training across 10,000+ concurrent environments.

Introduction

Most GUI research focuses on simulated or narrow tasks; this paper argues real-world utility requires agents that operate reliably on real devices, interleave GUI and CLI actions, and sustain long-horizon stateful workflows. The core insight is that combining diverse sandboxing with a large-scale real-device runtime and an AutoResearch-style loop lets a single foundation agent move from lab benchmarks toward practical, cross-platform automation.

Key Findings
  • Real-device performance: achieves 82.1% on MobileWorld, 92.2% on MobileWorld-Real, and 97.5% on AndroidDaily, showing strong mobile-use competence compared to prior agents. On desktop/browser tasks it records 79.5% on OSWorld-Verified, a 40.0% partial-progress score on OSWorld-v2, 73.6% on WebArena, and 81.5% on ScreenSpot-Pro for GUI grounding.
  • Architecture and training: uses a unified action space that interleaves GUI operations with CLI execution and supports batched multi-step actions per model turn; trains with online RL on trajectories exceeding 100 turns and scales rollouts with over 10,000 concurrent environments.
  • Data and iteration loop: an AutoResearch-style data flywheel has agents generate tasks/environments, diagnose failures, and plan iterations, enabling automated dataset and environment growth without fully manual annotation.
  • Comparative positioning: sets state-of-the-art on mobile benchmarks and is competitive on computer/browser tasks against frontier models referenced in the paper (Opus 4.8, Gemini 3.1 Pro, GPT-5.6 Sol).
Who It's For and Trade-offs

Great fit if you are evaluating foundation agents for real-device mobile automation or building cross-platform GUI+CLI workflows and can provision a device fleet and RL infrastructure. Look elsewhere if you need a lightweight, on-device-only agent or a solution for highly specialized GUIs without access to large-scale runtime and rollout capacity—the approach depends on substantial infrastructure (real-device runtime, parallel environments, and RL compute) and may require adaptation for proprietary or rapidly changing UI ecosystems.

Where It Fits

This work sits between benchmark-focused GUI agents and end-to-end automation platforms: it emphasizes research-forward iteration (data flywheel + RL) while demonstrating production-scale evaluation on real devices. Expect the most value for research teams and engineering groups that can run large-scale experiments and curate real-device fleets.

Information

  • Websitearxiv.org
  • AuthorsHanzhang Zhou, Panrong Tong, Xu Zhang, Quyu Kong, Chenglin Cai, Tianyu Xia, Gongjie Zhang, Jianan Zhang, Long Li, Long Chen …
  • Published date2026/07/30

Categories

More Items

Coordinates world creation, trajectory exploration, and Harness evolution to improve multimodal agents. It organizes tasks by shared capabilities rather than surface categories, reporting mean gains of 34 points on training benchmarks and 26 on unseen benchmarks.

Lets a main agent synthesize executable workflows to coordinate and verify sub-agents as a task unfolds. The paper reports 88.08% overall accuracy in Forced mode across six benchmarks.

Simulates closed-loop interactions between people and wearable AI assistants to test whether multimodal models can guide real-world tasks. Its 120 multi-turn benchmark tasks are scored against verifiable simulator states.