Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes

Reframes city-scale 3D visual grounding as active spatial search: a VLM agent interacts with a multi-view BEV environment to query landmarks, zoom, render oblique views, and commit world-aligned bounding boxes. Trains a tool-use policy with reinforcement learning on CityRefer, improving Hit@1, IoU, and inference speed over ranking baselines.

Visit Website

Introduction

City-scale 3D grounding is challenging because urban scenes are vast, densely detailed, and often described with long-range spatial cues and named landmarks; treating the task as static candidate ranking forces dependence on precomputed proposals and scales poorly. Urbex flips this paradigm: it exposes a compact, BEV-aligned multi-view interface and lets a vision-language model act through task-specific tools (landmark query, zoom-in, oblique rendering) to actively search and localize targets.

Key Findings

Active tool-driven search beats candidate ranking: optimizing a tool-use policy with reinforcement learning yields higher Hit@1 and IoU on CityRefer, meaning more accurate localization without relying on precomputed proposals.
Efficient exploration with shaping rewards: smooth terminal rewards plus lightweight tool-level shaping guide the agent to zoom and query judiciously, reducing the number of candidates scored and substantially improving online inference speed.
Cross-scene robustness: Urbex transfers zero-shot to STPLS3D-Refer better than ranking baselines, indicating that multi-view, action-based search generalizes across diverse urban geometries.
Metric-preserving interface: representing views with consistent view→world transforms lets the agent commit boxes in normalized 2D frames while preserving world-coordinate accuracy for evaluation.

Who it's for & tradeoffs

Great fit if you work on large-scale 3D grounding, embodied VLMs, or urban perception and care about reducing dependence on heavy proposal pipelines and improving runtime efficiency. Look elsewhere if you need plug-and-play instance-level detectors (Urbex assumes an interactive BEV interface and an action-budgeted agent) or if per-frame dense detection without interaction is the primary requirement.

Back

Information

Websitearxiv.org
OrganizationsWuhan University, Amap, Alibaba Group
AuthorsYu Wang, Rui Dai, Wei Chen, Yi Wang, Bo Dang, Kaikui Liu, Xiangxiang Chu, Yansheng Li
Published date2026/10/13

More Items

Reinforcement Learning Papers2026

Scoring Higher, Answering Worse: Mitigating Reward Hacking in Rubric-Based RL via Protocol-Level Rubrics

Maoqi Liu, Junwei He +6Beijing University of Posts and Telecommunications, ByteDance

Demonstrates that additive weighted aggregation in rubric-based reinforcement learning enables reward hacking—models improve rubric coverage but produce worse answers—and proposes ProRubric, an offline regrouping that aggregates checklist criteria into conjunctive protocol-level dimensions to raise appropriateness without losing coverage.

RL LLM evaluation research paper+2

Large Language Model Papers2026

TPPO: Tracking-Value PPO For Language Model Reasoning

Futing Wang, Chenxi Li +7Zhejiang University, Westlake University +5

Improves token-level credit assignment for RL fine-tuning of language models by parameterizing the critic as cumulative, position-wise log-odds increments (Tracking-Value PPO), enabling more accurate value-evolution tracking under sparse terminal rewards and boosting math-reasoning and coding benchmarks.

rl LLM reasoning qwen math+3

AI Agent Papers2026

Jeva: System One Models from First Principles

Qi Li, Xinchao WangNational University of Singapore, [email protected]; [email protected]

Presents Jeva, a decision-native architecture that encodes requests and each candidate separately to produce permutation-invariant, option-scalable probability distributions with explicit null detection and reusable option representations for low-latency decision tasks.

paper research benchmark evaluation ai-agent+1