Why this matters Most end-to-end navigation methods struggle from limited real-world diversity or sim-to-real gaps. ScaleNav flips the data bottleneck: it extracts goal-directed visual dynamics from large collections of in-the-wild egocentric videos, then teaches a navigation policy how to turn those visual priors into executable action chunks via simulation and joint fine-tuning.
Key Findings
- Three-stage training pipeline: (1) instruction-conditioned video pretraining on Internet egocentric clips to learn how observations change when approaching text-described targets; (2) action alignment using simulated, action-labelled trajectories so model outputs map to runnable action chunks; (3) joint fine-tuning for closed-loop navigation. This decouples visual dynamics learning from low-level action semantics.
- Strong empirical results: 86.5% success on a single-view adaptation of Short-Horizon OVON and 94.9% on HabitatGS-LN, outperforming evaluated baselines. Also validated on a real quadruped robot, supporting sim-to-real transfer of the learned priors.
- Data & release: large curated dataset of Internet segments plus simulated trajectories, released with code and preparation scripts to reproduce training/evaluation pipelines.
Who it's for and tradeoffs
Great fit if you research embodied AI or robot navigation and can invest in data preparation and GPU resources: the approach leverages large, diverse real-world videos to reduce reliance on costly robot-collected trajectories. Look elsewhere if you need low-resource solutions (the full pipeline uses substantial storage and GPU memory) or your task is long-horizon, multi-room planning—ScaleNav targets short-horizon, language-specified local navigation where the target is visible in the initial view.