AIAny
Icon for item

Video models are zero-shot learners and reasoners

Argues a single web-scale generative video model handles vision tasks zero-shot the way LLMs handle language. Probes Veo 3 on segmentation, edge detection, image editing, physical and affordance reasoning, and puzzles like maze solving and symmetry.

Introduction

GPT-3 made a quiet point that took the field years to absorb: scale a single next-token predictor on enough text and task-specific models start to feel like a detour. This paper asks whether video generation is now at the same inflection point — whether predicting the next frame, at web scale, quietly bundles together the perception and reasoning skills that vision research has spent a decade building one specialist model at a time.

Key Findings
  • A single frozen model (Veo 3) is shown solving tasks it was never trained for, just by conditioning on a prompt and an input image — no fine-tuning, no task heads. The work treats prompting a video model as the visual analogue of prompting an LLM.
  • The capability sweep is unusually broad: low-level perception (segmentation, edge detection), pixel-level editing, physical-property and affordance reasoning, and abstract visual puzzles like maze navigation and symmetry recognition all fall out of the same generative objective.
  • The framing matters more than any single benchmark: it reframes a video model not as a clip generator but as a candidate unified vision foundation model, the way GPT-style models became unified language models.
Why This Reframes the Debate

Most of computer vision is still organized around bespoke architectures per task. The provocation here is that emergent, zero-shot generality — the thing that made LLMs eat NLP's task zoo — may already be appearing in video. If the trajectory holds, the interesting question shifts from "which model for segmentation" to "which prompt."

Who Should Read It

Great fit if you work on vision foundation models, generative video, or are tracking where the next paradigm shift in perception comes from — the chains-of-frames-as-reasoning angle is the takeaway. Look elsewhere if you need production-ready accuracy today: this is an existence-and-direction argument about emergent ability, not a claim that a prompted video model beats tuned specialists on any given metric.

Information

  • Websitearxiv.org
  • OrganizationsGoogle DeepMind
  • AuthorsThaddäus Wiedemer, Yuxuan Li, Paul Vicol, Shixiang Shane Gu, Nick Matarese, Kevin Swersky, Been Kim, Priyank Jaini, Robert Geirhos
  • Published date2025/09/24

More Items

Provides a real-scale 3D Hong Kong sandbox to evaluate whether multimodal LLM agents can turn local street-view perception into sustained spatial action, supporting closed-loop first-person interaction, an interactive map, and controlled tests of grounding, long-range navigation, and robustness.

Measures whether video generators reproduce the correct distribution of possible physical behaviors under repeated rollouts. Introduces PAWBench and PAWEval to convert repeated generations into outcome-level empirical distributions and quantify probabilistic alignment; evaluates 50 scenarios and 11 models and finds no model consistently matches reference probabilities.

Evaluates visual reasoning in video generation models using 27 photorealistic tasks (810 instances), a two-level taxonomy of domains and skill tags, and task designs that enforce valid intermediate trajectories and calibrated difficulty.