Long-form agentic video generation needs more than passing prompts or recent frames between shots: it needs a shared, time-indexed world representation so future shots can correctly revisit places, replay earlier events, and preserve causal consequences. Any Time, Any Space (ATAS) argues that maintaining an explicit world-state — split into static scene geometry and dynamic, time-stamped event observations — and gating memory updates with review/state checks is the practical way to deliver consistent, revisit-capable multi-shot videos without retraining the generator.
Key Findings
- State-centric evidence: Converting screenplay directives into shot requests with explicit scene, camera, and event-time references lets the agent retrieve only the relevant geometry and event evidence needed for each shot, reducing irrelevant conditioning noise.
- Memory + verification loop: Reconstructed clips are registered into a scene memory and an event memory, but the system updates these memories only after automated review and state checks — preventing local visual errors from propagating across subsequent shots.
- Empirical gains: Across four backbone video generators, ATAS improves all measured cross-shot consistency metrics and achieves the highest aggregate benchmark scores per backbone. It leads on target-image spatial revisit and multiple event-recapture criteria, and obtains the highest human ratings on six evaluated criteria.
- Practical delivery: Film delivery is decoupled from memory admission, so final outputs can be produced faster while the memory admission and checks run as a separate verification step.
Who It's For and Trade-offs
Great fit if you are building long-horizon, multi-shot video agents that must support revisits, replay of prior events, and causal story progression — particularly when you can register scene geometry or capture viewpoint metadata. Look elsewhere if your pipeline cannot provide any scene geometry or camera metadata, if you need a purely end-to-end single-shot generator without external memory, or if you require minimal system complexity/latency (the memory/review loop adds implementation and runtime overhead).
Where It Fits
ATAS sits between retrieval-conditioned agentic pipelines and fully 3D-guided methods: unlike naive prompt- or keyframe-retrieval, it maintains an explicit, time-indexed world state and enforces admission checks; unlike heavy 3D reconstruction approaches, it focuses on registered geometry plus event memories and works as an agentic layer on top of existing video generators. The authors release video examples and code to reproduce evaluations and integrations.