Why this matters
Video generation models are increasingly treated as world models, yet many still violate basic physical priors (objects disappearing behind occluders or passing through solids). WROP demonstrates that these core failures—object permanence and solidity—can be systematically probed and substantially improved by training on cognitively grounded synthetic data and evaluating with human-grounded judgments.
Key Findings
- Scalable, cognitively inspired data: 150 hand-authored Blender task generators (6 cognitive families) with nuisance randomization, yielding a 1.5M-sample training corpus (≈10k samples per task).
- Fixed, human-judged exam: a 300-question evaluation set (two questions per generator) judged by pairwise human preferences (Elo) to assess physical-consistency in continuations against hand-authored ground truth.
- Broad model comparison: 14 video systems evaluated (3 reference-to-video, 7 edit, 4 continuation); PWM-WROP, a 16B continuation model fine-tuned on WROP, ranks first among continuation models in a blind human Elo study.
- Reproducible research artifacts: releases include the training corpus, the exam and model answers, model weights (PWM-WROP), the Blender generators, and a native-PyTorch training stack for AWS Trainium2, enabling controlled experiments on physical reasoning.
- Empirical tradeoffs: at native low resolution (320×192) PWM-WROP matches or exceeds continuation baselines on LPIPS and MS-SSIM when resolution-matched, but commercial reference-to-video systems remain competitive at higher native resolutions.
Methodological notes
Each sample splits a clip at the key physical event: the model receives the input half (ending at or just before occlusion/containment) and must predict the subsequent half where the occluded object's fate is resolved. Generators randomize lighting, speed, camera angle and other nuisances while preserving the cognitive structure, enabling generalization tests and per-sample metadata for analysis.
Who it's for and tradeoffs
Great fit if you want a controlled, cognitively grounded benchmark and dataset to train or evaluate video world models on physical reasoning (object permanence/solidity), reproduce human-preference evaluations, or fine-tune continuation architectures. Look elsewhere if your primary need is photorealistic, high-resolution real-world footage (domain gap from Blender may limit direct transfer), if you require purely automated VLM-based judging for subtle core-knowledge tasks, or if you need native high-resolution continuity without additional upscaling.
