Most text-to-video systems treat prompts as short, descriptive seeds — but as generators extend to multi-shot, cinematic outputs, prompts must plan camera, staging, lighting and temporal continuity across shots. WanPE reframes prompt enhancement as director-level planning rather than simple rewriting, and quantifies how much better video quality and intent fidelity benefit from that shift.
Key Findings
- Large-scale reverse-construction training: WanPE uses a reverse-supervision pipeline to derive hierarchical, shot-level cinematic conditions from high-quality videos and reconstruct compatible user requests, enabling planner-style outputs rather than surface rewrites.
- Scale and data: The paper reports a 397B-parameter prompt enhancement model trained on ~1.05M real-world videos, demonstrating consistent gains as model size grows (4B→397B ablations included).
- Semantic-Consistency GRPO (SC-GRPO): A nine-dimensional reward optimized with group relative policy optimization to penalize omissions, incorrect bindings between subjects/actions/dialogue, and temporal inconsistencies across shots, improving requirement preservation.
- Evaluation with WanPEval: A human-annotated 5–30s testbed and ~11K pairwise assessments show WanPE-397B boosts human preference over raw prompts by ~10.66–18.84 points at 5–15s and by ~50.86 points at 30s; reverse construction outperforms forward rewriting and SC-GRPO yields notable semantic-fidelity gains.
- Transferability: WanPE formats and transfers across downstream video generators after adaptation, and leads or competes with contemporary systems on short-clip benchmarks.
Who It's For and Tradeoffs
- Great fit if you develop or evaluate text-to-video pipelines and need structured, multi-shot prompt conditioning that enforces temporal and semantic fidelity across a short cinematic horizon.
- Look elsewhere if you only require single-frame or very short (1–3s) generation where lightweight prompt templates suffice, or if computational/latency budgets prohibit large prompt-enhancer models at inference time.
Method Snapshot
- Reverse construction: extract hierarchical cinematic conditions from videos, then learn to map user intents to those conditions, encouraging planner-like outputs.
- SC-GRPO reward: combines omission/alteration penalties and binding/temporal consistency checks into group-relative policy optimization to keep enhanced prompts faithful to user intent.
These elements position WanPE as a focused study on elevating prompt engineering from enrichment to explicit cinematic planning for modern T2V systems.