Many workflows that reconstruct dynamic 3D humans require dense, multi-view video coverage; casual monocular captures are far cheaper but produce inconsistent novel-view outputs when naively scaled to the tens of views needed for 4D reconstruction. The paper identifies this failure as a bounded-attention-context problem and shows that fixing how reference and target contexts are represented and routed across diffusion denoising steps substantially improves consistency for downstream 4D Gaussian Splatting (4DGS).
Key Findings
- Reference Context Packing (RCP): compresses a growing set of reference views into a fixed-length, mixed-resolution context so conditioning complexity becomes O(1). This preserves cross-view appearance guidance when many reference frames are used.
- Target Context Routing (TCR): rotates/groupings of target views during denoising so information flows between disjoint target groups at high-noise steps and stabilizes per-view details at low-noise steps, mitigating global structural drift across large numbers of target cameras.
- Dataset and training: introduces MVGameHuman (engine-rendered) and mixes it with light-stage and in-the-wild videos to improve training diversity for large-view generation.
- Empirical impact: generated synchronized multi-view videos are sufficiently consistent to feed into 4DGS pipelines and produce higher-quality 4D reconstructions than prior approaches on benchmarks like DNA-Rendering and DyMVHumans.
Who it's for and trade-offs
Great fit if you need to convert casual monocular or sparse multi-view footage into dense, reconstruction-ready multi-view video for 4D pipelines (e.g., 4D Gaussian Splatting) without relying on per-frame geometry priors. The method is most useful for research and production pipelines that can afford training or fine-tuning and the inference compute of multi-view diffusion models. Look elsewhere if you require real-time capture-to-asset workflows, have extreme occlusions or very loose camera motion where learned context compression may lose critical detail, or if you need guarantees on topology-preserving reconstructions for complex loose clothing.
Where it fits
This work sits between single-view neural rendering and full multi-camera capture workflows: it lowers capture cost by turning casual single-camera takes into synthetic dense captures usable by reconstruction backends, trading capture complexity for offline compute and learned consistency mechanisms.
Brief method note
The core contribution is architectural and algorithmic: instead of increasing per-step attention budget as views scale, compress the reference set (RCP) and schedule communication across target groups (TCR) during diffusion denoising. Combined with a mixed training corpus, these choices reduce cross-view drift and produce stable multi-view videos that downstream 4DGS optimizers can ingest directly.