Real-world multi-camera captures often use only a few low-overlap views, leaving large areas poorly observed and causing noticeable artifacts in volumetric reconstructions. This work addresses that gap by separating background densification from human modeling: it leverages video diffusion to synthesize dense background supervision while using a robust deformable Gaussian human initialization and a recursive, motion-adaptive consistency injection to harmonize the composed result. The result is markedly improved novel-view synthesis and cleaner 4D outputs suitable for editing and re-rendering.
4D Human-Scene Reconstruction from Low-Overlap Captures
Reconstructs 4D dynamic human scenes from sparse, low-overlap multi-camera captures by decoupling background synthesis and human modeling. Synthesizes hundreds of camera-controlled background views with a video diffusion model, initializes deformable Gaussian humans via cross-view identity and triangulated keypoints, then applies motion-adaptive recursive enhancement to reduce artifacts.
Introduction
More Items
Converts image-level rewards into explicit intermediate targets for diffusion-model denoising via an on-policy self-distillation loop. Constructs bounded positive/negative targets around anchors from reward gradients, fits those targets with finite updates, and refreshes a behavior policy by EMA—improving aligned performance across backbones while reducing GPU hours.
Generates unified embeddings for text, images, video, visual documents and interleaved multimodal inputs with configurable output dimensions and Matryoshka truncation to trade accuracy for cost. Model weights and code are released under Apache-2.0; the 9B variant scores 80.6 on MMEB-v2.
AI Video Papers2026
Yunheng Li, Guohong Mu +5VCIP, School of Computer Science, Nankai University, Brain and Artificial Intelligence Lab, Northwestern Polytechnical University +2
Treats human annotations as oracle rollouts and separates them from on-policy baselines to improve reinforcement learning for video multimodal LLMs. Key features include a decoupled advantage estimator, sign-balanced pruning, and scalable gains across model sizes and data budgets.