Creates an open-ended interactive world simulator with an unbounded interaction horizon via causal pretraining, a distilled real-time runtime that drives 720p@60fps, a wider action/event repertoire, and a pilot–director agent split for behavior planning and environment synthesis.
AI Video Papers2026
Recovers and predicts RGB video from sparse event-camera streams by fine-tuning pre-trained video diffusion priors; jointly addresses reconstruction, long-horizon prediction, and bidirectional frame interpolation with mechanisms to reduce temporal drift and enforce interpolation consistency.
AI Video Papers2026
Uses large-scale text-to-video generative pretraining to create GenCeption, a feed-forward perception model that performs diverse vision tasks from text instructions—depth, surface normals, camera pose, referring segmentation, and 3D keypoints—often matching or surpassing specialized models while requiring far less task-specific data.
Explores unsupervised visual pretraining on visually rich documents to improve language-model intelligence; shows visual-pretrained models outperform text-only counterparts on the same corpora. Key aspects: direct use of images/layouts (no OCR-only pipeline), scalable across backbones and benchmarks.
Reconstructs 4D dynamic human scenes from sparse, low-overlap multi-camera captures by decoupling background synthesis and human modeling. Synthesizes hundreds of camera-controlled background views with a video diffusion model, initializes deformable Gaussian humans via cross-view identity and triangulated keypoints, then applies motion-adaptive recursive enhancement to reduce artifacts.
AI Agent Papers2026
Unifies high-level visual-language reasoning and low-level control for visual navigation by decoupling cognition and control: a slow vision-language reasoner produces pixel goals with explicit chain-of-thought, and a fast action expert converts those anchors into continuous waypoints for robust urban and indoor navigation.
Uses pretrained multimodal LLMs as zero-shot, training-free reward models for text-to-image RL by scoring how well the original text prompt can be recovered from a generated image via image-conditioned prompt log-likelihood; includes a Self-SpectraReward closed-loop variant.
Continuously records egocentric visual and audio streams into a lightweight streaming memory that organizes experiences into current, short-term, and long-term tiers and retrieves multimodal evidence to answer queries about past events. Built for on-device use (smartphones/AI glasses) with dynamic retrieval routing.
AI Agent Papers2026
A vision-language-action foundation model trained on 100k+ hours of real-world robot manipulation trajectories to follow natural-language instructions and adapt to downstream tasks with minimal fine-tuning. Uses a two-stage (pre-/post-) training recipe and a scalable auto-labeling pipeline; shows clear scaling benefits and state-of-the-art sim-to-real transfer on standard benchmarks.
Quantifies active visual observation in multimodal LLMs with ActiveVision, a 17-task benchmark that forces repeated perception rather than one-shot description. Finds frontier MLLMs fail badly (top model 10.6% vs humans 96.1%) and that model-generated vision code does not close the gap.
AI Video Papers2026
Predicts variable-cardinality sets of evidence intervals in videos to temporally ground queries using multimodal large language models. Combines caption-derived multi-span supervision, a temporal Wasserstein matching-free reward, and temporal IoU, yielding strong mIoU gains across multiple benchmarks.
AI Video Papers2026
Personalizes subject-driven videos to preserve human identity and accurate human–object interactions by integrating multimodal references and MLLM-derived semantics. Introduces global multimodal guidance in self-attention and modality-reference embeddings to align MLLM features with VAE tokens, supporting both inter- and intra-subject inputs (e.g., OCR, multi-view).