Groups visually grounded appearances of the same physical instance into persistent, retrievable “biographies” so agents can follow objects across hours or days for long-video question answering. Links identity-aware observations to episodic context and visual evidence; improves EgoLifeQA to 72.0% and increases evidence-window reach from 37.6% to 58.9%.
Predicts compact 'prospective tokens' that summarize upcoming information needs and uses them to select a small set of past frames for conditioning long-horizon video generation, improving long-range consistency, visual quality, and action alignment while remaining plug-and-play across diverse generators.
Generates stereo audio whose perceived locations evolve consistently with visible source motion in joint video–audio generation. Introduces StereoBind (motion-conditioned stereo synthesis), a large-scale training set StereoWorld-29K, and an evaluation suite StereoWorldBench for dynamic spatial correspondence.
Generates quantized point-and-box visual primitives for precise grounding across detection, referring, OCR, GUI and layout tasks; uses a shared coordinate vocabulary and staged pretraining to improve downstream robotic manipulation and autonomous driving when used as a visual backbone.
Uses a multimodal model's own critiques as privileged context and applies on-policy self-distillation over diffusion sampling trajectories to internalize corrective guidance, improving text-to-image generation without an external teacher; shows measurable gains on GenEval and GenEval2.
Turns live video into reusable textual memory and timely responses by training a streaming video LLM to proactively generate time-grounded captions and event summaries. Key components: Proactive Hierarchical Caption Memory (PHCM) for multi-scale records and Proactive State Transition Learning (PSTL) to balance response timing; trained on the OneStreamer-1M streaming dataset.
Combines joint distribution distillation from a video teacher with marginal (frame-level) distillation from an image teacher to improve few-step video generation. Introduces LatentBridge to align incompatible latents and Latent Variation Sampling to distribute frame supervision, boosting per-frame visual quality and semantic alignment while largely preserving motion.
Introduces LoHi, a training-free, single-pass method that mixes dense low-resolution video streams with sparse high-resolution frames to improve long-video vision-language model accuracy under strict token budgets while cutting front-end decoding latency.
Scales visual history for real-time robot control by combining autoregressive video pretraining with a streaming, asynchronous predict-then-act pipeline; shows longer context improves long-horizon manipulation and runs full inference in 107.4 ms per action chunk on an RTX 5090.
Generates synchronized egocentric video streams for multiple interacting agents in a shared environment, enforcing cross-view action consistency, shared environment memory, and consistent propagation of interaction-induced state changes — aimed at embodied AI, VR, and multi-agent vision research.
Lets a pretrained multimodal LLM interpret navigation requests and orchestrate motion via tool calls for generalist robot navigation across unfamiliar scenes. Key features: an agent harness with Navigation Skills, a unified visual-point interface, task-progress tracking, and tool-based motion execution without navigation-specific fine-tuning.
Predicts identity-preserving dense pixel correspondences between image pairs that violate spatio-temporal priors (e.g., edits and reference-guided generation). Fuses generative (FLUX2) and semantic (DINOv3) foundation representations with heterogeneous supervision and teacher-guided iterative refinement to generalize beyond classical optical-flow assumptions.