Generates image-to-video world-model outputs using a distilled 14B causal model optimized for chunked, KV-cached inference across long-horizon interactive scenes; offers a real-time 'causal-fast' variant capable of driving near‑real‑time video streams and an agentic harness for action-driven scene synthesis (CC BY‑NC‑SA).
Pretrains a DiT-based Mixture-of-Experts video foundation model for embodied intelligence by augmenting internet videos with robot-centric footage and using a multi-dimensional reward system to prioritize physical realism and task completion while scaling MoE for better capacity vs. inference trade-offs.
Generates videos from text and image+text prompts using a 30B Mixture-of-Experts model tuned for embodied intelligence; includes a refiner and structured prompt rewriter, and supports diffusers/SGLang runtimes with multi-GPU inference.
Creates an open-ended interactive world simulator with an unbounded interaction horizon via causal pretraining, a distilled real-time runtime that drives 720p@60fps, a wider action/event repertoire, and a pilot–director agent split for behavior planning and environment synthesis.
Recovers and predicts RGB video from sparse event-camera streams by fine-tuning pre-trained video diffusion priors; jointly addresses reconstruction, long-horizon prediction, and bidirectional frame interpolation with mechanisms to reduce temporal drift and enforce interpolation consistency.
Uses large-scale text-to-video generative pretraining to create GenCeption, a feed-forward perception model that performs diverse vision tasks from text instructions—depth, surface normals, camera pose, referring segmentation, and 3D keypoints—often matching or surpassing specialized models while requiring far less task-specific data.
Reconstructs 4D dynamic human scenes from sparse, low-overlap multi-camera captures by decoupling background synthesis and human modeling. Synthesizes hundreds of camera-controlled background views with a video diffusion model, initializes deformable Gaussian humans via cross-view identity and triangulated keypoints, then applies motion-adaptive recursive enhancement to reduce artifacts.
Generates minute-scale, temporally coherent dance videos from full music tracks using a hierarchical two-stage approach: global keyframe planning plus local temporal refinement; suitable when long-range musical structure and rhythmic continuity matter.
Generates a new camera viewpoint from a reference video: an IC‑LoRA adapter for LTX‑Video 2.3 that re‑renders the same scene from a requested discrete camera angle while preserving subject and content. Trained on synthetic multi‑view data, proof‑of‑concept with limited viewpoint range and best for small, chained angle shifts.
Provides 1,800+ hours of synchronized egocentric multi-view recordings with 3D hand reconstructions, wide‑FOV depth, and hierarchical task/subtask annotations for embodied AI and robot learning. Includes six fisheye views, hand meshes, and per-episode temporal labels across 44k+ episodes.
Timestamp-aware realtime video→text model that processes incoming frames continuously, answers questions mid-stream or emits silence when evidence is insufficient, and can revise earlier outputs as new frames arrive. Built for timestamped multimodal interaction with a 256K context and an 11B-parameter backbone.
Comprehensive benchmark and automated evaluation framework for keyframe-conditioned video generation—decomposes keyframe execution into six metrics and assesses overall video quality with evidence-grounded MLLM judgments and specialized perception models.