Provides 55 million scene-level video clips (each with captions, language labels, and timestamps) extracted from an 80M-video, 10-million-hour raw pool to support multimodal pre-training across video, audio, and frames. Access is gated for academic/non-commercial research.
Open egocentric multimodal dataset for embodied AI and robot learning captured on commodity iPhone Pro: ~200 hours and ~10M RGB frames with LiDAR depth, ARKit 6‑DoF poses, IMU, two‑hand MANO mocap, room meshes, and hierarchical action captions.
Diagnoses physical plausibility in generated videos by scoring adherence to 13 observable laws. It pairs benchmark prompts and model outputs with quality-controlled human ratings for fine-grained evaluation.
Provides tick-aligned Counter-Strike 2 player POV video clips with per-tick inputs and world-state sidecars — near-lossless 1280×720@32fps video, per-player stereo audio, and parquet indexes for event/kill/round filtering; suited for RL, video classification and clip mining.
Converts video inputs into text outputs — supports captioning, temporal grounding, and video-text-to-text queries using a Qwen-3.5-2B finetuned multimodal backbone. Suited for prototyping video understanding and caption-generation pipelines.
Delivers image and video generation, editing, and understanding inside a single 3B-parameter multimodal model trained from scratch with a multi-task recipe. Notable for strong unified benchmarks at 3B scale; inference requires large GPU memory (≈40GB+ VRAM).
Generates minute-scale, 720p videos from a single image using a 2.6B image-to-video diffusion transformer with precise 6‑DoF camera control and an optional LTX‑2 refiner; designed for long-context, memory-efficient modeling but requires large refiner checkpoints (~41 GB).
Generates temporally coherent MP4 videos from a single input image plus text instructions, with configurable resolution, frame count, and optional AAC audio. Optimized for NVIDIA GPU stacks and integrates with vLLM‑Omni and Hugging Face Diffusers for production inference and research workflows.
Generates audio-driven avatar videos from text, images, or audio inputs with production-grade stability (accurate lip sync, identity consistency) and an 8-step distillation inference mode for faster serving; suitable for broadcasting, virtual hosts, animation, and multi-person scenarios.
Provides a 289-case (1,058-turn) multi-turn benchmark that evaluates interactive video world models across 22 metrics and five dimensions (quality, setting, interaction, consistency, physics). Includes first-/third-person and navigation splits plus a 20-model leaderboard for head-to-head comparisons.
Performs hour-scale video understanding and fine-grained temporal localization while exposing agent-style multimodal tool/code/search abilities. Built on a sparse-attention long-context architecture (DSA) and a specialized inference stack—best used in GPU-backed research or production evaluation.
Performs training-free early-stage visual token compression inside the vision encoder to cut time-to-first-token (TTFT) and FLOPs for Video-LLMs. Introduces a decoupled spatial token selection strategy and reports up to 2.65× TTFT reduction and 61% FLOPs savings on LLaVA-OneVision-7B (NVIDIA A100) while preserving full-token accuracy — aimed at latency-sensitive video understanding.