Tag

OmniDirector: General Multi-Shot Camera Cloning without Cross-Paired Data

Encodes and clones camera motion from reference videos to generate multi-shot videos — uses a visual "camera grid" to represent camera parameters, trains on million-scale grid–video pairs, and employs a hierarchical prompt-expansion agent to coordinate camera, subject, and action control for multimodal diffusion models.

Rethinking RAG in Long Videos: What to Retrieve and How to Use It?

Proposes chunk-level multimodal retrieval and chunk-adaptive reranking for retrieval-augmented generation on long egocentric videos; introduces V-RAGBench to decouple retrieval vs. generation evaluation and CARVE to run parallel retrievers and select per-chunk configurations.

unsloth/MiniMax-M3-GGUF

Provides experimental GGUF-format quantized weights for MiniMax-M3 to run local multimodal (image‑text‑video) inference via llama.cpp or Unsloth Studio. The model is very large (~428B params) and requires GPU offload or large CPU RAM; llama.cpp currently falls back from sparse to dense attention.

DreamX-World 1.0: A General-Purpose Interactive World Model

Controllable long-horizon text/image-to-video generation that supports camera navigation, revisits, and promptable events across photorealistic and stylized domains. Introduces camera-aware positional encoding (E-PRoPE), memory-conditioned scene persistence, causal-forcing distillation, and RL alignment to retain camera control and reduce drift.

TurboServe: Serving Streaming Video Generation Efficiently and Economically

Serves interactive, long-lived streaming video-generation sessions by jointly scheduling session placement and GPU autoscaling to meet tight per-chunk latency. Combines migration-aware placement, load-driven autoscaling, coalesced chunk processing, GPU–CPU offloading and NCCL GPU–GPU migration; reports ~37% reductions in worst-case per-chunk latency and GPU operating cost.

WGO-Bench

Provides a small, manually annotated benchmark for evaluating vision–language models that convert robot and egocentric manipulation videos into timestamped subtask segments and concise action labels. Contains 100 episodes, 743 gold segments, and MP4 bytes embedded per row.

bigfacing/GOKU-2M

Provides ~2 million instruction-aligned video-edit pairs for training and evaluating instruction-based video editing and generation models. Covers multi-task and structural edits (e.g., camera/subject movement), produced via a synthesis pipeline with progressive filtering; licensed CC BY-NC-4.0.

GOKU-2M

Multimodal video dataset for text-to-video and video-to-video research: about 2 million short English videos and extracted frames for instruction-based video editing and generation. Hosted on Hugging Face and licensed CC BY‑NC 4.0 (non-commercial).

CS2-10k

Provides 600,000+ first-person player-round videos (10,000+ hours) with per-frame keyboard, mouse-delta, and 3D trajectory annotations in WebDataset shards—built for training world models, action-conditioned video, and imitation-learning workflows (non-commercial license).

LTX‑2.3 IC‑LoRA — 3D render to Photoreal

Converts low‑poly 3D viewport or game/CG renders into photorealistic cinematic video while preserving the input's composition, camera motion and layout; offers Light and Strong LoRA variants to trade fidelity for aggressive photorealism.

kyutai/rocket-science

Provides synchronized four-perspective Rocket League match recordings with per-frame H.264 video, player action streams, event logs, and privileged physics state — released as WebDataset shards in a ~4,000-hour slice (1,000 match-hours × 4 perspectives). Includes 720p@20fps video, multi-hot keyboard actions, and CC BY-NC-SA-4.0 license.

Gaming Dataset (gaming-1)

Provides ~494.7 hours of trimmed native PC/console gameplay screen recordings organized by game, with per-session clips plus input and per-frame event annotations. Each workflow includes clip.mp4, events.json, frame_events.json, and metadata — suitable for training vision-action, behavior-cloning, and gameplay understanding models.