Tag
Explore by tags
Large-scale mid-training corpora for multimodal models: 10,809 ~60s video shards, caption splits (30s/60s/180s/>10min), 84 spatial-reasoning shards, and CSV mappings to source YouTube IDs. Small Parquet preview configs are provided for schema inspection.
Orchestrates end-to-end video production with agentic pipelines that research, script, generate assets, edit, and render finished videos. Distinguishes itself by supporting true real-footage retrieval (Archive.org, NASA, Wikimedia), Remotion/HyperFrames composition, and usable zero-key workflows alongside cloud providers.
Provides a diagnostic suite that audits video-understanding benchmarks to find samples solvable without visual or temporal input, filters those shortcuts, and produces a distilled video-native testbed that reveals major capability gaps in current Video-LLMs.
Provides curated short video clips (49- and 81-frame) with layered ground truth—edit layers, alpha mattes, and composite targets—for training and evaluating content-preserving layered diffusion video editing. Contains background-replace and object-add edits; Apache-2.0 licensed.
Desktop app for local voice cloning, real-time dictation, and end-to-end video dubbing using zero-shot TTS across 600+ languages; features multi-engine TTS/ASR, speaker diarization, vocal isolation, batch pipelines, and invisible audio watermarking — all run fully offline.
Automates video editing driven by LLM agents: reads word-level transcripts to propose and execute cuts, remove filler words, auto grade color, burn subtitles, and generate animation overlays. Self-evaluates every cut before showing a preview; aimed at talking-heads, tutorials and interviews.
Delivers an ultra-efficient, edge-friendly multimodal image-and-video-to-text model optimized for on-device deployment. Uses mixed 4x/16x visual token compression, a low-FLOPs visual encoder, and multiple quantized variants for mobile and embedded inference.
Reconstructs camera poses and dense 3D point clouds from video streams using a feed‑forward foundation model. Combines a Geometric Context Transformer (anchor + local window + trajectory memory) with paged KV‑cache attention to enable stable, long‑sequence streaming inference (~20 FPS at 518×378).
Turns books, long videos, and podcasts into executable, testable AI agent skills using a structured RIA‑TV++ pipeline. Produces multi-file skill packs (BOOK_OVERVIEW.md, SKILL.md, INDEX.md, DIGEST.md), applies triple verification and pressure tests, and can install skills into Claude Code/Cursor for agent use.
Unified multimodal LLM for enterprise workflows: ingests video, audio, image and text to perform transcription, OCR, Q&A, summarization and long-context reasoning. Provides BF16/FP8/NVFP4 weights and integrations with vLLM, TensorRT-LLM and other runtimes.
An HDR LoRA fine-tune for Lightricks' LTX-2.3 (22B) that enables image‑conditioned any‑to‑any image-to-video and text-to-video generation. Designed for HDR-aware synthesis workflows; requires the LTX-2.3 base model and a LoRA-capable runtime.
Performs task-aware generative video restoration and editing in latent video space — restoration, super-resolution, watermark and subtitle removal — adapting LTX‑2.3 with IC‑Edit/IC‑LoRA adapters to prioritize temporal consistency and occlusion-aware reconstruction.