Provides PyTorch code, pretrained checkpoints, and evaluation tooling for V-JEPA 2 — a Meta FAIR family of self-supervised video encoders and an action-conditioned world model. Includes training recipes, HuggingFace checkpoints, evaluation probes, and robot post‑training artifacts.
Real-time, streaming world model for interactive long-horizon video and scene generation — provides persistent memory across interactions, streaming inference for live scenarios, and example demos for building interactive environments.
Timeline-based video editor for web, desktop and mobile that integrates AI-assisted workflows (MCP server and generative-model integrations) with a Rust core and plugin-first, cross-platform architecture; supports headless batch rendering and no-watermark exports.
An agentic framework that analyzes, plans, and executes multi-step video understanding and editing workflows using multimodal LLM-driven agents—features intent decomposition, graph-based workflow orchestration, and automated shot planning for long-form video tasks.
Detects, segments, and tracks every instance of an open-vocabulary concept in images and video from a text phrase or visual exemplar, not just one object per prompt. An 848M-param model reaching ~75-80% of human accuracy across 270K concepts.
Framework for building multi-modal AI agents that watch, listen, and reason over live video, pairing vision models (YOLO, Roboflow, Moondream) with LLMs like Gemini and OpenAI. Agents join calls in ~500ms and keep audio/video latency under 30ms.
Generates explorable, 3D-consistent virtual worlds from a single image or short video. Includes official implementations of Lyra‑1 (feed‑forward 3D/4D scene generation via video-diffusion self-distillation) and Lyra‑2 (long-horizon, explorable generative 3D worlds). Best for research and creative prototyping; requires substantial GPU compute.
Provides an NVFP4‑optimized training and inference infrastructure for long-form video diffusion models — supports multi-shot AR training, KV-cache and NVFP4 quantized inference, sequence-parallelism and async decoding for higher FPS and longer outputs.
Argues a single web-scale generative video model handles vision tasks zero-shot the way LLMs handle language. Probes Veo 3 on segmentation, edge detection, image editing, physical and affordance reasoning, and puzzles like maze solving and symmetry.
Automatically generates complete short-form videos from a single topic: drafts script with an LLM, produces AI images/video, synthesizes multilingual TTS (including voice cloning), adds background music, and composes the final video. Supports local ComfyUI/RunningHub or direct model APIs and customizable templates.
Runs text-to-video, image-to-video, text-to-image, and image editing inference with acceleration, offloading, quantization, and distributed execution for large visual generation models.