Tag
Explore by tags
Automates multi-step web tasks by perceiving webpages as pixels and issuing low-level mouse, keyboard and scroll actions. A 7B-parameter multimodal agent trained on 145K synthetic trajectories (FaraGen), designed for on-device deployment and efficient task completion (~16 steps/task).
Automatically generates complete short-form videos from a single topic: drafts script with an LLM, produces AI images/video, synthesizes multilingual TTS (including voice cloning), adds background music, and composes the final video. Supports local ComfyUI/RunningHub or direct model APIs and customizable templates.
Performs open-vocabulary promptable segmentation and tracking on images and videos, returning instance masks and unique IDs from short noun-phrase or exemplar prompts. Trained on a 4M-concept, 52M-mask dataset; uses a presence head and a decoupled detector–tracker design, with near real-time inference for multi-object video.
Collects ~200,000 human responses to 20 visual/semantic association questions (e.g., Bouba–Kiki), with per-response image options and demographic metadata — useful for cross‑cultural perception and evaluation of multimodal systems, but not guaranteed as a rigorously controlled experimental sample.
Large-scale, real-world dual-arm video corpus for embodied robotics and reinforcement-learning research — over 1TB of multimodal recordings on Hugging Face, intended for training and evaluating agents in real manipulation scenarios; CC BY‑SA 4.0.
Provides a DiT-based audio–video foundation model plus an official Python inference and LoRA trainer. Ships multiple production-ready pipelines (text/image/audio→video), checkpoints, and performance optimizations (FP8, distilled pipelines) for high-fidelity synchronized audio–video generation.
Orchestrates low-latency, multi-stage pipelines for omni and multimodal models by running each stage with its own scheduler and using zero-copy shared memory for tensor transfer. Emphasizes per-stage bottleneck tuning and OpenAI-compatible streaming endpoints, suitable for TTS and multimodal serving.
Fetches multi-source content (webpages, YouTube, PDFs, WeChat, paywalled articles, podcasts), uploads it to Google NotebookLM, and generates outputs such as podcasts, PPTs, mind maps, or quizzes. Differentiators: automatic paywall-bypass pipeline, Claude Code Skill integration, and CLI + MCP components for WeChat and document scraping.
Provides a physical reconstruction benchmark of OmniDocBench v1.5 by producing five real-world photographic variants (Scanning, Warping, Screen‑Photography, Illumination, Skew) for each of 1,355 pages, inheriting original ground-truth to enable controlled, scenario-wise evaluation of document parsing robustness.
Paired brain MRI scans and radiology text annotations for multimodal vision–language research. Provides image-level labels and image–text pairs suited for VQA, classification, and image-to-text tasks; CC BY-NC-SA 4.0 and ~10K–100K samples — research/non-commercial use.
Runs local AI models on Apple Silicon as an OpenAI‑compatible server, emphasizing low latency, prompt caching, and reliable tool-calling. Optimized for M1–M4 Macs with multimodal support and drop‑in compatibility for IDEs and agent frameworks.
Converts scene intent into production-ready Seedance 2.0 prompts, reference-role mappings, and IP-safe rewrites for multimodal (text/image/audio/video) video generation. Ships as a modular agent-skill OS with multilingual examples, troubleshooting tools, and pro filmmaker handoff artifacts.