A research codebase and model family for vision–language models that experiments with data‑centric post‑training strategies and long‑context multimodal reasoning. Includes model reports, released research weights (non‑commercial), grounding tools (LocateAnything) and integrations for inference/optimization.
Provides point-accurate annotations of interactive parts in high-resolution indoor laser-scan point clouds, plus affordance labels, motion axes and natural-language task descriptions; includes aligned iPad RGB-D video slices with 2D projections for multimodal research.
High-resolution image and video generation codebase and models that run with far lower compute and memory than typical diffusion systems. Uses linear-attention DiT variants, aggressive latent compression, and inference-scaling to support text-to-image (up to 4K), fast one/few-step generation, and efficient video pipelines.
Reference architectures and microservices for building GPU-accelerated vision agents that enable natural-language video search, long-video summarization, visual Q&A, and alert verification. Integrates NVIDIA NIM models, embeddings, VLMs/LLMs, and agent workflows for deployable video-analytics stacks.
Provides an open platform of omnimodal world models, datasets, and tools to build Physical AI — joint perception, generation, and action reasoning for robots, autonomous vehicles, and smart infrastructure. Supports images, video, audio, and action-conditioned workflows.
Multi‑modal closed-ended academic benchmark with 2,500 multiple-choice and short-answer exam questions spanning math, natural sciences, and humanities for automated grading. Curated by subject-matter experts, released under MIT, and includes a canary string to help prevent dataset leakage into model training.
Desktop AI client that unifies cloud and local LLMs, tool calling (MCP), installable Skills, and ACP agent integration into a single multi-window workspace. Supports local Ollama models, multi-provider configuration, remote control, and privacy-focused local storage.
An asynchronous, high-throughput framework for large-scale reinforcement learning and agentic training that scales to 1T+ MoE models and 1000+ GPUs, with native verifiers integration, end-to-end SFT/RL/evals, and Slurm/Kubernetes deployment; requires NVIDIA GPUs.
A benchmark dataset for evaluating MLLM-driven interactive webpage code generation: provides prototyping screenshots, action.json interaction metadata, and example generation scripts across 127 webpages and 374 interactions to test dynamic UI-to-code capabilities.
Curated collection of production-oriented AI projects that implement OCR, RAG, multi-agent systems, and multimodal pipelines. Each entry provides runnable code, setup notes, and engineering patterns to help developers move prototypes toward production.
Benchmark for evaluating OCR systems that convert PDFs and scans into Markdown and structured text: 1,403 PDFs and 7,010 unit tests covering text presence/absence, reading order, tables, and math formula accuracy. Diverse sources and ODC-BY-1.0 license for research use.
Provides 1.7M+ synthetic and real infographic charts paired with their tabular data for training and evaluating multimodal models on infographic understanding, chart-to-table extraction, chart code generation, and example-based chart synthesis.