AI Video2024
Reference architectures and microservices for building GPU-accelerated vision agents that enable natural-language video search, long-video summarization, visual Q&A, and alert verification. Integrates NVIDIA NIM models, embeddings, VLMs/LLMs, and agent workflows for deployable video-analytics stacks.
AI Audio2024
Captures, transcribes, and summarizes meetings entirely on the user's machine with real-time local transcription and speaker diarization. Privacy-first design keeps audio, transcripts, and models local; supports Ollama, Claude, Groq, OpenRouter or custom OpenAI-compatible endpoints.
Embodied AI2024
Provides an open platform of omnimodal world models, datasets, and tools to build Physical AI — joint perception, generation, and action reasoning for robots, autonomous vehicles, and smart infrastructure. Supports images, video, audio, and action-conditioned workflows.
AI Dataset2025
A curated dataset of ~30,000 CUDA kernels generated by an agentic pipeline, including reference PyTorch implementations, runtime metrics, NCU/Torch/Clang-Tidy profiles, error messages and correctness labels — released under CC-BY-4.0 for model fine-tuning and offline RL/optimization research.
AI Infra2025
High-performance CUDA tensor-core GEMM kernel library for LLM workloads: supports FP8/FP4/BF16, fused Mega MoE and MQA scoring, and runtime JIT-compiled kernels. Targets NVIDIA SM90/SM100 and PyTorch—for teams working on low-level GPU kernel optimization.
AI Infra2025
Provides high-throughput, low-latency GPU communication kernels for Mixture-of-Experts (MoE) and expert-parallel workloads, with NVLink↔RDMA-aware forwarding, FP8/BF16 support, and low-latency RDMA hooks for inference decoding.
AI Infra2025
Optimized MLA (Multi-head Latent Attention) decoding kernels powering DeepSeek-V3/V3.2 inference on Hopper and Blackwell GPUs. Dense decoding reaches ~3000 GB/s and 660 TFLOPS on H800; the sparse path stores the KV cache in FP8.
AI Agent2025
Framework-agnostic library for connecting and optimizing teams of AI agents built in LangChain, LlamaIndex, CrewAI, Semantic Kernel, or Google ADK. Profiles them down to individual tokens, traces execution, and runs built-in evaluation.
Embodied AI2025
A vision-language-action foundation model and reference stack for generalized humanoid and cross-embodiment robot manipulation. Provides pretrained checkpoints, demo datasets, and tooling for fine-tuning, evaluation, and deployment (ONNX/TensorRT); released as Early Access.
AI Deploy2025
Splits LLM inference into separate prefill and decode GPU pools, then routes requests with KV-cache awareness to cut redundant recomputation. Reports up to 30x throughput on DeepSeek-R1 (GB200 NVL72) and works across TensorRT-LLM, vLLM, and SGLang.
AI Dataset2025
Provides 5 million instruction–response pairs for supervised fine-tuning of code LLMs, with inputs, outputs, unit tests, and automated LLM judgments. Uses hybrid automated/synthetic generation and is released under CC BY 4.0 for large-scale SFT workflows.
Embodied AI2025
GPU-accelerated physics simulation engine for robotics and simulation research — built on NVIDIA Warp with MuJoCo Warp backend, offering differentiable simulation, OpenUSD support, and extensions for RL/embodied-AI workflows. ([github.com](https://github.com/newton-physics/newton))