AI Deploy2016
Converts trained PyTorch, TensorFlow, and ONNX models into GPU-tuned inference engines via layer fusion, kernel auto-tuning, and reduced precision. Cuts latency, raises throughput on NVIDIA GPUs from Turing (INT8), with FP8 on Ada+ and FP4 on Blackwell+.
AI Infra2016
Provides a NumPy/SciPy-compatible GPU array library for Python, enabling existing NumPy/SciPy numerical code to run on NVIDIA CUDA and AMD ROCm with minimal changes. Exposes low-level CUDA features (RawKernels, Streams) and offers prebuilt binaries for multiple CUDA/ROCm versions.
AI Deploy2018
Serves machine learning and deep learning models for cloud, data center, edge and embedded environments. Supports multiple frameworks and backends, dynamic and sequence batching, HTTP/gRPC APIs, Docker deployment and NVIDIA-optimized runtimes.
AI Infra2018
Turns NumPy-style Python into differentiable, compiled, vectorized programs for CPU, GPU, and TPU. Its leverage is composable transformations: grad, jit, vmap, and sharding combine instead of living in separate APIs.
AI Train2019
Train and experiment with multi-billion to trillion-parameter transformer models on large GPU clusters using GPU-optimized building blocks and reference training scripts; offers advanced parallelism and mixed-precision support for research teams and ML engineers.
AI Audio2019
Build, fine-tune, and deploy speech AI on NVIDIA GPUs: ASR, text-to-speech, and speech LLMs in one PyTorch stack. Ships pretrained Parakeet/Canary recognition and Magpie TTS checkpoints; broader LLM/multimodal training now lives in v2.7.0.
AI Audio2019
Provides a toolkit and codebase for building, training, and deploying speech and multimodal models — Automatic Speech Recognition, Text-to-Speech, and speech-aware LLMs — with modular neural components and pre-trained checkpoints for PyTorch. Supports streaming/low-latency inference, multi-language models, and optional compiled kernels for acceleration.
AI Train2019
Trains transformer models from 2B to 462B parameters across thousands of GPUs by combining tensor, pipeline, context, and expert parallelism. Ships composable building blocks (Megatron Core) plus reference scripts, with FP8/FP4 and ~47% MFU on H100s.
AI Infra2022
Compiles plain Python functions into GPU or CPU kernels at runtime via a JIT decorator, with differentiable output that plugs into PyTorch, JAX, and Paddle. Ships physics, robotics, geometry, and FEM primitives — particles, meshes, ray-casting, FFT.
AI Infra2022
Fused CUDA kernels that compute exact attention without ever writing the full N×N score matrix to GPU memory, cutting memory from quadratic to linear and speeding up training and inference on A100/H100. Ships FlashAttention-2/3 plus KV-cache decode paths.
Embodied AI2022
GPU-accelerated robot-learning framework on NVIDIA Isaac Sim, running thousands of parallel environments on one GPU for reinforcement and imitation learning. Ships 30+ ready-to-train tasks and 16+ robot models wired to RSL RL, SKRL, and RL Games.
