AI Image2019
Bundles hundreds of pretrained image backbones — ResNet, EfficientNet, ViT, ConvNeXt, Swin and more — behind one consistent API for classification and feature extraction, with training and inference scripts that reproduce published ImageNet results.
AI Train2019
Train and experiment with multi-billion to trillion-parameter transformer models on large GPU clusters using GPU-optimized building blocks and reference training scripts; offers advanced parallelism and mixed-precision support for research teams and ML engineers.
AI Train2019
Turns raw PyTorch training loops into structured modules that scale from a laptop to multi-node GPUs without rewriting model logic. It handles precision, checkpointing, logging, and distributed execution while preserving PyTorch control.
AI Train2019
Implements deep RL algorithms (PPO, DQN, SAC, TD3, DDPG, C51, PPG) as standalone single-file scripts — the PPO Atari variant is ~340 readable lines. Built for research debugging and reproducibility, with W&B and TensorBoard tracking.
AI Audio2019
Build, fine-tune, and deploy speech AI on NVIDIA GPUs: ASR, text-to-speech, and speech LLMs in one PyTorch stack. Ships pretrained Parakeet/Canary recognition and Magpie TTS checkpoints; broader LLM/multimodal training now lives in v2.7.0.
AI Image2019
Modular implementations of object detection, instance/semantic/panoptic segmentation and related vision models for research and deployment. Offers a large model zoo, export to TorchScript/Caffe2, and PyTorch-native optimizations for faster training and extensibility.
AI Train2019
Trains transformer models from 2B to 462B parameters across thousands of GPUs by combining tensor, pipeline, context, and expert parallelism. Ships composable building blocks (Megatron Core) plus reference scripts, with FP8/FP4 and ~47% MFU on H100s.
AI Train2020
thu-ml (Tsinghua University Machine Learning Group)Tsinghua University, appliedAI Institute for Europe
Deep reinforcement learning library on pure PyTorch and Gymnasium, with 30+ algorithms across on-policy, off-policy, and offline RL. Exposes both a one-call high-level interface and a procedural API, plus vectorized envs and reproducible MuJoCo benchmarks.
AI Infra2020
Optimizes distributed PyTorch training and inference for very large models with ZeRO memory partitioning, parallelism, MoE, offload, and compression. Best when GPU memory, training cost, or cluster throughput is the bottleneck.
AI Train2020
PyTorch library for operator learning: neural networks that map between whole function spaces, not fixed grids, so a model trained at one resolution runs at any other. Bundles FNO, Tensorized FNO and related architectures, mainly for solving PDEs.
AI Infra2021
Runs, manages, and scales AI workloads across 20+ clouds, Kubernetes, Slurm, and on-prem from one YAML or Python spec. Auto-provisions GPUs/TPUs, fails over across regions and providers when capacity is short, and routes jobs to the cheapest option.
AI Train2021
HPC-AI Technology Inc. (Colossal-AI team), Shenggui Li +1HPC-AI Technology Inc., National University of Singapore
Scales a single-GPU training script to thousands of GPUs through a unified interface, combining data, pipeline, tensor, and sequence parallelism. Its Gemini memory manager offloads tensors across GPU, CPU, and NVMe so models far larger than VRAM still fit.