Tag

MLC LLM

Compiles one LLM into device-native binaries running on CUDA, ROCm, Metal, Vulkan, WebGPU, and CPU — same model from server to browser to phone. On Apache TVM, it ships MLCEngine with an OpenAI-compatible API across Python, JS, REST, iOS, and Android.

Wraps a local, OpenAI-compatible inference server in one messages API so you can build private AI apps with no data leaving your network: document ingestion, retrieval with inline citations, and built-in tools (web search, code execution, MCP).

LitGPT

Provides from‑scratch implementations and end‑to‑end workflows to pretrain, finetune, evaluate, and deploy 20+ LLMs with reproducible YAML recipes, multi‑GPU/TPU support, quantization, and LoRA/QLoRA options.

AirLLM

Runs 70B-class LLM inference on a single 4GB GPU without quantization and supports Llama3.1 405B on 8GB VRAM. Uses layer-splitting and block-wise model compression (4/8-bit) to reduce disk load and can speed up inference loading by up to ~3x; integrates with Hugging Face models.

Xorbits Inference (Xinference)

Run any open-source LLM, embedding, speech, image, or multimodal model behind one OpenAI-compatible API — swap GPT for an open model in a single line. Routes across vLLM, llama.cpp, GGML, and TensorRT, scaling from a laptop to a multi-node GPU cluster.

LMDeploy

Compresses, deploys, and serves LLMs via two engines: TurboMind for raw speed, a PyTorch engine for flexibility. Claims ~1.8x vLLM throughput through persistent batching, blocked KV cache, and split-and-fuse; ships 4-bit AWQ and KV-cache quantization.

Generative Models by Stability AI

Reference implementation for Stability AI's diffusion models: SDXL base/refiner/Turbo for text-to-image, plus Stable Video Diffusion, SV3D, and SV4D for image-to-video and 4D synthesis. A modular engine separates samplers, guiders, and conditioners.

3D Gaussian Splatting for Real-Time Radiance Field Rendering

Enables real-time (≥30 fps) 1080p novel-view synthesis by representing scenes as optimized anisotropic 3D Gaussians plus a visibility-aware splatting renderer; provides the paper's reference implementation, pretrained models and viewers — high-quality training requires CUDA GPU and significant VRAM.

WeatherNext

Provides code and pretrained models for WeatherNext 2 and WeatherNext Cyclones — ML-based global, medium-range atmospheric forecasting models from Google DeepMind/Google Research. Includes Colab demos, pretrained weights, and guides for cloud access and inference.

stable-diffusion.cpp

stable-diffusion.cpp is a pure C/C++ implementation for diffusion model inference, based on ggml, supporting models like Stable Diffusion (SD1.x, SD2.x, SDXL), Flux, Wan, Qwen Image, Z-Image, and more. It's lightweight with no external dependencies, supports backends like CPU, CUDA, Vulkan, Metal, and features like LoRA, ControlNet, LCM for efficient local image generation on platforms including Linux, Mac, Windows, and Android.

LiteLLM

Calls 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure — through one OpenAI-compatible API, as a Python SDK or self-hosted proxy. The proxy adds virtual keys, spend tracking, rate limits, and load balancing across models and providers.

Fine-tunes and deploys 600+ LLMs and 400+ multimodal models in one framework, with SFT, pretraining, RLHF (DPO, PPO, GRPO), and lightweight methods like LoRA and QLoRA. Adds Megatron parallelism, vLLM/SGLang/LMDeploy inference, and a training web UI.