AI Deploy2023
Compiles one LLM into device-native binaries running on CUDA, ROCm, Metal, Vulkan, WebGPU, and CPU — same model from server to browser to phone. On Apache TVM, it ships MLCEngine with an OpenAI-compatible API across Python, JS, REST, iOS, and Android.
Chatbot2023
Wraps a local, OpenAI-compatible inference server in one messages API so you can build private AI apps with no data leaving your network: document ingestion, retrieval with inline citations, and built-in tools (web search, code execution, MCP).
AI Infra2023
Runs 70B-class LLM inference on a single 4GB GPU without quantization and supports Llama3.1 405B on 8GB VRAM. Uses layer-splitting and block-wise model compression (4/8-bit) to reduce disk load and can speed up inference loading by up to ~3x; integrates with Hugging Face models.
AI Deploy2023
Run any open-source LLM, embedding, speech, image, or multimodal model behind one OpenAI-compatible API — swap GPT for an open model in a single line. Routes across vLLM, llama.cpp, GGML, and TensorRT, scaling from a laptop to a multi-node GPU cluster.
AI Deploy2023
Compresses, deploys, and serves LLMs via two engines: TurboMind for raw speed, a PyTorch engine for flexibility. Claims ~1.8x vLLM throughput through persistent batching, blocked KV cache, and split-and-fuse; ships 4-bit AWQ and KV-cache quantization.
AI Image2023
Reference implementation for Stability AI's diffusion models: SDXL base/refiner/Turbo for text-to-image, plus Stable Video Diffusion, SV3D, and SV4D for image-to-video and 4D synthesis. A modular engine separates samplers, guiders, and conditioners.
Enables real-time (≥30 fps) 1080p novel-view synthesis by representing scenes as optimized anisotropic 3D Gaussians plus a visibility-aware splatting renderer; provides the paper's reference implementation, pretrained models and viewers — high-quality training requires CUDA GPU and significant VRAM.
AI Model2023
Provides code and pretrained models for WeatherNext 2 and WeatherNext Cyclones — ML-based global, medium-range atmospheric forecasting models from Google DeepMind/Google Research. Includes Colab demos, pretrained weights, and guides for cloud access and inference.
AI Image2023
stable-diffusion.cpp is a pure C/C++ implementation for diffusion model inference, based on ggml, supporting models like Stable Diffusion (SD1.x, SD2.x, SDXL), Flux, Wan, Qwen Image, Z-Image, and more. It's lightweight with no external dependencies, supports backends like CPU, CUDA, Vulkan, Metal, and features like LoRA, ControlNet, LCM for efficient local image generation on platforms including Linux, Mac, Windows, and Android.
AI API2023
Calls 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure — through one OpenAI-compatible API, as a Python SDK or self-hosted proxy. The proxy adds virtual keys, spend tracking, rate limits, and load balancing across models and providers.
AI Train2023
Fine-tunes and deploys 600+ LLMs and 400+ multimodal models in one framework, with SFT, pretraining, RLHF (DPO, PPO, GRPO), and lightweight methods like LoRA and QLoRA. Adds Megatron parallelism, vLLM/SGLang/LMDeploy inference, and a training web UI.