Category
Explore by categories
Compresses, deploys, and serves LLMs via two engines: TurboMind for raw speed, a PyTorch engine for flexibility. Claims ~1.8x vLLM throughput through persistent batching, blocked KV cache, and split-and-fuse; ships 4-bit AWQ and KV-cache quantization.
stable-diffusion.cpp is a pure C/C++ implementation for diffusion model inference, based on ggml, supporting models like Stable Diffusion (SD1.x, SD2.x, SDXL), Flux, Wan, Qwen Image, Z-Image, and more. It's lightweight with no external dependencies, supports backends like CPU, CUDA, Vulkan, Metal, and features like LoRA, ControlNet, LCM for efficient local image generation on platforms including Linux, Mac, Windows, and Android.
Calls 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure — through one OpenAI-compatible API, as a Python SDK or self-hosted proxy. The proxy adds virtual keys, spend tracking, rate limits, and load balancing across models and providers.
Self-hostable platform for building enterprise GenAI apps with visual workflow orchestration — loops, parallelism, human-in-the-loop — plus RAG, agents, unified model management, and in-house OCR for handwriting and rare characters.
Compiles LLMs into optimized TensorRT inference engines for NVIDIA GPUs via a Python API. Layers in kernel fusion, quantization, paged attention, KV caching, and continuous in-flight batching, scaling from a single GPU to multi-node deployments.
Builds custom AI inference servers in pure Python on top of FastAPI, keeping full control over request logic while batching, GPU autoscaling, streaming, and OpenAI-spec endpoints come built in. Claims a 2x+ throughput edge over plain FastAPI.
Lets AI agents place and answer business phone calls, holding spoken conversations to collect structured data, answer questions, and escalate to humans. Built on Azure Communication Services and Azure OpenAI, with RAG over your own documents.
Creates personalized digital avatars (AI twins) by fine-tuning LLMs on users' chat history and binding them to chatbots. Provides an end-to-end pipeline — chat export, preprocessing with privacy filters, SFT/LoRA training, and deployment (Telegram/Discord/Slack). Best with larger models and substantial chat data.
GPU kernel library for LLM inference attention, sampling, and KV-cache, built on block-sparse formats with JIT-compiled customizable templates. Reports 29-69% inter-token-latency cuts vs compiler backends; powers SGLang, vLLM, and MLC-Engine.
Runs AI-generated code in isolated, elastic sandboxes with SDK, API, and CLI access for agent workflows that need stateful execution and environment control.
Accelerates video generation with a unified framework for inference, finetuning, LoRA, distillation, sparse attention, and distributed execution for research and demos.