Streamlines post-training and fine-tuning for large language and multimodal models with a single YAML-driven pipeline. Supports LoRA/QLoRA, full fine-tuning, preference tuning, RL methods, multi-GPU/FSDP/DeepSpeed, and many model backends (Hugging Face, local checkpoints).
Self-hosted gateway putting OpenAI, Claude, Gemini, DeepSeek and 20+ providers behind one OpenAI-compatible endpoint. Adds per-token quotas, channel load balancing and usage billing, so teams or resellers meter keys without sharing upstream credentials.
Self-hostable chat UI that connects to any LLM and adds Agents, Web Search, RAG, connectors, code execution and image generation. Ships connectors to 40+ sources and deployment guides for Docker/K8s. Best for teams needing private, extensible chat platforms.
Wraps a local, OpenAI-compatible inference server in one messages API so you can build private AI apps with no data leaving your network: document ingestion, retrieval with inline citations, and built-in tools (web search, code execution, MCP).
Runs retrieval-augmented Q&A over your own documents on local hardware, so files never leave your machine. Blends semantic, keyword, and late-chunking retrieval, with a router that picks RAG or a direct LLM answer per query and verifies it.
Fine-tunes 100+ LLMs and VLMs from one config file or a no-code web UI, unifying LoRA, QLoRA, full tuning, DPO, PPO, KTO and ORPO behind a single interface. Bundles GaLore, Unsloth, FlashAttention-2 and 2-8bit quantization to fit a single 24GB GPU.
Turns local documents into a private, self-hosted ChatGPT-style assistant with no-code agents for web browsing and workflow automation. Runs across LLM providers — OpenAI, Anthropic, Ollama — and routes tools smartly to cut token use.
Provides a RESTful integration layer that connects WhatsApp and other messaging services to external systems; supports both Baileys (Web) and WhatsApp Cloud API, multiple third-party integrations, media storage, and Docker deployment.
Run any open-source LLM, embedding, speech, image, or multimodal model behind one OpenAI-compatible API — swap GPT for an open model in a single line. Routes across vLLM, llama.cpp, GGML, and TensorRT, scaling from a laptop to a multi-node GPU cluster.
Compresses, deploys, and serves LLMs via two engines: TurboMind for raw speed, a PyTorch engine for flexibility. Claims ~1.8x vLLM throughput through persistent batching, blocked KV cache, and split-and-fuse; ships 4-bit AWQ and KV-cache quantization.
Calls 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure — through one OpenAI-compatible API, as a Python SDK or self-hosted proxy. The proxy adds virtual keys, spend tracking, rate limits, and load balancing across models and providers.