Tag

Manifest

Smart model router for personal AI agents that sends each request to the cheapest model capable of handling it — cutting API costs by up to ~70%. Uses a fast 23-dimension scorer, automatic fallbacks, per-tier controls, and supports local Docker self-hosting or a cloud app; ideal for cost-sensitive personal agents.

Text Generation Inference

Rust-and-Python toolkit that serves open-source LLMs (Llama, Falcon, Mixtral, StarCoder) over HTTP/gRPC with tensor parallelism, continuous batching, Flash/Paged Attention and quantization. Now in maintenance mode, pointing users toward vLLM and SGLang.

LangBot

Connect LLMs to major chat platforms so teams can build, deploy, and operate multi-platform AI chatbots and agents. Provides multi-platform adapters, a plugin marketplace, an MCP server and built-in RAG plus production features like access control, rate limiting and monitoring.

AstrBot

Connects one LLM agent to 15+ chat platforms — QQ, WeChat Work, Feishu, Telegram, Discord, Slack — from a single self-hosted backend. Routes to OpenAI, Anthropic, Gemini, DeepSeek or Ollama, and adds a WebUI, MCP tools, and a 1000+ plugin marketplace.

TextGen

A local, offline web UI and API for running and interacting with LLMs (text + vision). Supports multiple backends, tool-calling, fine-tuning and portable builds—designed for private, offline use and flexible model/backend switching.

OpenObserve

Centralizes logs, metrics, traces, frontend RUM and LLM observability into one self-hostable platform, using Parquet + S3-native storage and SQL/PromQL querying to reduce long‑term storage costs and unify telemetry analysis.

Lightpanda Browser

Provides a minimal, Zig-written headless browser tailored for AI agents and automation — runs JavaScript, supports key Web APIs, exposes the Chrome DevTools Protocol for Puppeteer/Playwright, and targets low memory usage and fast startup for large-scale scraping and agent workflows.

Langflow

Visual canvas for composing, testing, and deploying LLM-based pipelines and multi-agent workflows. Supports major LLMs and vector databases, exports flows as APIs or MCP servers, and offers a desktop bundle for local experimentation and iteration.

vLLM

Open-source LLM inference and serving engine built around PagedAttention, which manages the KV cache like OS virtual memory to cut waste and raise throughput. Supports continuous batching, KV cache sharing, quantization, and an OpenAI-compatible API.

SillyTavern - LLM Frontend for Power Users [Official Homepage]

Locally hosted frontend that connects to many text, image, and TTS backends (KoboldAI, Ooba, Tabby, OpenAI, Claude, OpenRouter, Mistral, NovelAI, Horde). Built around character cards, lorebooks, group chats, and extensions for deep prompt control.

E2B

Runs AI-generated code in secure, isolated cloud sandboxes you control via Python or JavaScript SDKs; supports self-hosting (Terraform) and AWS/GCP, enabling agents and code-interpreting workflows to execute real-world tools safely.

LocalAI

Puts OpenAI-, Anthropic- and Ollama-compatible endpoints in front of 60+ inference backends, so existing client code runs unchanged against local models for text, vision, audio, image and embeddings. Runs CPU-only or accelerated, data stays local.