LiteLLM

LiteLLM is an open-source LLM gateway and Python SDK that lets developers call more than 100 commercial and open-source models through a single OpenAI-compatible interface, complete with cost tracking, rate-limiting, load-balancing and guardrails.

Visit Website

Introduction

Overview

LiteLLM provides a drop-in replacement for the OpenAI client plus an HTTP proxy server, abstracting away provider-specific differences so platform teams can ship multi-LLM products faster .

Key Capabilities

OpenAI-compatible SDK & proxy – call 100+ LLMs (OpenAI, Azure, Anthropic, Bedrock, Google Gemini, Hugging Face etc.) via the /v1 schema
Cost & usage analytics – real-time spend tracking tagged by key / user / team with S3 & GCS logging support
Rate limiting & budgets – set RPM/TPM limits and per-project budgets to control usage
Automatic fallbacks & retries – route requests across multiple providers to improve reliability
Extensible guardrails – plug in custom validation, redaction and audit hooks to meet compliance needs

Back

Information

Websitewww.litellm.ai
AuthorsBerriAI
Published date2023/07/26

More Items

One API

2023

Song Quanpeng

One API is a self-hosted key-management and distribution gateway that unifies OpenAI-style access to dozens of LLM providers, enabling centralized quota, billing and user management through a single binary or Docker image.

ai-development ai-api ai-api-management

xAI API

2024

xAI

xAI API gives developers OpenAI-compatible endpoints for the Grok family of language, vision and image-generation models.

ai-development ai-api ai-agent ai-image xai+1

OpenRouter

2023

Alex Atallah, Louis Vichy

OpenRouter is a unified API gateway and marketplace that lets developers access, compare, and route requests across 400+ large-language models from 60+ providers through a single, fully OpenAI-compatible endpoint.

ai-development ai-api