Before "LLM-as-judge" became standard, ranking open chatbots was guesswork. FastChat's real contribution isn't another serving stack — it's that the same repo bundles training, an OpenAI-compatible serving layer, and the evaluation machinery (Chatbot Arena + MT-Bench) that turned subjective chatbot quality into a measurable Elo number backed by 1.5M+ human votes.
What Sets It Apart
- One toolchain spans the whole lifecycle: fine-tune (Vicuna recipe), serve via a distributed multi-model controller-worker system, then evaluate — no stitching three projects together.
- Drop-in OpenAI-compatible RESTful APIs mean existing clients point at self-hosted open models with a base-URL swap, not a rewrite.
- MT-Bench operationalized GPT-4-as-judge for multi-turn quality, giving a reproducible alternative to crowd voting when you can't run a live arena.
- The serving design scaled to lmarena.ai's 10M+ requests across 70+ models, so the architecture is battle-tested rather than a demo.
Who It's For
Great fit if you're standing up self-hosted open models behind an OpenAI-style endpoint, or need a defensible way to benchmark candidate models against each other. Look elsewhere if you want a polished single-model inference server optimized purely for throughput — projects like vLLM or SGLang go deeper there, and FastChat's breadth means more moving parts than a focused serving engine.
