FastChat

An open platform for training, serving and evaluating chat-oriented LLMs—powering Vicuna & Chatbot Arena.

Visit Website

Introduction

Back

Information

Websitegithub.com
AuthorsLarge Model Systems (LMSYS)
Published date2023/04/03

More Items

Ray

2017

RISELab (UC Berkeley), Anyscale Inc.

Ray is an open-source distributed compute engine that lets you scale Python and AI workloads—from data processing to model training and serving—without deep distributed-systems expertise.

ai-development ai-framework ai-train ai-serving

LLaMA-Factory

2024

hiyouga

Zero-code CLI & WebUI to fine-tune 100+ LLMs/VLMs with LoRA, QLoRA, PPO, DPO and more.

ai-development ai-framework ai-train

Megatron-LM

2019

NVIDIA

NVIDIA’s model-parallel training library for GPT-like transformers at multi-billion-parameter scale.

ai-development ai-framework ai-train nvidia

Overview

FastChat offers end-to-end tooling: fine-tuning recipes, benchmarking (MT-Bench), and a scalable serving stack with OpenAI-compatible APIs and web UI.

Key Capabilities

Distributed multi-model serving with dynamic routing
One-click Vicuna training scripts & evaluation harness
Chatbot Arena integration and Elo leaderboard
vLLM & xFasterTransformer back-ends for high TPS
REST/GRPC APIs plus React/Web UI frontend