Most voice assistants that feel responsive quietly stream your audio to a cloud API. This project takes the opposite bet: a fully local cascade where every stage — detection, transcription, reasoning, synthesis — is an open model you choose and can swap, so latency, privacy, and language coverage become things you tune rather than rent.
How the Pipeline Fits Together
- Four stages chained end to end: Silero VAD detects speech, an STT model (Whisper, Parakeet TDT, or Paraformer) transcribes it, an LLM reasons over it, and a TTS model (Kokoro-82M, ChatTTS, or Qwen3-TTS) speaks the reply. Because the boundaries are explicit, you can debug or upgrade any single stage.
- Each component is independent: swap a heavy Whisper for a distilled variant, or point the LLM at a local Transformers/MLX model versus OpenAI, vLLM, OpenRouter, or llama.cpp — without touching the rest of the chain.
- Three deployment shapes ship in the box: pure local mic/speaker, a server/client TCP setup, and an OpenAI-compatible
/v1/realtimeWebSocket endpoint, so clients already built for that protocol work unchanged. - It runs across CUDA, CPU, Apple Silicon (MPS), and Docker, and handles English, French, Spanish, Chinese, Japanese, and Korean with auto-detection plus live partial transcripts.
Who It's For
Great fit if you want a privacy-preserving or offline voice agent, need to benchmark different STT/LLM/TTS combinations, or want to self-host an endpoint that speaks the OpenAI realtime protocol. Look elsewhere if you need a polished turnkey product — this is a framework that expects you to assemble and tune the pieces, and a cascaded pipeline carries more latency than an end-to-end speech model.
