Most streaming ASR systems trade off user-facing stability for accuracy by emitting provisional tokens that are later revised, causing distracting text flicker in live captions and agent pipelines. R2T2 flips that trade-off: it exposes only stable prefixes so incremental outputs are final, while keeping latency in the 200–600 ms range and accuracy close to offline recognition.
Key Capabilities
- Longest Stable Prefix (LSP) streaming: the model is trained and decoded to decide when a token sequence is safe to commit, eliminating revisions in the incremental transcript — so clients can consume emitted text immediately without reconciliation logic.
- Fine-grained chunking: configurable decoding chunks from 80 ms to 2 s let you choose latency vs. accuracy operating points; typical setups report average latencies of ~200–600 ms.
- Robust accuracy and multilingual support: built as a Qwen3-ASR fine-tune with techniques like stable-prefix data and token-level audio segmentation; optimized for Chinese and English and usable on many other languages.
- Production-friendly integrations: ships examples for vLLM and Hugging Face transformers backends, a WebSocket server for multi-client streaming, and support for context/hotword prompts.
Who it's for and trade-offs
Great fit if you need real-time, append-only transcripts for live captioning, simultaneous translation pipelines, LLM agents that act on streaming text, or downstream NLP systems that cannot tolerate text revisions. The project includes server/client examples and a vLLM-based backend for high-throughput inference.
Look elsewhere if you require fully permissive model weights for arbitrary model training or commercial deployment at very large scale without reviewing license terms: the code is Apache-2.0 but the model weights are distributed under NetEase Model Use License Agreement, which imposes reuse and commercial-scale restrictions. Also, production deployment expects GPU/vLLM or compatible runtime; CPU-only deployments will have higher latency.
Where it fits
R2T2 positions itself between pseudo-streaming systems (which revise partial outputs) and offline ASR: it aims to deliver offline-grade accuracy while providing stable, low-latency incremental outputs for real-time applications.
Implementation notes
The project includes Docker configurations, a streaming WebSocket server, and Python examples for both offline and streaming inference. Typical usage configures chunk size, unfixed token window, and optional hotword/context prompts; example runtimes use vLLM for throughput and audio.cpp/gguf derivatives for lightweight local inference.
