Breeze TTS 2 matters because interactive voice experiences need more than raw naturalness: they need controllable identity, on-the-fly direction, and millisecond-scale responsiveness. This model packages natural-language voice design, reference-based cloning, and low-latency streaming in a single open-weight PyTorch system so creators can iterate on characters and performances in real time without switching toolchains.
Key Capabilities
- Voice design: produce a distinctive voice from a plain-language instruction (e.g., “warm, thoughtful young woman”) so you can generate many role-appropriate voices without collecting reference audio — useful for games, audiobooks, and conversational agents.
- Voice clone & direction: clone timbre and style from clean reference audio plus its transcript, then steer emotion, pace, and delivery with natural-language instructions, preserving speaker identity while changing performance.
- Low-latency streaming: supports eager streaming and an optimized fast path with CUDA graphs; reported time-to-first-audio under 40 ms on warmed H100 and ≈0.32 RTF on H100 fast path, enabling responsive conversational flows.
- GPU-efficient inference: eager execution uses ~7.7 GiB of GPU memory (12 GiB recommended); fast paths require more memory but reduce runtime latency, letting small teams run real-time pipelines on modern NVIDIA GPUs.
Who it's for and tradeoffs
Great fit if you need controllable, interactive TTS for prototypes, indie games, dialogue systems, or studio workflows that value voice variation and real-time responsiveness. It’s also useful for researchers benchmarking voice design/direction. Look elsewhere if you require a commercial license out of the box (commercial use requires written authorization), if you must run entirely on CPU, or if you need a production SLA/hosted service — this release is focused on research/non-commercial use and self-hosted inference.
Where it fits
Breeze TTS 2 sits between research-weight open models and hosted commercial TTS: it exposes strong instruction-following and expressive control comparable to frontier systems while remaining self-hostable for experimentation. Use it to iterate on voice personas and performance behaviors locally; pair with downstream latency engineering or hosted services when moving to production.
