Most speech systems make one side wait: they listen, finish processing, then respond. This model instead treats listening and speaking as simultaneous streams, aiming to capture the interruptions, backchannels, and overlapping turns that make conversation feel natural. That makes it a research platform for real-time spoken interaction, not a drop-in factual assistant.
Key Capabilities
- At 7.5 frames per second, each 133 ms step consumes user audio while predicting the model’s next audio and text. A shared timeline avoids serializing the conversation into listen-then-speak turns.
- Audio is represented as a 64-dimensional continuous latent per frame and generated in one MeanFlow sampling step. This avoids discrete audio tokens and a separate audio-generation transformer, reducing sequential generation work.
- A character prompt and reference clip control persona and voice. Training covers over 160 languages, with primary focus on English, Spanish, Portuguese, German, French, Chinese, Japanese, Russian, and Korean.
Who It Suits
A good fit for researchers exploring low-latency speech-to-speech interaction, turn-taking, and voice-conditioned dialogue. Look elsewhere if you need a dependable instruction-following or fact-answering assistant: this is a pretrained base model, and its voice imitation should only be used with voices you have permission to use. Real-time use also benefits from capable local hardware and wired audio devices.