Real-time multimodal conversation — seeing, listening, and speaking at the same time — has mostly belonged to cloud giants like GPT-4o. The bet here is that the same experience can run on the device in your pocket. The 4.5 release folds vision, speech, audio, and video into a single 9B model that scores 77.6 on OpenCompass while needing roughly 19GB to run, and the earlier 2.6 was already matching GPT-4o-202405 on multimodal live streaming.
Key Capabilities
- Full-duplex streaming: it ingests continuous video and audio while generating text and speech out, so it can react mid-stream instead of waiting for you to finish — closer to a live call than turn-based chat.
- On-device focus: a 9B omnimodal model that runs through llama.cpp, Ollama, and vLLM and is small enough to target phones and tablets, putting multimodal inference within reach without a datacenter.
- Real-time bilingual voice with voice cloning and configurable emotion and speed, plus proactive responses driven by ongoing scene understanding rather than only replying when prompted.
- Frontier-adjacent quality: 4.5 approaches Gemini 2.5 Flash on vision, speech, and full-duplex live streaming despite the small footprint.
Great Fit If
Reach for it if you're building voice assistants, live-camera apps, or edge devices that need omnimodal understanding without round-tripping to a cloud API, or if you want to study how open-source full-duplex streaming is actually engineered. Look elsewhere if you only need a text LLM (the multimodal stack is overhead you won't use), if 19GB-class hardware is out of reach for your real-time target, or if your use case is commercial — check the model's license terms first.
