Connects Claude (via the Model Context Protocol) to Ableton Live so the LLM can create and edit tracks, clips, instruments, and control playback through a socket-based MCP server and an Ableton MIDI Remote Script.
Runs open-source LLMs and multimodal models entirely on mobile devices for offline, private inference. Offers Agent Skills, Thinking Mode, Ask Image, audio scribe, model management and benchmarks, with Gemma 4 and Hugging Face integration.
Open-source TTS that clones a voice from a short reference clip across 23+ languages, with adjustable emotional intensity via exaggeration/cfg controls and a built-in Perth neural watermark on every output.
A toolkit and open-weights system for real-time streaming music generation — offers two model sizes (230M / 2.4B), a Python inference library (JAX/MLX), and a C++ engine optimized for Apple Silicon for embedding into DAWs and apps; real-time streaming requires M‑series chips.
Turns OpenAI Whisper into a live streaming transcriber: audio flows in over WebSocket and text returns word-by-word instead of after full utterances. Adds SimulStreaming and LocalAgreement decoding, Silero VAD, and speaker diarization, all self-hosted.
Synthesizes up to 90 minutes of multi-speaker speech in one pass, with as many as four voices in a single conversation. Pairs continuous acoustic and semantic tokenizers at a 7.5 Hz frame rate with a next-token diffusion head on an LLM backbone.
Isolates any single sound from a complex audio mixture using a text description, a visual cue from a video frame, or a time span, returning both the isolated target and the residual. Released in small, base, and large sizes plus visual-prompt variants.
Provides a 10,000-hour Sichuanese (Chuan-Yu) speech corpus with rich annotations (timestamps, speaker age/gender/emotion, SNR, DNSMOS) and unified metadata for ASR and TTS research; includes metadata.jsonl, evaluation benchmarks, and an LLM-assisted transcription pipeline.
Build and self-host production voice agents with a drag-and-drop workflow builder, real-time telephony integration, and pluggable LLM/STT/TTS backends. Docker-first with an optional managed cloud offering for teams that want faster onboarding.
On-device macOS dictation that transcribes speech locally and offers an optional local AI enhancement (Fluid Intelligence) for smart formatting and post-processing. Key features: low-latency model choices, live transcription overlay, per-app prompts and privacy-by-default; best on Apple Silicon.
Runs text-to-speech with instant voice cloning fully on-device, from phones to GPUs. Built on small LLM backbones (120M-360M params) plus a 50Hz neural codec; clones a voice from ~3 seconds of audio across English, Spanish, German, and French.
Delivers multilingual, on-device text-to-speech via ONNX Runtime with prebuilt ONNX assets and cross-platform SDKs (Python, Node, mobile); targets low-latency, privacy-preserving TTS with ready demos and 31-language support in v3.