AIAny
AI Audio2024
Icon for item

MLX-Audio

Runs text-to-speech, speech-to-text, and speech-to-speech models natively on Apple Silicon via MLX — no CUDA or cloud. Supports 20+ TTS and 15+ STT models (Kokoro, Whisper, Qwen3), low-bit quantization, an OpenAI-compatible API, and a Swift package.

Introduction

Almost every open speech model ships assuming an NVIDIA GPU, so running TTS or transcription on a Mac usually means a slow CPU fallback or a cloud bill. This project rebuilds the whole speech stack — synthesis, transcription, and voice conversion — directly on Apple's MLX runtime, so a MacBook's unified memory does the work locally with no CUDA and no API keys.

What Sets It Apart
  • Breadth over a single model: 20+ TTS engines (Kokoro, Qwen3-TTS, OmniVoice) and 15+ STT engines (Whisper, Qwen3-ASR, Parakeet) share one MLX-native interface, so swapping models is a config change rather than a rewrite.
  • Quantization from 3-bit to 8-bit lets multi-billion-parameter voices fit in the RAM of a consumer Mac — the difference between "runs" and "won't load."
  • An OpenAI-compatible REST endpoint means existing speech clients point at localhost unchanged, and a Swift package drops the same inference into iOS/macOS apps.
  • Voice cloning, adjustable speed, and a 3D-visualizer web UI come bundled rather than as separate tools.
Who It's For

Great fit if you live on Apple Silicon and want local, private speech — offline transcription, on-device TTS for an app, or trying the latest voice models without renting a GPU. Look elsewhere if you deploy on Linux/NVIDIA servers, need Windows support, or depend on a model that hasn't been ported to MLX yet; the project tracks the MLX ecosystem and a single maintainer's release pace.

Information

  • Websitegithub.com
  • OrganizationsIndependent
  • AuthorsPrince Canuma (Blaizzy)
  • Published date2024/11/27

Categories

More Items

Hugging Face
AI Audio2026

Generates low-latency, instruction-driven English and Chinese speech for voice cloning, voice design, and directed performances; supports real-time streaming, reference-free voice creation, and reference-guided cloning. Open-weight PyTorch model released under a research/non-commercial license with GPU recommendations.

Hugging Face
AI Audio2026

Zero-shot multilingual text-to-speech checkpoint for speech generation and voice cloning with a compact footprint. Features an ~170M-parameter main model plus a bundled ~120M-parameter codec decoder, with primary support for Chinese and English; other languages show more variable quality and long/noisy references reduce fidelity.

GitHub
AI Audio2026

Local-first voice workflows for cloning, multi-engine TTS/ASR, video dubbing, dictation, transcription and audiobook production across 646 languages. Desktop app with a local OpenAI-compatible API, engine catalogue (TTS/ASR/LLM), and explicit opt-ins for remote features to keep audio and projects on-device.