AIAny
AI Audio2024
GitHub

MLX-Audio

Runs text-to-speech, speech-to-text, and speech-to-speech models natively on Apple Silicon via MLX — no CUDA or cloud. Supports 20+ TTS and 15+ STT models (Kokoro, Whisper, Qwen3), low-bit quantization, an OpenAI-compatible API, and a Swift package.

Introduction

Almost every open speech model ships assuming an NVIDIA GPU, so running TTS or transcription on a Mac usually means a slow CPU fallback or a cloud bill. This project rebuilds the whole speech stack — synthesis, transcription, and voice conversion — directly on Apple's MLX runtime, so a MacBook's unified memory does the work locally with no CUDA and no API keys.

What Sets It Apart
  • Breadth over a single model: 20+ TTS engines (Kokoro, Qwen3-TTS, OmniVoice) and 15+ STT engines (Whisper, Qwen3-ASR, Parakeet) share one MLX-native interface, so swapping models is a config change rather than a rewrite.
  • Quantization from 3-bit to 8-bit lets multi-billion-parameter voices fit in the RAM of a consumer Mac — the difference between "runs" and "won't load."
  • An OpenAI-compatible REST endpoint means existing speech clients point at localhost unchanged, and a Swift package drops the same inference into iOS/macOS apps.
  • Voice cloning, adjustable speed, and a 3D-visualizer web UI come bundled rather than as separate tools.
Who It's For

Great fit if you live on Apple Silicon and want local, private speech — offline transcription, on-device TTS for an app, or trying the latest voice models without renting a GPU. Look elsewhere if you deploy on Linux/NVIDIA servers, need Windows support, or depend on a model that hasn't been ported to MLX yet; the project tracks the MLX ecosystem and a single maintainer's release pace.

Information

  • Websitegithub.com
  • OrganizationsIndependent
  • AuthorsPrince Canuma (Blaizzy)
  • Published date2024/11/27

Categories

More Items

GitHub
AI Audio2025

Provides text-to-speech synthesis and one-shot voice cloning via compact ONNX models and a larger KittenTTS 2 speech model. Offers CPU-first inference (25–506 MB models), 47 built-in voices, multilingual output, expression tags, streaming, and a Python SDK.

Hugging Face
AI Audio2026

Generates directed, multi-character, long-form speech from text with natural-language directions and voice cloning. Built on a 35B MoE backbone (3B active per token) with a codebook multi-token-prediction decoder to enable multi-turn revision, infilling edits, streaming, and expressive control.

GitHub
AI Audio2025

Generates full songs from lyrics and style prompts by first writing an editable melody-and-chord score and then rendering vocals and accompaniment. Unifies symbolic planning and audio synthesis in one pipeline, enabling readable score inspection, zero‑shot covers, and agent-driven editing.