AIAny
AI Audio2023
Icon for item

Amphion

Unifies text-to-speech, singing voice synthesis, voice conversion, and text-to-audio/music in one PyTorch framework with shared vocoders and a common evaluation pipeline. Ships recipes, pretrained checkpoints, and visualizations of classic models.

Introduction

Audio generation research has a reproducibility problem: text-to-speech, singing synthesis, and voice conversion each grew their own ecosystem of single-purpose repos, incompatible data pipelines, and bespoke evaluation scripts. Amphion's bet is that these tasks share far more than they differ — the same vocoders, the same feature extractors, the same metrics — so it puts them under one roof and turns any-input-to-audio into a single, comparable workflow.

What Sets It Apart
  • Breadth under one framework: TTS, singing voice synthesis (SVS), voice conversion (VC), singing voice conversion (SVC), accent conversion, and text-to-audio all share infrastructure, so swapping a vocoder or comparing two acoustic models doesn't mean rebuilding the stack.
  • Classic-model visualizations: rather than only shipping weights, it diagrams how landmark architectures actually work — a deliberate on-ramp for junior researchers who would otherwise reverse-engineer papers.
  • A unified evaluation suite: objective metrics for speech and singing live next to the models, so results are reproducible against a shared yardstick instead of each paper's private script.
Great Fit / Look Elsewhere

Great fit if you're a researcher or grad student who wants reproducible baselines across several audio-generation tasks, or you want to compare vocoders and acoustic models on equal footing. Look elsewhere if you need a turnkey production TTS API — Amphion is a research toolkit, recipe- and training-heavy, and assumes comfort with PyTorch, GPUs, and reading papers rather than calling a hosted endpoint.

Information

  • Websitegithub.com
  • OrganizationsThe Chinese University of Hong Kong, Shenzhen, Shanghai AI Lab, Shenzhen Research Institute of Big Data
  • AuthorsOpenMMLab, Amphion contributors
  • Published date2023/11/15

Categories

More Items

Hugging Face
AI Audio2026

Generates low-latency, instruction-driven English and Chinese speech for voice cloning, voice design, and directed performances; supports real-time streaming, reference-free voice creation, and reference-guided cloning. Open-weight PyTorch model released under a research/non-commercial license with GPU recommendations.

Hugging Face
AI Audio2026

Zero-shot multilingual text-to-speech checkpoint for speech generation and voice cloning with a compact footprint. Features an ~170M-parameter main model plus a bundled ~120M-parameter codec decoder, with primary support for Chinese and English; other languages show more variable quality and long/noisy references reduce fidelity.

GitHub
AI Audio2026

Local-first voice workflows for cloning, multi-engine TTS/ASR, video dubbing, dictation, transcription and audiobook production across 646 languages. Desktop app with a local OpenAI-compatible API, engine catalogue (TTS/ASR/LLM), and explicit opt-ins for remote features to keep audio and projects on-device.