AIAny
AI Audio2023
Icon for item

Amphion

Unifies text-to-speech, singing voice synthesis, voice conversion, and text-to-audio/music in one PyTorch framework with shared vocoders and a common evaluation pipeline. Ships recipes, pretrained checkpoints, and visualizations of classic models.

Introduction

Audio generation research has a reproducibility problem: text-to-speech, singing synthesis, and voice conversion each grew their own ecosystem of single-purpose repos, incompatible data pipelines, and bespoke evaluation scripts. Amphion's bet is that these tasks share far more than they differ — the same vocoders, the same feature extractors, the same metrics — so it puts them under one roof and turns any-input-to-audio into a single, comparable workflow.

What Sets It Apart
  • Breadth under one framework: TTS, singing voice synthesis (SVS), voice conversion (VC), singing voice conversion (SVC), accent conversion, and text-to-audio all share infrastructure, so swapping a vocoder or comparing two acoustic models doesn't mean rebuilding the stack.
  • Classic-model visualizations: rather than only shipping weights, it diagrams how landmark architectures actually work — a deliberate on-ramp for junior researchers who would otherwise reverse-engineer papers.
  • A unified evaluation suite: objective metrics for speech and singing live next to the models, so results are reproducible against a shared yardstick instead of each paper's private script.
Great Fit / Look Elsewhere

Great fit if you're a researcher or grad student who wants reproducible baselines across several audio-generation tasks, or you want to compare vocoders and acoustic models on equal footing. Look elsewhere if you need a turnkey production TTS API — Amphion is a research toolkit, recipe- and training-heavy, and assumes comfort with PyTorch, GPUs, and reading papers rather than calling a hosted endpoint.

Information

  • Websitegithub.com
  • OrganizationsThe Chinese University of Hong Kong, Shenzhen, Shanghai AI Lab, Shenzhen Research Institute of Big Data
  • AuthorsOpenMMLab, Amphion contributors
  • Published date2023/11/15

Categories

More Items

GitHub
AI Audio2025

Generates full songs from lyrics and style prompts by first writing an editable melody-and-chord score and then rendering vocals and accompaniment. Unifies symbolic planning and audio synthesis in one pipeline, enabling readable score inspection, zero‑shot covers, and agent-driven editing.

Hugging Face
AI Audio2026

On-device speech-to-text for short clips (up to 30s) in seven languages, yielding transcripts, word-level timestamps and per-frame speech embeddings in a single 16.9 MB model. Runs on the Needle CPU engine with 2–4 bit quantization, supports keyword biasing and returns empty transcripts for silence.

Hugging Face
AI Audio2026

Transcribes English speech into punctuated, capitalized text — a 164 MB quantized ASR model that averages 5.21% WER across seven Open ASR Leaderboard sets. Optimized for on-device and CPU/GPU inference, with fast runtimes on Apple M5 and Docker/GPU support.