AIAny
AI Audio2023
Icon for item

Amphion

Unifies text-to-speech, singing voice synthesis, voice conversion, and text-to-audio/music in one PyTorch framework with shared vocoders and a common evaluation pipeline. Ships recipes, pretrained checkpoints, and visualizations of classic models.

Introduction

Audio generation research has a reproducibility problem: text-to-speech, singing synthesis, and voice conversion each grew their own ecosystem of single-purpose repos, incompatible data pipelines, and bespoke evaluation scripts. Amphion's bet is that these tasks share far more than they differ — the same vocoders, the same feature extractors, the same metrics — so it puts them under one roof and turns any-input-to-audio into a single, comparable workflow.

What Sets It Apart
  • Breadth under one framework: TTS, singing voice synthesis (SVS), voice conversion (VC), singing voice conversion (SVC), accent conversion, and text-to-audio all share infrastructure, so swapping a vocoder or comparing two acoustic models doesn't mean rebuilding the stack.
  • Classic-model visualizations: rather than only shipping weights, it diagrams how landmark architectures actually work — a deliberate on-ramp for junior researchers who would otherwise reverse-engineer papers.
  • A unified evaluation suite: objective metrics for speech and singing live next to the models, so results are reproducible against a shared yardstick instead of each paper's private script.
Great Fit / Look Elsewhere

Great fit if you're a researcher or grad student who wants reproducible baselines across several audio-generation tasks, or you want to compare vocoders and acoustic models on equal footing. Look elsewhere if you need a turnkey production TTS API — Amphion is a research toolkit, recipe- and training-heavy, and assumes comfort with PyTorch, GPUs, and reading papers rather than calling a hosted endpoint.

Information

  • Websitegithub.com
  • OrganizationsThe Chinese University of Hong Kong, Shenzhen, Shanghai AI Lab, Shenzhen Research Institute of Big Data
  • AuthorsOpenMMLab, Amphion contributors
  • Published date2023/11/15

Categories

More Items

GitHub

Runs a self-hosted meeting bot and transcription API that joins Google Meet, Teams and Zoom and streams speaker-attributed transcripts in real time. Compiles meetings into a git-backed Markdown workspace and runs sandboxed agents on your infrastructure; Apache-2.0 and air-gap capable.

Hugging Face
AI Audio2026

Generates streaming, low‑latency neural speech for real‑time dialogue by autoregressively producing audio frames as text arrives; joint text–speech training preserves natural prosody. Optimized for vLLM streaming (~50 ms first chunk), supports short‑clip voice cloning and four languages.

Hugging Face
AI Audio2026

Transcribes Arabic speech to text using a CohereLabs-trained ASR model compatible with the Hugging Face Transformers pipeline. Provides safetensors weights, endpoint compatibility and a DOI-tagged release; suitable for Arabic transcription workflows but may require adaptation for diverse dialects or noisy audio.