Bet that one neural net, scaled with HPC, could transcribe both English and Mandarin without hand-built pipelines — reaching human-competitive accuracy by training fast enough to iterate on architecture in days, not weeks.
AI Train2017
Sequence modeling toolkit for training custom models for translation, summarization, and language modeling. Reference implementation behind RoBERTa, BART, mBART, XLM-R, and wav2vec 2.0, with multi-GPU and mixed-precision training.
AI Audio2019
Build, fine-tune, and deploy speech AI on NVIDIA GPUs: ASR, text-to-speech, and speech LLMs in one PyTorch stack. Ships pretrained Parakeet/Canary recognition and Magpie TTS checkpoints; broader LLM/multimodal training now lives in v2.7.0.
AI Audio2019
Provides a toolkit and codebase for building, training, and deploying speech and multimodal models — Automatic Speech Recognition, Text-to-Speech, and speech-aware LLMs — with modular neural components and pre-trained checkpoints for PyTorch. Supports streaming/low-latency inference, multi-language models, and optional compiled kernels for acceleration.
AI Audio2022
Multilingual sequence-to-sequence speech model and toolkit for speech recognition, speech-to-text translation, and language identification. Offers several model sizes (tiny → large/turbo) for different speed/accuracy trade-offs and ships with a CLI and Python API for offline transcription workflows.
AI Audio2022
Alibaba DAMO Academy, Northwestern Polytechnical University (NWPU) +5Alibaba DAMO Academy, ModelScope
Bundles ASR, voice activity detection, punctuation, and speaker diarization into one pipeline, with pretrained models like Paraformer and SenseVoice. SenseVoice runs ~17x realtime on CPU; also ships streaming ASR and an OpenAI-compatible API.
AI Audio2023
Reimplements OpenAI's Whisper speech-to-text on the CTranslate2 inference engine, running up to 4x faster at the same accuracy while using less memory. Adds a batched pipeline, 8-bit quantization, VAD filtering, and word-level timestamps.
AI Audio2023
Converts microphone or streamed audio to text with sub-second latency, pairing WebRTC/Silero voice-activity detection and wake-word activation with swappable local backends — faster-whisper by default, plus whisper.cpp, Moonshine, and sherpa-onnx.
AI Video2023
Converts videos between languages by transcribing audio, translating subtitles, and producing AI dubbing—supports local and online ASR/LLM/TTS providers, speaker diarization, voice cloning, and GUI/CLI workflows for batch or headless use.
AI Audio2023
Terminal CLI for on-device Whisper ASR using Hugging Face Transformers + Optimum, with optional Flash Attention 2, batching, and diarization support — focused on high-throughput transcription on NVIDIA GPUs and Apple Silicon (mps).
AI Audio2023
Performs speaker diarization (who spoke when) with pyannote-audio: combines voice-activity detection, speaker-change and overlapped-speech detection to produce time-stamped speaker segments; compatible with Hugging Face Endpoints and ASR pipelines.
AI Client2023
Hands-free voice-first companion with a Live2D avatar for real-time conversations with LLMs. Cross-platform web and desktop clients, runs locally or via cloud APIs, supports local ASR/TTS and modular customization for personas and models.