Progressively prunes and distills audio encoders for speech LLMs to cut inference cost while preserving decoder-facing embeddings, using behavioral probes, representation alignment, cross-scale distillation and LoRA finetuning; reports reduced macro-error on Chinese–English benchmarks.
Provides a realtime audio–language foundation model for fully duplex spoken interaction that listens, interprets acoustic cues, reasons, and speaks in parallel. Key features include Deep Perception for acoustic understanding, Seamless Duplex for synchronized streams, and Think‑While‑Speaking for parallel private reasoning; reports strong benchmark results.
Encodes real audio into YuE2 semantic tokens and ships a matching NAR-branch LoRA so the YuE2 decoder renders realistic latents—enables tokenizing recordings, training artist LoRAs, and generating songs or covers. Includes an 8-layer tokenizer head and a rank-32 decoder LoRA; requires the YuE2 base models and GPU resources.
Provides transcripts, translations, and multi-task text labels for speech across 48 languages and about 463,000 hours. The release contains manifests only; audio must be obtained from the original data providers.
Performs low-latency streaming speech-to-text, emitting one token per selectable 80/120/160 ms clock with configurable transcription delay and a 30s rolling KV cache for unlimited 24/7 transcription. Bilingual (zh/en) and includes semantic VAD.
Performs speaker diarization (who spoke when) for live and recorded audio using an open-weight, 100M-parameter streaming-capable model that supports up to eight anonymous speaker channels, overlapping speech, chunked processing, and configurable latency for ASR integration.
Evaluates multimodal memory in large audio language models by testing four acoustic evidence types and four memory operations across multi-session spoken histories, stratified across 8K–64K token context budgets.
Transcribes English speech into punctuated, capitalized text — a 164 MB quantized ASR model that averages 5.21% WER across seven Open ASR Leaderboard sets. Optimized for on-device and CPU/GPU inference, with fast runtimes on Apple M5 and Docker/GPU support.
Collection of 1.44M unique Turkish voice‑assistant sentences (≈1,819 hours estimated), normalized for TTS and organized by service domains (appointments, banking, e‑commerce). Designed for training and evaluating TTS and text-generation models; licensed CC BY 4.0 with required attribution.
On-device speech-to-text for short clips (up to 30s) in seven languages, yielding transcripts, word-level timestamps and per-frame speech embeddings in a single 16.9 MB model. Runs on the Needle CPU engine with 2–4 bit quantization, supports keyword biasing and returns empty transcripts for silence.
Provides 3,451 hours (2,051,810 clips) of AI‑generated 48 kHz Turkish speech with transcripts, spoken forms and per‑clip voice descriptions for TTS and ASR development. Includes 2,752 designed voices and is licensed CC BY 4.0 / CC BY‑SA 4.0 (attribution to PatientDesk AI required).
Converts Turkish text to 48 kHz speech offline using a compact 5.6M DiT acoustic model plus a 3M vocoder (8.6M parameters, ~34 MB). Streams audio with very low latency (~3.86 ms first audio on an RTX 4090), supports fast batching and runs under an Apache‑2.0 license.