AIAny
Icon for item

Trelis/duplex-pre-training-yodas3-mimi

Provides 408.25 hours of English speech for dual-track speech-to-speech pre-training. Mimi audio tokens keep it compact, with machine-generated speaker, word-timing, and other annotations; raw audio is not included.

Introduction

Speech-to-speech training often needs more than a transcript: models must learn who spoke, when turns overlap, and how separate voices relate to the mixture. This dataset makes that structure available in token form, but the trade-off is important: it is not a self-contained audio archive, and its tokens are tied to the Mimi representation.

What the Data Enables
  • The 3,630 spans total 408.25 hours of English speech selected for little or no music, giving pre-training a long-form speech source rather than isolated utterances.
  • In 856 two-speaker spans, each voice has its own token track alongside the mixed track. This directly supports learning speaker-specific output and overlapping conversation.
  • Most spans have speaker-attributed transcripts with word timings; some also include emotion labels and vocal-burst annotations. These labels are machine-generated, so they are useful for scale but should not be treated as human-verified ground truth.
  • The audio is represented as eight Mimi codebooks at 12.5 Hz, while raw recordings are omitted. Source coordinates can help locate audio in YODAS3, but changed source files may no longer match the recorded hashes.
Who Should Use It—and the Trade-Off

A good fit for researchers pre-training dual-track speech-to-speech systems or studying turn-taking and speaker separation from tokenized data. Look elsewhere if you need raw waveforms, multilingual coverage, codec-independent audio, or carefully human-verified labels. The dataset follows the YODAS3 CC BY 3.0 license.

Information

Categories

More Items

Scales verifiable gameplay data by having a Builder and visual and coding players co-develop web games. Progress-weighted training improves agent performance on both its own benchmark and an external game benchmark.

Hugging Face

Provides executable software-engineering repair tasks for evaluating coding agents. Its 2,955 synthetic instances span 53 Python repositories and include bug-injection patches, reference repairs, and test information.

Hugging Face

Pairs multilingual documents with JSON Schemas and target extractions for training and evaluating structured information extraction. Its held-out document types make it possible to test generalization beyond the tasks seen during training.