Training a model to read DNA takes more than feeding it genomes: it must also retain general language ability, answer biology questions, and perform sequence tasks that can be checked exactly. This dataset makes that training mix inspectable, from broad mid-training sources through task-specific tuning and verifiable RL prompts.
What Sets It Apart
- The mid-training corpus contains 514.3B estimated tokens, with the Carbon-X run consuming 302B according to a documented sampling recipe. DNA accounts for 55% of the training mix, alongside biology text, English, and instruction data.
- Separate configurations preserve source and task boundaries, making it possible to study or reuse genomic sequences, DNA-containing papers and web text, and instruction data without treating them as one undifferentiated corpus.
- The SFT and RL sets target concrete DNA capabilities, including sequence computation, classification, continuation, and variant-effect prediction. RL prompts include answer keys or constraints that can be checked by reward code.
- The released data and recipes support reproducibility and analysis of how training data shapes a DNA-focused model, rather than offering only model weights.
Who It’s For
A good fit for researchers reproducing or adapting DNA-language-model training, auditing data mixtures, or building evaluations for sequence tasks. Look elsewhere if you need a general-purpose biology corpus or unrestricted reuse of every upstream record: FineDNA applies its own Apache 2.0 license, but some underlying sources have separate terms that must be checked.
