Why this matters
Distillation outputs from many frontier models are noisy and scattered across sources; this dataset unifies 16M+ distilled samples and 7,090 compressed repository snapshots into a single, streaming-ready Hugging Face dataset. That consolidation makes it practical to experiment with large-scale SFT, agentic coding traces, and cross-domain reasoning curricula without rebuilding heterogeneous pipelines.
What Sets It Apart
- Unified extraction and schema: every sample uses a consistent 5-field schema (source, source_dataset, instruction, response, category), so mixing shards or interleaving categories is straightforward for curriculum training.
- Large, mixed modality of training signals: ~11M+ coding traces, 2.7M+ reasoning traces, dedicated cybersecurity and math subsets, plus 7,090 repo archives for full-file context — so users can combine instruction tuning, code pretraining, and repository-context tasks from one repo.
- Streaming-first and on-demand archives: 516 JSONL shards for direct streaming and compressed tar.gz archives for selective full-file retrieval, reducing storage overhead while preserving fidelity of original repo snapshots.
- Provenance fields and per-sample upstream ids: every row includes its upstream dataset slug, enabling license checks and source-level filtering during dataset curation.
Who It's For and Trade-offs
Great fit if you are fine-tuning open-source LLMs for instruction following or code agents, building multi-stage curricula (math/science → coding → distilled reasoning), or researching distillation artifacts at scale. Look elsewhere if you require authoritative, human-verified ground truth for high-stakes domains: most samples are model-generated and may hallucinate. Also expect truncated fields (instruction/response capped ≈4,000 chars) unless you retrieve raw files from the archives.
Where It Fits
This dataset sits between raw upstream model-output dumps and tightly curated human-labeled SFT corpora: it trades gold-standard human labeling for breadth, reproducible provenance, and engineering-ready formats that speed experiments in SFT, code-agent training, and multi-source distillation research.
