Agent training and agentic evaluation hinge on breadth and reproducibility of task data — TaskTrove centralizes that need by aggregating a very large, source-preserving corpus of Harbor-compatible task binaries. By keeping original Parquet shards and labeling which tasks include verifiers, it lets teams run large-scale datagen or RL evaluation pipelines without reformatting or losing provenance.
What Sets It Apart
- Large, diverse corpus: >750,000 unique tasks collected from 100+ HuggingFace sources, covering code, sandboxed RL tasks, and SFT-style instructions — useful for both behavior cloning and RL datagen.
- Source-preserving storage: each original dataset is retained under an org__name directory with raw Parquet shards and READMEs preserved exactly as downloaded, minimizing transformation-induced drift.
- Verifier-aware split: tasks are categorized into those that include verifiers (suitable for automated RL scoring) and those without verifiers (better for teacher-model or human grading), so you can pick data tailored to your pipeline.
- Designed for Harbor workflows: tasks are stored as Harbor task binaries and intended to plug directly into Harbor-based datagen and trace-generation runs (the project documents examples and flags for running large vLLM-backed datagen jobs).
- Linked to AgentTrove traces: TaskTrove served as the task source for AgentTrove’s ~1.7M aggregated trajectories, demonstrating end-to-end datagen usage and reproducibility.
Who It's For and Trade-offs
Great fit if you are running large-scale agent datagen or RL evaluation and need a single, provenance-preserving repository of Harbor-compatible tasks to feed Harbor/vLLM pipelines. It’s also useful for teams who want to reproduce or extend AgentTrove traces.
Look elsewhere if you require small, hand-curated benchmark sets with human-verified gold labels, need non-English task coverage (TaskTrove is English-focused), or expect pre-normalized, unified schemas beyond preserving the original source files. Note the dataset is large (100K–1M size category) and assumes downstream tooling (Harbor, vLLM, trace pipelines) for efficient use.
