Most benchmarks either give static examples or require complex runtimes. This dataset collection gives you self-contained, deterministically-gradable data-analysis tasks so you can reliably measure an agent or LLM without spinning up sandboxed environments.
What Sets It Apart
- Verified, checkable answers: every question–answer pair was reproduced by strong agent rollouts in a sandbox and is graded by a deterministic grader (exact → numeric tolerances → list/percent normalization → symbolic equivalence), so evaluation does not depend on an LLM judge. This makes reward signals stable across experiments.
- Real tabular inputs at scale: built from 471 Kaggle notebooks, the release exposes 5,000 train tasks plus two harder held-out splits (250 test, 144 eval) with explicit difficulty tiers. That lets you benchmark generalization to harder unseen tasks, not just in-distribution performance.
- Multiple consumption modes: use the plain dataset rows for prompting and grading, the SFT split (4,677 verified agent trajectories) for supervised fine-tuning, or the Harbor suites for runnable sandboxed environments — the same underlying tasks across formats.
Who It's For and Trade-offs
Great fit if you want reproducible, rewardable tasks for training or evaluating data-centric agents and small LLMs (prompting, SFT, or RL hill-climbing). The deterministic grader and real-table inputs make it especially useful for measuring exact-answer accuracy and for curriculum vs. shuffled training experiments. Look elsewhere if you need multimodal benchmarks, synthetic adversarial examples, or large-scale unlabeled corpora — this collection focuses on tabular data-analysis tasks with strict answer checking and is not a general-purpose language or vision dataset.
