Why this matters
Large-scale agentic trajectories—sequences where an LLM issues shell commands, inspects files, and edits code—are increasingly used to teach models how to interact with repositories and toolchains. The key bottleneck has been execution: building and running thousands of repo-specific Docker images is expensive and discards large fractions of real-world PRs. SWE-ZERO 12M sidesteps that ceiling by intentionally avoiding execution, unlocking an order-of-magnitude scale-up while still exposing models to realistic investigation-and-edit behaviors.
What Sets It Apart
- Execution-free pipeline: Trajectories are generated without building containers or running tests, which removes Docker-related failure modes. So what: it enables coverage of many PRs and repos that containerized pipelines would discard, boosting dataset scale and diversity.
- Massive scale and diversity: ~12.29M rollouts (~112B tokens) spanning 122K PRs, 3.2K repositories, and 16 languages. So what: this provides broad exposure to repository layouts, debugging patterns, and multi-repo idioms that benefit mid‑training of code/agent models.
- Agentic, bash-native format: Each trajectory is a multi-turn sequence of bash-like actions and harnessed outputs (mini-swe-agent-1 format). So what: models learn how to drive a terminal, navigate code, and propose edits without relying on test feedback—useful for instilling tool-use priors.
- Designed for mid-training (not verification): The corpus prioritizes skill acquisition and scale over checked correctness; most rollouts are incomplete or unverified. So what: expect gains in agentic behavior and exploration priors, but not guaranteed task completion or clean patches.
Who it's for — and tradeoffs
Great fit if you want a mid-training corpus to instill agentic tool-use priors in base models, scale pretraining for coding agents, or study large-scale, execution-free agent behaviors. It is valuable for teams wanting broad repository coverage without the infrastructure cost of containerized validation.
Look elsewhere if you need verified, test-passing patches, production-ready fixes, or data where compilation and test success are guaranteed. Because generation is execution-free, patches may not apply, compile, or pass tests; downstream SFT/RL stages and verification are still required for production capability.
Where it fits (short)
SWE-ZERO 12M sits at the execution-free extreme of agentic-coding corpora: higher scale and repo coverage than containerized datasets but without built-time verification. Use it as a mid-training stepping stone before supervised fine-tuning or RLHF on verified task datasets.
Brief methodology notes
Rollouts originate from real PR snapshots and are sampled (100 rollouts/PR) from a compact code model at temperature 1.0, capped at 15 model turns. Each trajectory is a messages list where the assistant issues one shell command per turn and the harness returns the command output. The dataset ships in parquet format under an Apache-2.0 license and is intentionally optimized for throughput and diversity rather than per-sample correctness.
