These Harbor environments turn OpenAI's formalized Lean theorems into reproducible, RL-style proof-generation tasks where an agent must replace every sorry with a complete Lean proof and pass an exact-match verifier.
What Sets It Apart
- Directly reuses OpenAI's openai/math formalizations but packages them as 369 verified Harbor tasks (from 405 formalized results). Each task bundles the Lean statement, a sandbox image recipe (Lean 4 + Mathlib + Comparator), and a strict verifier that rewards only exact, kernel-accepted proofs. The dataset includes a manifest, task metadata, and a generator for rebuilding images.
- Designed for code agents and theorem-proving research: tasks allocate CPU/memory/disk (typical task: 4 CPUs, 8 GB RAM, 10 GB disk, 4 hours) and the oracle can reproduce OpenAI's proofs for validation. The grading is binary and narrow by design—only the exact original statement proved with no new axioms scores positive reward.
Who It's For and Trade-offs
- Great fit if you build or benchmark code agents, autonomous provers, or RL systems that generate formal Lean proofs and need a reproducible, high-fidelity verifier. Also useful for studying reward design in long-horizon, symbolic tasks.
- Look elsewhere if you need permissive grading, partial-credit evaluation, or larger-scale formalizations: the verifier is intentionally strict (exact-statement match) and some proofs require >8 GB memory or external Lean libraries and so are excluded or flagged. Expect hard, research-level problems rather than toy tasks.
