Open mathematical problems function as a hard, non-memorisable stress test for AI: they require literature triage, formulation, partial progress and rigorous argumentation rather than single-answer recall. UnsolvedMath packages thousands of such problems with structured metadata and AI-assisted research audits so models can be evaluated on realistic research-style workflows rather than narrow QA tasks.
What Sets It Apart
- Size and coverage: 5,426 problems spanning ~17 mathematical domains and multiple curated collections (Millennium Prize, Hilbert, Erdős, AMR lists). So what: gives benchmarks room to test across domain, difficulty, and problem-set distributions instead of a single-task slice.
- Machine-readable LaTeX + structured fields: each problem includes a LaTeX statement, category, difficulty, provenance and status fields. So what: simplifies automated parsing, rendering and programmatic filtering for evaluation pipelines and tool-augmented research episodes.
- AMR research audit and per-problem reports: thousands of AMR-triaged reports, per-problem research_classification and research_results files. So what: enables experiments that compare model-led literature triage, partial-progress generation, and citation-aware verification against a precomputed research baseline.
- Designed for research workflows: dataset provides multiple JSON files (problems.json, research_results.json, categories, statistics) plus example loading snippets. So what: integrates easily with model evaluation, proof-checking workflows, and tool chains that combine symbolic/math systems and retrieval.
Who it's for + Tradeoffs
Great fit if you want to evaluate or develop models on genuine mathematical research tasks (literature search, proof sketching, counterexample finding, automated triage) across varied difficulty levels and curated problem sets. It is also useful for education and LaTeX-processing research.
Look elsewhere if you need a small, fully verified corpus of peer-reviewed theorem proofs for formal verification — the dataset prioritises breadth and machine-assisted triage over peer-reviewed closure. Note: status fields and AMR research notes can change; some SOLVED-* or PARTIAL-PROGRESS entries are machine-assisted and should be independently verified before citation.
Where It Fits
Compared with closed private problem collections used for automated numeric verification, UnsolvedMath emphasizes open problems, LaTeX fidelity and human-readable research reports aimed at evaluating end-to-end research behaviours (idea generation, literature triage, partial proofs) rather than single-answer scoring.
