The best math data wasn't missing — it was unfindable.
By early 2024 everyone agreed math needed better training data, and everyone mined the same veins: arXiv papers, textbooks, a curated slice of the web. This paper's bet was that Common Crawl — the raw, public snapshot of the web — already held far more high-quality math than all of those combined, hiding in forums, homework sites and lecture notes that no keyword search would surface. So the team built a finder instead of a corpus: a cheap classifier that hunts math pages, then uses what it caught to learn where else to look. Four rounds of that loop pulled out 120B tokens of math — nine times the best public math corpus.
The second bet is quieter but traveled further. To squeeze the last points out of the finished model, the team wanted reinforcement learning but not PPO's price tag — so they cut PPO's biggest line item, the value model, and let each question's own sampled answers grade each other. That algorithm, GRPO, later became the engine of DeepSeek-R1.
Data you can find beats parameters you can buy: their 7B base model out-scored Minerva 540B — a model 77 times its size.
The bet
Where the math scoreboard stood, and what a 7B model claimed to do about it.
Step 1 · The scoreboard
Open models were 16 points behind on competition math. This paper closed 14 of them.
The measuring stick here is MATH — a benchmark of 5,000 competition problems (algebra, number theory, geometry, …) where the model must write out a full solution, not pick from choices. Grade-school word problems live in a second benchmark, GSM8K. On MATH, in early 2024, the picture was lopsided:
Look at the gap in the chart between the two gray bars at the top and the cluster below: the best open model (InternLM2-Math 20B, 37.7) trailed GPT-4 by about 15 points, and models as big as Qwen 72B did worse. Then a 7B model — small enough to run on one good GPU — posted 51.7. No calculator tools, no sampling 64 answers and voting. One greedy pass.
The obvious suspicion is a bigger model or secret benchmark leakage. The actual answer is two engineering ideas, and they're the two halves of this page: a data pipeline that mined Common Crawl for math (Part B), and a cheaper reinforcement-learning algorithm applied at the very end (Part C).
To keep the pipeline concrete, we'll follow one web page through the whole machine: a question page on mathoverflow.net — a real research-math forum, and the paper's own example in §2.1. Right now it's just one HTML file among billions in Common Crawl. Watch what has to happen for its tokens to end up inside a model.
MATH 51.7% at 7B parameters, chain-of-thought only. Every step on this page exists to explain where those points came from: roughly 36 of them from data, 11 from instruction tuning, and the last 5 from RL.
Step 2 · The journey map
One model, four stations: pretrain on found math, tune, then reinforce.
51.7 came from “data, tuning, RL” — but in what order, starting from what? Here is the whole training journey as one map. We'll ride it left to right for the rest of the page.
The team did not train from scratch. They started from DeepSeek-Coder-Base-v1.5 7B — a code model — and kept pretraining it on 500B tokens dominated by mined math. Why a code model? That's a finding we'll unpack in Step 6: starting from code beats starting from a general language model, on math.
Read the map's three dark boxes left to right: continued pretraining lifts a code model to Base (MATH 36.2 — already better than every open base model, and better than Minerva 540B); supervised fine-tuning — training on 776K worked solutions so the model answers in clean step-by-step style — lifts Base to Instruct (46.8); reinforcement learning lifts Instruct to RL (51.7).
Notice what feeds the first arrow from below: the teal box, the 120B-token math corpus. It is the single biggest contribution of the paper, and it's where the real cleverness lives — because in January 2024 that corpus did not exist. No one knew how to get 120B tokens of math out of the open web. That problem is Part B.
Simplification flag: the map hides evaluation forks (tool-use mode, theorem proving) and the reward-model training inside the RL box. We'll restore the details that matter as we go; the paper's §2–§4 has all of them.
The data engine
How to pull 120B tokens of math out of a crawl that doesn't know what math is.
Step 3 · The chicken-and-egg problem
To find math you need a math detector — trained on math you haven't found yet.
The teal corpus box on the journey map is the destination. The raw material is Common Crawl — a public, regularly updated scrape of the web. After removing duplicate URLs and near-duplicate pages, the team had 40 billion HTML pages to sift. Somewhere in there sits our MathOverflow question, along with an unknown amount of other math. You can't read 40B pages, and you can't keyword-search for “math” — most math pages never say the word, and plenty of non-math pages do.
So you train a classifier. The tool is fastText: a deliberately simple text classifier that looks at word-fragment patterns (n-grams) and outputs a score — no GPU, fast enough to grade billions of pages. To train it you need examples. For positives, the team sampled 500,000 pages from OpenWebMath — an existing, carefully curated 13.6B-token collection of math web text — and for negatives, 500,000 random Common Crawl pages.
Then: score all 40B pages, sort by score, and keep the top of the ranking. How deep to cut? They tested it empirically — pretrain a small model on the top 40B, 80B, 120B and 160B tokens and see which corpus teaches best. Round one kept the top 40B tokens. Follow our MathOverflow page in the figure below — it doesn't make the cut:
Why does the classifier miss it? Because fastText can only recognize what resembles its 500K positives, and OpenWebMath has a particular flavor — lots of LaTeX-dense lecture notes and blog posts. A MathOverflow thread is conversational: short paragraphs, names, votes, “thanks, that helped”. Different surface, same underlying math. To the classifier it just looks unlike the training set, so it scores low.
Here's the part that stopped us when we first read §2.1, and it's worth sitting with: you cannot search for what your classifier missed. The misses are, by definition, pages you don't have. No error message, no list of false negatives — just silence. Any fix has to conjure evidence of the invisible out of what you did catch. That fix is the next step, and it's the cleverest move in the paper.
fastText settings, for reproducers: vector dimension 256, learning rate 0.1, word n-grams up to 3, minimum word occurrence 3, 3 epochs. The 40/80/120/160B cut experiment is in §2.1.
Step 4 · The snowball — core mechanism
Misses are invisible page by page — but at the domain level they cast a shadow.
Round 1 caught 40B tokens of OpenWebMath-lookalikes and silently missed everything else — including our MathOverflow page. The problem: find the misses without being able to see them.
The trick is to stop looking at pages and start looking at domains — all pages sharing a base URL. Group the entire crawl by domain and ask, for each one: what fraction of this domain's pages did round 1 collect? A general news site will sit near zero. But a site that is actually about math will have had many of its pages caught — even if the classifier missed the rest. The paper's rule: if over 10% of a domain's pages were collected, flag the domain as math-related.
That flag is exactly the evidence we said we couldn't get.
mathoverflow.net gets flagged — enough of its pages resembled
OpenWebMath to trip the 10% rule. Then humans step in, briefly:
annotators look at each flagged domain and mark which URL
paths hold the math (for MathOverflow, that's
mathoverflow.net/questions). Every page under those
paths that round 1 failed to collect — our page among
them — is added to the seed as a new positive. Retrain fastText on
the enriched seed, and round 2's classifier now knows what
forum-flavored math looks like.
The only human labor in the whole pipeline is station ④ — marking URL paths on flagged domains. So the 10% threshold is doing real work: set it too high and genuine math sites never get flagged (the snowball starves); set it too low and annotators drown in general-purpose sites that were never worth marking. Try both failure modes yourself:
Drag the threshold. Watch which domains get flagged for human annotation.
τ = 10% — the paper's rule. Three genuine math sites flagged, zero wasted inspections. mathoverflow.net makes it by 4 points.
Share-caught percentages and page counts here are illustrative — the paper reports the 10% rule and the mathoverflow.net example (§2.1), not per-domain statistics. Teal bars: flagged domains that really hold math. Coral: flagged dead ends that waste annotator time. Gray: below threshold, left unflagged.
Interactive demo needs JavaScript — showing the default τ = 10% state.
Individually invisible misses become visible in aggregate: a domain where 14% of pages scored high is a math site whose other 86% you're missing. The loop converts round N's partial success into round N+1's training data.
Also happening quietly: the paper notes this recipe isn't math-specific — the same seed → classify → fingerprint → annotate loop works for any domain, e.g. code. Keep that in mind for the legacy section.
Step 5 · Four rounds and a stop rule
The snowball rolls until a round stops finding anything new — four rounds, 120B tokens.
One turn of the loop: catch pages, fingerprint domains, annotate paths, enrich the seed, retrain. Now let the loop actually run and count what each round brings home.
Walk the four rounds in the chart below. Round 1 keeps 40B tokens; round 2, with its forum-savvy retrained classifier, nearly doubles the corpus to 89B. Then the curve flattens — and that flattening is the stop signal. In round 4, the team checked overlap and found about 98% of what round 4 collected was already in round 3's corpus. When a fresh detector mostly re-finds old pages, the web has been mined out at this quality level. They stopped: 35.5 million pages, 120B tokens.
One more gate before any of this touches a model. If benchmark problems leak into training data, every score in Step 1 is meaningless — the model would be reciting, not reasoning. So the corpus is decontaminated: any text segment sharing a 10-gram (ten consecutive words) exactly with GSM8K, MATH, CMATH or AGIEval is removed; benchmark texts shorter than ten words but at least three long are matched exactly. Our MathOverflow page sails through — research questions about étale cohomology don't collide with grade-school word problems — and its tokens are finally, officially, in the corpus.
What one round costs
Score 40B pages with fastText (cheap, CPU-scale), rank, cut; group by domain and count catch rates (bookkeeping); humans annotate URL paths on flagged domains — the only manual step; retrain fastText (minutes-to-hours, not GPU-days).
What one round buys
Round 2 added ~49B tokens the first classifier could not see. That is more than three entire OpenWebMaths (13.6B), gained by widening what “math page” means.
Scale check, from the paper's own comparisons: the final corpus is ~7× the math web pages Minerva trained on and ~9× OpenWebMath. And it's multilingual — mostly English and Chinese — which pays off in the next step.
Step 6 · The taste test
Same model, same budget, five corpora — the mined corpus wins every benchmark.
We have 120B tokens that a classifier says are math. A skeptic should ask: is any of it actually good? Bigger isn't better if it's noise.
The paper answers with a controlled experiment. Take one small model (DeepSeek-LLM 1.3B), train it separately on each candidate corpus for the same 150B-token budget, and compare benchmark scores. Same model, same compute, only the data changes — so the data takes all the credit or all the blame.
The cinnabar bar wins every group, but the third group is the loud one: on CMATH — Chinese elementary math — the mined corpus scores 41.5 while MathPile manages 1.2. That tiny second bar is worth staring at. MathPile is over 85% arXiv papers, English-only by construction; the snowball, which never specified a language, scooped up Chinese math sites simply because their domains got fingerprinted like any other. Multilingual coverage wasn't a feature someone designed — the pipeline found it.
Two more pretraining lessons round out Part B, both from ablations at small scale. First: starting from code helps math. Train 1.3B on code first, then math, and GSM8K hits 21.9 versus 19.1 when starting from general text — this is why the 7B run began from DeepSeek-Coder-Base-v1.5, and it's the paper's partial answer to the folk question “does code training improve reasoning?”
Second, a genuinely useful negative result: arXiv papers, the default math ingredient everywhere else, didn't help. Models trained on arXiv-only corpora (MathPile; the 28B-token ArXiv-RedPajama) showed no improvement across every math benchmark tried — and on the miniF2F theorem-proving benchmark they got worse than no math training at all. Formal papers apparently teach a model how mathematicians write, not how problems get solved step by step.
Every data decision in this paper — how deep to cut the ranking, when to stop iterating, whether arXiv earns its place — was settled by training a small probe model and reading the score. The corpus is engineered, end to end, against evidence.
Honest edges the authors flag themselves (§5.1.2): the arXiv verdict comes from arXiv-only corpora at 1.3B–7B scale; mixing arXiv with other data, or going to larger models, might still help. Their conclusion is “seems ineffective,” not “useless.”
To 51.7%
Riding the corpus through pretraining, tuning, and a leaner kind of RL.
Step 7 · Base and Instruct
Fed the mined corpus, a 7B base model out-scores one 77 times its size.
Part B ended with a verified corpus: 120B tokens, quality-checked by probe models, decontaminated. Time to feed the real model — here's the journey map again, now with the ingredient list on it.
Two readings of the map. First, the diet bar up top: even in a math model, only 56% is math — 20% is GitHub code (Step 6 said code helps), 10% general web text to keep ordinary language skills, and — yes — 10% arXiv, kept in the mix despite the ablation, since the negative result was about arXiv alone. Second, the reference line under the Base box: Minerva 540B, Google's math-specialized giant, scores 33.6 on MATH. The 7B model fed on found data scores 36.2. Parameter count lost to data quality, in public, for the first time at this scale.
The pretraining also paid outside math: versus its code-model starting point, Base jumped from 42.9 to 54.9 on MMLU and 42.9 to 59.5 on BBH — general knowledge and reasoning benchmarks — while keeping its coding scores. Math-heavy text, it turns out, is good brain food generally.
Then supervised fine-tuning, quick and standard: 776K worked problems (English and Chinese, from grade school to competition level), showing solutions three ways — plain chain-of-thought, program-of-thought, and tool-integrated reasoning. Result: Instruct, MATH 46.8 — past every open model of any size, and past Inflection-2 and Gemini Pro. Still four points shy of GPT-4. The last stretch needs a different kind of training.
Base training: 500B tokens, max learning rate 4.2e-4, batch 10M tokens (§2.3). SFT: 500 steps, batch 256, lr 5e-5 (§3.2). Tool-use mode is its own scoreboard — Instruct with Python reaches 57.4 on MATH — but this page follows the no-tools track.
Step 8 · The second invention — GRPO
Fire PPO's value model; let the group grade itself.
Instruct sits at 46.8, four points from GPT-4, with the easy gains spent. Reinforcement learning can push further — but in 2024, RL meant PPO, and PPO is a memory monster.
Count what PPO — the standard RL recipe for LLMs — keeps in GPU memory at once. The policy (the model being trained). A frozen reference model (to keep the policy from drifting too far). A reward model (to score answers). And a value model — a fourth network, typically the same size as the policy, whose whole job is to estimate “how well is this answer going so far?” so each token's reward can be compared against a baseline. That's two 7B models to train, two more to run, before a single math problem gets solved.
GRPO — Group Relative Policy Optimization, introduced in §4 of this paper — starts from an observation: for a math question you're already sampling many answers anyway. So drop the value model entirely, and use the group itself as the baseline. For each question, sample a group of G answers (G = 64 here), score each with the reward model, then grade every answer relative to its groupmates:
one advantage per answer — every token in answer i inherits Âi
The formula above is a z-score — “how many standard deviations better than the group average is this answer?” Positive  pushes the policy toward an answer's tokens; negative pushes away. Feel how the group does the value model's old job — grade answer C below, then flip its neighbors and watch C's grade change without C changing at all:
Tap an answer card to flip its reward between 1 (right) and 0 (wrong).
2 right, 2 wrong: a clean ±1.00 split — winners pulled up exactly as hard as losers are pushed down.
A toy group of G = 4 with 0/1 rewards (the real run used G = 64 and a learned reward model with graded scores). Population std.
Interactive demo needs JavaScript — the static grades above show the 2-right/2-wrong case.
Set all four answers right and something instructive happens: the std hits zero and the advantages vanish. A group that agrees teaches nothing — GRPO's learning signal comes entirely from disagreement within the group. That's also why it fits math so well: a checkable subject gives you honest disagreement cheaply.
We're deliberately not re-deriving the full update here — the clipped-ratio objective, the KL estimator, how  multiplies ∇log π to move actual weights. We taught that from scratch, with worked numbers, on our DeepSeek-R1 page — R1 is where GRPO became famous, but this paper is where it was born. What the R1 page doesn't show is this paper's receipts:
Note the third chip. The RL stage trained only on English GSM8K and MATH-style questions — about 144K of them — yet Chinese benchmarks improved too. Whatever RL strengthened, it wasn't just the questions it saw. Hold that puzzle; Step 9 explains it.
Config (§4.2): 64 samples per question, KL coefficient 0.04, policy lr 1e-6, reward model trained from Base. Folded away here: process (per-step) supervision and iterative reward-model retraining, both in §4.1 — and both taken further by later DeepSeek work.
Step 9 · One lens for all post-training
SFT, RFT, DPO, PPO, GRPO — same update, different coefficient.
GRPO left us a puzzle: RL on 144K English questions improved Chinese benchmarks it never trained on. Before answering, the paper zooms all the way out — and finds that every post-training method is secretly the same move.
Section §5.2's equation looks like notation soup on first read — it took us a couple of passes — but the idea underneath fits in one sentence. Every method nudges the model toward some answers and away from others; all that differs is where the answers come from and how hard each nudge is. Formally, every gradient has the shape:
GC — the “gradient coefficient” — is how hard this answer's probability gets pushed, and in which direction
Now line up the methods in this one frame — read the GC column top to bottom and watch it get smarter:
| method | answers come from | graded by | gradient coefficient GC |
|---|---|---|---|
| SFT | human-curated dataset | — | always 1 — imitate everything equally |
| RFT | SFT model, sampled once (offline) | rule: right answer? | 1 if correct, 0 if not |
| Online RFT | current policy, live | rule | 1 if correct, 0 if not |
| DPO | SFT model, in pairs | preference | sigmoid weight on each pair |
| PPO | current policy, live | reward model | At — needs the value model |
| GRPO | current policy, in groups | reward model | Âi + KL term — graded, per group, critic-free |
Two experimental findings fall out of the table (Figure 5 of the paper). Live beats canned: Online RFT overtakes plain RFT late in training — sampling from the current policy keeps the training signal pointed at the model's current mistakes. And graded beats binary: GRPO beats Online RFT, whose GC can only say 1-or-0; GRPO pushes a slightly-wrong answer down gently and a disastrous one down hard.
And now the puzzle. The team measured two quantities as K answers are sampled per problem: Maj@K (take a majority vote among K answers) and Pass@K (is any of the K correct?). Look at what RL did to each in the figure — the solid pair separates, the dashed pair doesn't:
Read the dashed curves first: at K = 64, some sampled answer is correct about 87% of the time — for both models equally. RL did not teach the model to solve new problems. What it moved is the solid curves: after RL, the correct answer is what the model says most often. In the paper's words, RL boosts the correct response from the model's top-K to its top-1 — it makes the output distribution more robust rather than expanding capability. That also dissolves the Chinese-benchmark puzzle: “put your probability mass on the answer you can already find” transfers across languages, because it never was about English.
GRPO's five points on MATH are a reranking dividend. The paper says so plainly and lists what might unlock more — better prompt coverage, tree-search sampling, noise-robust reward models (their probe found ~20% wrong labels in a well-known process-reward dataset). That to-do list reads, in hindsight, like an R1 roadmap.
Simplified here: the paper's Eq. 5 carries the full per-token notation, and the RFT/DPO/PPO coefficient derivations live in Appendix A.1. Process-vs-outcome supervision (grading each reasoning step, not just the final answer) also beat outcome-only in their runs — §4.1.3.
Step 10 · The report card
51.7 on MATH — and a candid list of what a 7B specialist still can't do.
Data engine (Part B), then pretrain → tune → GRPO (Part C). Time to put the final model back on Step 1's scoreboard and see what moved.
MATH benchmark · chain-of-thought accuracy · early 2024
The paper's own limitations section is unusually specific, and worth repeating. In their dry runs the model can't handle geometry problems about triangles and ellipses — the authors suspect the data pipeline under-collected geometry, a reminder that a learned finder inherits its seed's blind spots. Few-shot examples don't help it the way they help GPT-4 (a 7B ceiling, likely). And Step 9 already showed the third limit: RL reranks what's there; it didn't push Pass@K.
Before the legacy section, close the loop yourself. Each card asks for one gear of the machine; the answer is under the fold.
Round 1 caught only OpenWebMath-lookalikes. How did round 2 do better without anyone reading the misses?
Domain fingerprinting: group the crawl by domain and compute the share of each domain's pages round 1 caught. Over 10% → the domain is math-related; humans mark its math URL paths; pages under those paths that round 1 missed become new seed positives; retrain fastText.
What happens to the snowball if the 10% threshold is raised to 25%?
Almost no domains get flagged, so no new positives join the seed — round 2's classifier equals round 1's, and the corpus stops growing at 40B. Too low a threshold fails the other way: annotators drown in general-purpose sites.
Why does the 7B run start from a code model instead of a general LLM?
Ablation evidence: at 1.3B, code-then-math training beat both general-then-math (21.9 vs 19.1 on GSM8K) and math-only (20.5). Code pretraining also lifts tool-use math. Hence DeepSeek-Coder-Base-v1.5 as the starting point.
What did GRPO delete from PPO, and what does the job instead?
The value model — a trained network as large as the policy. Its baseline job goes to the group: sample G answers per question, and each answer's advantage is its reward's z-score within the group, (ri − mean)/std.
RL trained only on English GSM8K/MATH questions. Why did Chinese scores rise?
Because RL improved Maj@K, not Pass@K — it concentrates probability on answers the model can already produce rather than teaching new math. That reranking effect isn't tied to the training language, so it transfers.
Why bother deleting 10-gram matches against GSM8K and MATH from the corpus?
Because 120B mined tokens will otherwise contain benchmark problems verbatim, and every score in this lesson would measure recitation instead of reasoning. Decontamination is what makes 51.7 a claim about ability.
Where to go next — read this paper's §4 side by side with our DeepSeek-R1 explainer: R1 is GRPO one year later, scaled up, with the reward model swapped for rule-based checks. Watching the same algorithm graduate is the best way to check you understood it here.
“The mathematical reasoning capability of DeepSeekMath is attributed to two key factors: First, we harness the significant potential of publicly available web data through a meticulously engineered data selection pipeline. Second, we introduce Group Relative Policy Optimization (GRPO)…” — Shao et al., 2024, abstract
A math-data paper wrote the field's default RL algorithm.
DeepSeekMath was read in 2024 as a strong domain model. Its two inventions aged very differently — one became a playbook, the other became infrastructure.
GRPO grows up in R1
Born here as a memory optimization, GRPO became the training engine of DeepSeek-R1 — swap the reward model for rule-based answer checks, scale the groups, and long chains of reasoning emerge. RL-for-reasoning work across the field now treats critic-free, group-relative updates as a default.
Seed, classify, fingerprint, repeat
The snowball — seed corpus → cheap classifier → domain discovery → human-marked URL paths → retrain — became a reference recipe for mining domain data at web scale, and the paper itself points out it's not math-specific: the same loop works for code or any specialty.
The 50% wall fell in the open
Open-source MATH went from 37.7 to 51.7 in one release — the first open model past 50 — with the corpus recipe and the RL algorithm published for anyone to reuse. “Small model, better data” stopped being a slogan and became a result: 7B over Minerva 540B.
Ask what RL algorithm today's reasoning models run, and the usual answer traces back to §4 of a paper about finding math on the web.