Longer answers can reflect the loss function rather than better reasoning. The useful lens here is to separate what pretraining already provides from what reinforcement learning actually changes, before attributing self-reflection or benchmark gains to RL.
Key Findings
- DeepSeek-V3-Base already produces self-reflection; observing it after training alone does not establish its origin.
- GRPO normalization weights responses by length and questions by reward dispersion. Dr. GRPO removes both factors, reducing wasteful growth in incorrect answers.
- The authors report 43.3% on AIME 2024 for their 7B recipe; their release specifies 27 hours on eight A100s.
Who Should Read It
Great fit if you compare RL objectives or diagnose growing rollout costs. Look elsewhere if you need evidence across general agent tasks: these experiments focus on mathematical reasoning, and the hypothesized Qwen pretraining explanation is not a confirmed account of its data.