A model that can retrieve the right passage is not necessarily able to learn from it. This work tests a harder capability: whether reinforcement learning can teach an LLM to navigate a textbook, acquire useful concepts, and apply them to an unseen proof—rather than relying on a preselected context or a fixed retrieval strategy.
Key Findings
- On textbook-based proof problems, the self-learning policy improves overall accuracy from 51.5% to 79.5%, a 28-point gain. It also beats the strongest fixed-retrieval baseline, which reaches 73.1% after RL.
- Gains transfer to held-out books and subjects: accuracy reaches 85.2% on out-of-domain books and 86.7% on out-of-domain subjects. This supports the possibility that the learned behavior is not limited to memorizing a particular textbook.
- The policy also improves over the base model on BrowseComp-Plus and CL-bench by 1.7 and 2.3 percentage points, respectively. However, the gains are selective: all tested RL policies perform worse than the base model on CL-bench's Procedural Task Execution category.
How the Training Setup Works
The authors build a textbook-based mathematical problem-solving task from 158 undergraduate- and graduate-level books, with 5,183 problems and solutions withheld. A BookNavigator interface lets the model search and study long-form material as needed. Fine-grained RL rewards encourage the sequence of acquiring knowledge and using it in a proof, not just producing an answer from an already assembled context.
Who Should Read It
A good fit for researchers studying tool-using LLMs, long-context learning, and RL objectives that reward information acquisition as part of reasoning. The results are promising for knowledge-intensive tasks, but the evaluation is centered on mathematical proofs and gains do not generalize uniformly: systems needing reliable execution of prescribed procedures may require different training signals.