AIAny

DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research

Trains an 8B model to search, synthesize evidence, and write cited research reports. Its reinforcement-learning rewards use evolving rubrics grounded in retrieved evidence and contrasting model responses.

Introduction

A research report can be wrong in many ways that an exact-answer checker cannot see. The useful idea here is to improve the evaluator alongside the agent: new search evidence and competing answers expose factual gaps that a fixed rubric may miss.

Key Findings
  • Search-grounded, evolving criteria give long-form training a reward signal beyond short-answer correctness; the paper tests this across four research benchmarks.
  • Positive criteria capture newly useful evidence, while negative criteria penalize recurring failures. Keeping discriminative criteria focuses feedback as the policy improves.
  • The released training and evaluation stack makes this approach inspectable, rather than leaving only a hosted demo. The official repository includes agent infrastructure, supervised fine-tuning, and reinforcement-learning code.
Who Should Read It

Great fit if you study rewards for research agents or need an open training stack to adapt. Look elsewhere if you need a minimal retrieval pipeline: this approach adds rubric generation, judge-model scoring, and tool orchestration. Treat benchmark results as evidence under the paper's evaluation setup; research citations still merit checking.

Information

  • Websitearxiv.org
  • OrganizationsUniversity of Washington, Allen Institute for AI, Carnegie Mellon University, Massachusetts Institute of Technology, Seattle Children's Hospital, University of California, Berkeley
  • AuthorsRulin Shao, Akari Asai, Shannon Zejiang Shen, Hamish Ivison, Varsha Kishore, Jingming Zhuo, Xinran Zhao, Molly Park, Samuel G. Finlayson, David Sontag …
  • Published date2025/11/24

More Items

Embeds an agent's reasoning together with its search query to improve retrieval. Trains a 4B embedding model on synthesized deep-research trajectories, helping preserve intent across multi-step searches.

Trains long-horizon web researchers by combining agentic mid-training, synthetic trajectories, and reinforcement learning. A 30.5B-parameter model activates 3.3B parameters per token and learns to plan, search, and synthesize evidence.

Tests research agents on 100 expert-written tasks across 22 fields. Separates report quality from citation reliability, using adaptive reference-based grading and checks that cited pages support individual claims.