AIAny
Icon for item

PaperCritic: Training AI Reviewers to Assess the Soundness of Papers

Trains AI reviewers to identify substantive, evidence-based concerns about papers. Its reinforcement-learning objective balances the precision of critiques with their coverage of known high-quality concerns.

Introduction

A useful AI review is not simply a long list of possible objections: unsupported criticism can be as damaging as missed flaws. This work treats review quality as a balance between making sound, substantive points and covering the important concerns a paper raises—a framing that rewards careful criticism rather than volume.

Key Findings
  • The authors build PaperSound-23K from 1,000 ICLR submissions and 23,000 human-written and LLM-generated review points, labeled for soundness and substantiveness. This gives training a more specific target than imitating whole human reviews.
  • PaperCritic-4B is trained with reinforcement learning using review-level F1 as its reward. On the paper-review test set, it reaches 45.1 F1, compared with 23.7 for the best reported conventionally trained 4B baselines, and exceeds the listed frontier models.
  • The review-focused training transfers beyond ICLR submissions: results improve on Nature-family paper reviews and on code review, legal contract review, and story feedback. This suggests the learned skill may extend to evaluating claims and evidence across domains.
Who It’s For—and the Trade-Off

A good fit for researchers studying AI-assisted peer review or evaluating methods for producing evidence-grounded critiques. It is not a complete substitute for human review: it focuses on soundness, not novelty, clarity, interestingness, or accept/reject decisions, and its recall measures coverage of known reference concerns rather than every valid issue.

How the Objective Works

A generated review point counts toward precision only when its premises are supported and its concern is substantive. Recall measures how many distinct high-quality reference concerns are covered. Their harmonic mean rewards balance, so a model cannot compensate for unreliable criticism with sheer coverage—or for narrow coverage with a few excellent points.

Information

  • Websitearxiv.org
  • OrganizationsKAIST AI, Carnegie Mellon University Language Technologies Institute, LG AI Research
  • AuthorsHyun Ryu, Ji Yong Cho, Eunho Yang, Moontae Lee, June Yong Yang, Sean Welleck, Carolyn Rose
  • Published date2026/11/13

More Items

Generates coordinated portfolio weights directly from multi-asset market histories, then refines them with portfolio-level reinforcement learning. In cryptocurrency futures tests, refinement improved net annualized return while reducing maximum drawdown.

Turns screen observations into keyboard and mouse actions to complete long-horizon game tasks. Sparse reasoning, persistent memory, and online reinforcement learning support real-time interaction and multi-step decisions.

Tests decision models as actors, trajectory judges, and reinforcement-learning teachers for long-horizon agents. It finds that a model can be a weak actor yet still improve policy learning by evaluating actions after seeing their consequences.