AIAny
Icon for item

ReAct: Synergizing Reasoning and Acting in Language Models

Interleaves chain-of-thought reasoning with tool-using actions in one LLM loop: the model plans, queries a source like Wikipedia, then revises from results. Cuts hallucination versus reasoning-only prompting and beats trained agents on interactive tasks.

Introduction

Chain-of-thought lets a model reason but leaves it trapped in its own head, free to hallucinate facts it never checked. ReAct's move is almost embarrassingly simple: let the model emit a "thought," then an "action," then read the resulting "observation," and repeat. Reasoning decides what to look up; the lookup grounds the next thought. That single feedback loop is the seed under most of today's agent frameworks.

Key Findings
  • On HotpotQA and Fever, interleaving Wikipedia search actions with reasoning sharply reduces the fact-fabrication that plagues pure chain-of-thought, because the model is checking claims instead of inventing them.
  • On ALFWorld and WebShop interactive tasks, ReAct prompting with one or two in-context examples outperforms imitation-learning and reinforcement-learning agents trained on thousands of episodes, by absolute success-rate margins in the tens of points.
  • The reasoning trace is human-readable, so you can watch where the agent went wrong and intervene, rather than debugging an opaque policy.
Who Should Read It

Great fit if you build LLM agents or tool-using systems and want the conceptual blueprint that ReAct-style scaffolding, LangChain, and function-calling loops all descend from. Look elsewhere if you need a turnkey library or production code; this is a 2022 research paper, and the prompting tricks have since been refined and partly absorbed into native tool-calling APIs. Read it for the idea, not the implementation.

Information

  • Websitearxiv.org
  • OrganizationsGoogle Research, Princeton University
  • AuthorsShunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao
  • Published date2022/10/06

More Items

Turns each research paper into a training environment to generate verifiable research plans by synthesizing questions from goals/background and deriving evaluation criteria from methods/experiments. Key features: four-stage extraction that reduces criterion leakage to 3.7%, a two-stage rubric-centered training (self-distillation then GRPO), and the PaperGym-20k corpus with two held-out benchmarks.

Analyzes on-policy distillation for LLM fine-tuning, shows teacher token-level supervision is often noisy and not the main driver of gains, and introduces OPSA, a supervision-free, entropy-adaptive method that suppresses low-probability tokens to improve downstream accuracy.

Trains LLM agents to proactively edit and manage their working context for long-horizon tasks using an expanded toolset (planning, long-term memory, soft offloading) and a fine-grained RL algorithm that identifies critical edits and assigns action-level credit. Improves accuracy while keeping contexts compact on long-context QA and deep search.