ReAct: Synergizing Reasoning and Acting

A plain prompt walked into our two-hop question and said “iPod”. The two obvious upgrades — let the model think first, or let it search — both existed by 2022, and on this same question both go down. Watch closely: they fail for opposite reasons.

Upgrade one is chain-of-thought (CoT): ask the model to reason in writing before answering. If you've read our GPT-2 page, this is prompting at its purest — no tools, just the model talking to itself. Here is its entire output on our question, from the paper's Figure 1, with the invented part underlined in dashes:

The paper's diagnosis of that last line (§2): choosing the final action requires reasoning over everything seen so far — question, three searches, three results — and an action-only format gives the model nowhere to do that reasoning. CoT had the reasoning and no facts; Act-only had the facts and no reasoning. Two half-agents, each missing exactly what the other has.

CoT invented facts because it couldn't look; Act-only looked but couldn't digest what came back. Before we show the fix, take the acting agent's seat at its most instructive moment — the search that fails.

You've searched Apple Remote and learned it was built for something called Front Row. You search Front Row. The environment replies:

If you picked the software page — notice what you did in your head first. Something like: “the exact title doesn't exist, but the suggestion list has a software entry, and Front Row is software.” You reasoned, silently, and then acted. The paper's entire proposal is to make that silent step a written step — because a language model can only reliably “think” in text it actually generates, and because text it generates can be read back, checked, and even corrected later.

To be fair to the act-only agent: on this particular fork it stumbled through by imitating patterns in its prompt. Where it actually died was the final answer, three observations later — the place where the most reasoning had piled up undone.

You just felt the missing ingredient: a written thought between observation and action. Here's what that looks like sustained for a whole task — the paper's Figure 1(1d), on our question, complete. Read it slowly; every later section refers back to these twelve lines.

Look at where the amber lines sit and what each one does. T1 plans before anything is searched. T2 pulls the one fact that matters out of O1 and turns it into the next goal. T3 is the recovery you authored in step 3. T4 does the thing Act-only never could: it reads O3 and assembles the final answer in words before committing to it. Four thoughts, four different jobs — and none of them touched the environment.

Four thoughts, four actions, four observations — one solved question. Now the tidy part: every baseline that failed in step 2 is this same trajectory with something deleted, and seeing that makes the formal definition almost obvious.

This isn't just a nice picture — it's literally how the paper built its baselines (§3.2): take the hand-written ReAct trajectories and ablate parts out. So every comparison later on is controlled: same questions, same exemplars, pieces removed.

Now the definition. An agent normally picks actions from some set A — search, click, move. ReAct enlarges the set:

An action picked from L — a thought — has a property that's easy to read past, and honestly we skimmed it on our first read too, so slow down here: a thought gets no observation back. The environment never learns it happened. Its entire effect is cₜ₊₁ = (cₜ, âₜ) — the sentence is appended to the context, and the policy π(aₜ | cₜ) that picks every later move reads that context. The model is, quite literally, leaving notes for its next step.

So a thought is an action that only writes to context. Two practical questions are left: how often should the model think, and where does it learn the format? The answers are “it depends” and “from you, in about an afternoon”.

For knowledge questions like ours, thoughts and actions alternate every step — the paper calls this dense reasoning. For long games with many mundane moves, thinking after every step would be noise; there, thoughts appear sparsely, only at decision points, and the model itself decides when. Here's a sparse one from the paper's household game (ALFWorld), where the task is “put some pepper shaker on a drawer”:

The recipe is a handful of hand-written trajectories and a weak Wikipedia API. Time to grade it — and the scoreboard holds a surprise the abstract doesn't advertise.

Two benchmarks: our HotpotQA (exact-match on the answer), and Fever, where the model must call a claim SUPPORTED, REFUTED or unverifiable. One more baseline joins: CoT-SC, or self-consistency — run CoT 21 times at some randomness and take the majority answer, a standard way to squeeze more out of pure reasoning. All methods, one table:

Read the middle rows first and let the awkward fact land: on HotpotQA, ReAct (27.4) loses to plain CoT (29.4). The search engine didn't pay for itself. We'll dissect why in the next step. But the two arrow rows win on both benchmarks — those are simple hand-offs: run ReAct, and if it can't finish within 7 steps, fall back to CoT-SC (and the reverse rule for the other row). The paper's reading: the model's internal knowledge and the retrieved external knowledge are complementary, and the win comes from using both.

The hand-off pays in efficiency too. Drag the sampling budget below — the paper's Figure 2 — and watch how many CoT-SC samples it takes to match what the combination gets almost immediately:

ReAct alone losing to CoT alone, on a knowledge benchmark, with a search engine in hand — if that surprised you, it surprised us too; it's worth slowing down for. The authors did the slow thing: they read 200 trajectories by hand and labeled what actually happened.

Fifty successes and fifty failures from each method, sorted into causes. This one table explains the whole trade-off:

Read the two bold columns against each other. Among CoT's failures, over half are hallucination; among ReAct's, hallucination is zero — every fact in a ReAct trace came from an actual observation, which is also why even its successes are more trustworthy (6% vs 14% lucky guesses). That's the win.

The cost is the other bold cell. Being forced to alternate thought–action–observation is a straitjacket for reasoning: 47% of ReAct's failures are reasoning errors, including a signature one where the model repeats its previous thought and action forever, unable to jump out of the loop. And when search returns junk (23% of failures), the derailed reasoning struggles to recover. Grounding traded flexibility for factuality — which is exactly why step 7's hand-off strategies, using each method where it's strong, beat both.

On knowledge tasks, grounding bought trust. The paper's second half moves to environments that change state as you act — a text-based household game and a shopping site — where planning and tracking progress matter more than facts.

ALFWorld gives goals like our pepper-shaker task from step 6 — with over 50 locations and solutions over 50 steps long. The baseline BUTLER is an imitation-learning agent trained on 100,000 expert trajectories per task type. ReAct gets two hand-annotated examples per task type. Success rates across 134 unseen games:

One ablation here deserves a sentence, because it sharpens what “thought” means. The authors rebuilt the prompts in the style of Inner Monologue — a prior method whose “thoughts” only restate the current observation and goal. That version (ReAct-IM) scores 53%, eighteen points below ReAct. Restating what you see is not the same as deciding what it means and what to do about it.

Every thought so far was the model's own. But a thought is only a line of text in the context — and text can be edited by anyone, mid-run. The paper's appendix turns that into the most forward-looking demo in the whole thing.

An ALFWorld run goes wrong: holding the first of two keychains, the model writes a thought that hallucinates where the second one is, and the trajectory decays into “Nothing happens.” A human steps in and — without touching the model, the game, or any parameter — rewrites two thought lines:

The paper's framing is worth quoting in spirit: for a human, fixing this agent went from typing tens of actions to editing a couple of thoughts. You can't do that to a policy network's weights, and you can't do it to an agent that never writes its reasoning down. Interpretability here isn't a dashboard bolted on afterwards — the reasoning is the interface.

Two honest boundary notes to close the evidence. First, ReAct needs a big model: with PaLM-8B and 62B, ReAct is the worst of the four prompting methods — the format only clicks at 540B. (Fine-tuned on just 3,000 bootstrapped trajectories, the ranking flips: finetuned ReAct at 62B beats every prompting method at 540B.) Second, it's not PaLM-specific — on GPT-3 the same prompts score better still: 30.8 EM on HotpotQA, 78.4% on ALFWorld.

One question, one loop, a family of deletions, an honest failure breakdown, two games, and a mind edited mid-run. Before the epilogue, test whether the loop comes back to you on demand — each answer is two sentences, each names its step.