AIAny

ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory

Distills reusable reasoning strategies from successful and failed agent runs. Retrieved lessons guide later tasks, while memory-aware test-time scaling turns extra exploration into better memories.

Introduction

An agent can retain every interaction and still repeat its mistakes. The useful unit of memory here is an actionable lesson, including what went wrong, rather than an entire transcript. That makes the paper relevant to systems that repeatedly encounter related tasks.

Key Findings
  • On WebArena with Gemini 2.5 Flash, success rises from 40.5% without memory to 48.8%, while mean interaction steps fall from 9.7 to 8.3. This measures reuse across a task stream, rather than model weight updates.
  • Lessons come from both successes and failures. Self-judged outcomes supply strategies and warnings without requiring ground-truth feedback during learning.
  • MaTTS compares multiple attempts or refines one attempt before extracting memories. Extra computation can therefore improve later tasks as well as the current answer. The authors release implementations for web browsing and software engineering.
Who Should Read It

Great fit if you are designing agents that should improve across recurring tasks, especially browser or coding workflows. Look elsewhere if you need a production-ready memory service: the released code is a research demonstration, and judge errors, retrieval quality, and added inference work still need evaluation in your environment.

Information

  • Websitearxiv.org
  • OrganizationsGoogle Cloud AI Research, Google Cloud AI, Yale University, University of Illinois Urbana-Champaign
  • AuthorsSiru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang …
  • Published date2025/09/29

Categories

More Items

Embeds an agent's reasoning together with its search query to improve retrieval. Trains a 4B embedding model on synthesized deep-research trajectories, helping preserve intent across multi-step searches.

Trains an 8B model to search, synthesize evidence, and write cited research reports. Its reinforcement-learning rewards use evolving rubrics grounded in retrieved evidence and contrasting model responses.

Trains long-horizon web researchers by combining agentic mid-training, synthetic trajectories, and reinforcement learning. A 30.5B-parameter model activates 3.3B parameters per token and learns to plan, search, and synthesize evidence.