{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2509-25140","title":"ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory","arxiv_id":"2509.25140","date":"2025-09-29","proceeding":"ICLR","authors":["Siru Ouyang","Jun Yan","I-Hung Hsu","Yanfei Chen","Ke Jiang","Zifeng Wang","Rujun Han","Long T. Le","Samira Daruki","Xiangru Tang","Vishy Tirumalashetty","George Lee","Mahsan Rofouei","Hangfei Lin","Jiawei Han","Chen-Yu Lee","Tomas Pfister"],"abstract":"With the growing adoption of large language model agents in persistent real-world roles, they naturally encounter continuous streams of tasks. A key limitation, however, is their failure to learn from the accumulated interaction history, forcing them to discard valuable insights and repeat past errors. We propose ReasoningBank, a novel memory framework that distills generalizable reasoning strategies from an agent's self-judged successful and failed experiences. At test time, an agent retrieves relevant memories from ReasoningBank to inform its interaction and then integrates new learnings back, enabling it to become more capable over time. Building on this powerful experience learner, we further introduce memory-aware test-time scaling (MaTTS), which accelerates and diversifies this learning process by scaling up the agent's interaction experience. By allocating more compute to each task, the agent generates abundant, diverse experiences that provide rich contrastive signals for synthesizing higher-quality memory. The better memory in turn guides more effective scaling, establishing a powerful synergy between memory and test-time scaling. Across web browsing and software engineering benchmarks, ReasoningBank consistently outperforms existing memory mechanisms that store raw trajectories or only successful task routines, improving both effectiveness and efficiency; MaTTS further amplifies these gains. These findings establish memory-driven experience scaling as a new scaling dimension, enabling agents to self-evolve with emergent behaviors naturally arise. Our code can be found at https://github.com/google-research/reasoning-bank.","url_abs":"https://arxiv.org/abs/2509.25140","url_pdf":"https://arxiv.org/pdf/2509.25140","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2509.25140","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2509.25140"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/google-research/reasoning-bank","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":14},"by_repo_kind":{"found_in_text":{"samples":14,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"238e7dce697a54e4","entry":"build_naive_last_frame_eval_prompt","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/autoeval/prompts.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/autoeval/prompts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"238e7dce697a54e4"}},{"code_sha256_prefix":"7ad3fb700204a1d0","entry":"build_naive_multi_frame_eval_prompt","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/autoeval/prompts.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/autoeval/prompts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7ad3fb700204a1d0"}},{"code_sha256_prefix":"d30039bf61d4ce96","entry":"build_obs_simplifier_prompt","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/autoeval/prompts.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/autoeval/prompts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d30039bf61d4ce96"}},{"code_sha256_prefix":"e35f25d166dc1c3b","entry":"embed_query_with_qwen","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/memory_management.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/memory_management.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e35f25d166dc1c3b"}},{"code_sha256_prefix":"4ae947fadb35f89b","entry":"extract_final_response","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/reeval_memevol_prompt.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/reeval_memevol_prompt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4ae947fadb35f89b"}},{"code_sha256_prefix":"5503345bf4b138db","entry":"extract_response","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/autoeval/evaluate_trajectory.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/autoeval/evaluate_trajectory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5503345bf4b138db"}},{"code_sha256_prefix":"38314d8e2e565dd6","entry":"extract_think_and_action","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/induce_memory.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/induce_memory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"38314d8e2e565dd6"}},{"code_sha256_prefix":"c4e4bc0ba1f63988","entry":"extract_think_and_action","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/autoeval/evaluate_trajectory.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/autoeval/evaluate_trajectory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c4e4bc0ba1f63988"}},{"code_sha256_prefix":"c6db4f1167ef1f97","entry":"format_trajectory","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/induce_memory.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/induce_memory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c6db4f1167ef1f97"}},{"code_sha256_prefix":"cf9eabc2e955701a","entry":"l2_normalize","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/memory_management.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/memory_management.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cf9eabc2e955701a"}},{"code_sha256_prefix":"868712f71f85de88","entry":"load_blocks","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/induce_scaling.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/induce_scaling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"868712f71f85de88"}},{"code_sha256_prefix":"06ce227a2f35045b","entry":"load_step_pkl","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/induce_scaling.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/induce_scaling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"06ce227a2f35045b"}},{"code_sha256_prefix":"ee92b111809ec4b6","entry":"random_group_sample","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/induce_memory.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/induce_memory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ee92b111809ec4b6"}},{"code_sha256_prefix":"68200314270ba3be","entry":"remove_invalid_steps","repo":"google-research/reasoning-bank","repo_kind":"found_in_text","path":"WebArena/induce_scaling.py","file_url":"https://github.com/google-research/reasoning-bank/blob/HEAD/WebArena/induce_scaling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"68200314270ba3be"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.AI","source":"arxiv_api"},"syntology_extracted_results":null}