{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reflexion-language-agents-with-verbal","title":"Reflexion: Language Agents with Verbal Reinforcement Learning","arxiv_id":"2303.11366","date":"2023-03-20","proceeding":"NeurIPS 2023 11","authors":["Noah Shinn","Federico Cassano","Edward Berman","Ashwin Gopinath","Karthik Narasimhan","Shunyu Yao"],"abstract":"Large language models (LLMs) have been increasingly used to interact with external environments (e.g., games, compilers, APIs) as goal-driven agents. However, it remains challenging for these language agents to quickly and efficiently learn from trial-and-error as traditional reinforcement learning methods require extensive training samples and expensive model fine-tuning. We propose Reflexion, a novel framework to reinforce language agents not by updating weights, but instead through linguistic feedback. Concretely, Reflexion agents verbally reflect on task feedback signals, then maintain their own reflective text in an episodic memory buffer to induce better decision-making in subsequent trials. Reflexion is flexible enough to incorporate various types (scalar values or free-form language) and sources (external or internally simulated) of feedback signals, and obtains significant improvements over a baseline agent across diverse tasks (sequential decision-making, coding, language reasoning). For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%. We also conduct ablation and analysis studies using different feedback signals, feedback incorporation methods, and agent types, and provide insights into how they affect performance.","url_abs":"https://arxiv.org/abs/2303.11366v4","url_pdf":"https://arxiv.org/pdf/2303.11366v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reflexion-language-agents-with-verbal","repo_url":"https://github.com/noahshinn024/reflexion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"paper_slug":"reflexion-language-agents-with-verbal","repo_url":"https://github.com/gammatauai/leetcode-hard-gym","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"reflexion-language-agents-with-verbal","repo_url":"https://github.com/imagination-research/lbt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"reflexion-language-agents-with-verbal","repo_url":"https://github.com/noahshinn/reflexion","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"reflexion-language-agents-with-verbal","repo_url":"https://github.com/yenchehsiao/autonomousllmagentwithadaptingplanning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"humaneval","task_name":"HumanEval"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"sequential-decision-making","task_name":"Sequential Decision Making"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[{"slug":"leetcode-hard","name":"LeetCode-Hard","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.11366","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.11366"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/imagination-research/lbt","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yenchehsiao/autonomousllmagentwithadaptingplanning","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gammatauai/leetcode-hard-gym","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/noahshinn/reflexion","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/noahshinn024/reflexion","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"summary":{"ran_draft_wrong":2,"unverified":7},"by_repo_kind":{"listed":{"samples":9,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"84d28fbaf38b0a59","entry":"format_step","repo":"noahshinn/reflexion","repo_kind":"listed","path":"hotpotqa_runs/agents.py","file_url":"https://github.com/noahshinn/reflexion/blob/HEAD/hotpotqa_runs/agents.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"84d28fbaf38b0a59"}},{"code_sha256_prefix":"dae7ab386661a4f4","entry":"normalize_answer","repo":"noahshinn/reflexion","repo_kind":"listed","path":"hotpotqa_runs/environment.py","file_url":"https://github.com/noahshinn/reflexion/blob/HEAD/hotpotqa_runs/environment.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dae7ab386661a4f4"}},{"code_sha256_prefix":"6f79fe1ce6d553fa","entry":"EM","repo":"noahshinn/reflexion","repo_kind":"listed","path":"hotpotqa_runs/environment.py","file_url":"https://github.com/noahshinn/reflexion/blob/HEAD/hotpotqa_runs/environment.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f79fe1ce6d553fa"}},{"code_sha256_prefix":"a7a0cc9d35a647f1","entry":"get_completion","repo":"noahshinn/reflexion","repo_kind":"listed","path":"alfworld_runs/utils.py","file_url":"https://github.com/noahshinn/reflexion/blob/HEAD/alfworld_runs/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a7a0cc9d35a647f1"}},{"code_sha256_prefix":"0cd0094dec049291","entry":"parse_action","repo":"noahshinn/reflexion","repo_kind":"listed","path":"hotpotqa_runs/agents.py","file_url":"https://github.com/noahshinn/reflexion/blob/HEAD/hotpotqa_runs/agents.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0cd0094dec049291"}},{"code_sha256_prefix":"23aab7577a21a4cf","entry":"parse_action","repo":"noahshinn/reflexion","repo_kind":"listed","path":"hotpotqa_runs/environment.py","file_url":"https://github.com/noahshinn/reflexion/blob/HEAD/hotpotqa_runs/environment.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"23aab7577a21a4cf"}},{"code_sha256_prefix":"dfb17eb3e6eaed35","entry":"process_ob","repo":"noahshinn/reflexion","repo_kind":"listed","path":"alfworld_runs/alfworld_trial.py","file_url":"https://github.com/noahshinn/reflexion/blob/HEAD/alfworld_runs/alfworld_trial.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dfb17eb3e6eaed35"}},{"code_sha256_prefix":"d228da106025a025","entry":"reactLLMMock","repo":"noahshinn/reflexion","repo_kind":"listed","path":"hotpotqa_runs/mocks.py","file_url":"https://github.com/noahshinn/reflexion/blob/HEAD/hotpotqa_runs/mocks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d228da106025a025"}},{"code_sha256_prefix":"48168012e82e413b","entry":"reflectLLMMock","repo":"noahshinn/reflexion","repo_kind":"listed","path":"hotpotqa_runs/mocks.py","file_url":"https://github.com/noahshinn/reflexion/blob/HEAD/hotpotqa_runs/mocks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48168012e82e413b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}