{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/strategyllm-large-language-models-as-strategy","title":"StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving","arxiv_id":"2311.08803","date":"2023-11-15","proceeding":null,"authors":["Chang Gao","Haiyun Jiang","Deng Cai","Shuming Shi","Wai Lam"],"abstract":"Most existing prompting methods suffer from the issues of generalizability and consistency, as they often rely on instance-specific solutions that may not be applicable to other instances and lack task-level consistency across the selected few-shot examples. To address these limitations, we propose a comprehensive framework, StrategyLLM, allowing LLMs to perform inductive reasoning, deriving general strategies from specific task instances, and deductive reasoning, applying these general strategies to particular task examples, for constructing generalizable and consistent few-shot prompts. It employs four LLM-based agents: strategy generator, executor, optimizer, and evaluator, working together to generate, evaluate, and select promising strategies for a given task. Experimental results demonstrate that StrategyLLM outperforms the competitive baseline CoT-SC that requires human-annotated solutions on 13 datasets across 4 challenging tasks without human involvement, including math reasoning (34.2\\% $\\rightarrow$ 38.8\\%), commonsense reasoning (70.3\\% $\\rightarrow$ 72.5\\%), algorithmic reasoning (73.7\\% $\\rightarrow$ 85.0\\%), and symbolic reasoning (30.0\\% $\\rightarrow$ 79.2\\%). Further analysis reveals that StrategyLLM is applicable to various LLMs and demonstrates advantages across numerous scenarios.","url_abs":"https://arxiv.org/abs/2311.08803v4","url_pdf":"https://arxiv.org/pdf/2311.08803v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"strategyllm-large-language-models-as-strategy","repo_url":"https://github.com/gao-xiao-bai/strategyllm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"math","task_name":"Math"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.08803","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.08803"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/gao-xiao-bai/StrategyLLM","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gao-xiao-bai/strategyllm","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e122b45f16bbdc68","entry":"is_correct","repo":"gao-xiao-bai/StrategyLLM","repo_kind":"official","path":"source/dataset/utils.py","file_url":"https://github.com/gao-xiao-bai/StrategyLLM/blob/HEAD/source/dataset/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e122b45f16bbdc68"}},{"code_sha256_prefix":"8a20371953a7f5ec","entry":"post_process","repo":"gao-xiao-bai/StrategyLLM","repo_kind":"official","path":"source/dataset/utils.py","file_url":"https://github.com/gao-xiao-bai/StrategyLLM/blob/HEAD/source/dataset/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a20371953a7f5ec"}},{"code_sha256_prefix":"cf46a2fecbc874d2","entry":"timeout","repo":"gao-xiao-bai/StrategyLLM","repo_kind":"official","path":"source/model/base.py","file_url":"https://github.com/gao-xiao-bai/StrategyLLM/blob/HEAD/source/model/base.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf46a2fecbc874d2"}},{"code_sha256_prefix":"b1b3d6edbf300b96","entry":"evaluate_acc","repo":"gao-xiao-bai/StrategyLLM","repo_kind":"official","path":"source/dataset/utils.py","file_url":"https://github.com/gao-xiao-bai/StrategyLLM/blob/HEAD/source/dataset/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1b3d6edbf300b96"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}