{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/automated-design-of-agentic-systems","title":"Automated Design of Agentic Systems","arxiv_id":"2408.08435","date":"2024-08-15","proceeding":null,"authors":["Shengran Hu","Cong Lu","Jeff Clune"],"abstract":"Researchers are investing substantial effort in developing powerful general-purpose agents, wherein Foundation Models are used as modules within agentic systems (e.g. Chain-of-Thought, Self-Reflection, Toolformer). However, the history of machine learning teaches us that hand-designed solutions are eventually replaced by learned solutions. We formulate a new research area, Automated Design of Agentic Systems (ADAS), which aims to automatically create powerful agentic system designs, including inventing novel building blocks and/or combining them in new ways. We further demonstrate that there is an unexplored yet promising approach within ADAS where agents can be defined in code and new agents can be automatically discovered by a meta agent programming ever better ones in code. Given that programming languages are Turing Complete, this approach theoretically enables the learning of any possible agentic system: including novel prompts, tool use, control flows, and combinations thereof. We present a simple yet effective algorithm named Meta Agent Search to demonstrate this idea, where a meta agent iteratively programs interesting new agents based on an ever-growing archive of previous discoveries. Through extensive experiments across multiple domains including coding, science, and math, we show that our algorithm can progressively invent agents with novel designs that greatly outperform state-of-the-art hand-designed agents. Importantly, we consistently observe the surprising result that agents invented by Meta Agent Search maintain superior performance even when transferred across domains and models, demonstrating their robustness and generality. Provided we develop it safely, our work illustrates the potential of an exciting new research direction toward automatically designing ever-more powerful agentic systems to benefit humanity.","url_abs":"https://arxiv.org/abs/2408.08435v1","url_pdf":"https://arxiv.org/pdf/2408.08435v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"automated-design-of-agentic-systems","repo_url":"https://github.com/shengranhu/adas","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2408.08435","atlas_url":"https://app.syntology.ai/?focus=2408.08435","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.08435"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shengranhu/adas","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":10,"unverified":3},"by_repo_kind":{"official":{"samples":13,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f08515656f31c846","entry":"answer_json_to_string","repo":"shengranhu/adas","repo_kind":"official","path":"_drop/utils.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_drop/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f08515656f31c846"}},{"code_sha256_prefix":"b8b06489c92899f0","entry":"answer_json_to_strings","repo":"shengranhu/adas","repo_kind":"official","path":"_drop/utils.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_drop/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b8b06489c92899f0"}},{"code_sha256_prefix":"981c8f72d0181b71","entry":"bootstrap_confidence_interval","repo":"shengranhu/adas","repo_kind":"official","path":"_gpqa/utils.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_gpqa/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"981c8f72d0181b71"}},{"code_sha256_prefix":"765cc65c8097e240","entry":"file_to_string","repo":"shengranhu/adas","repo_kind":"official","path":"_arc/utils.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_arc/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"765cc65c8097e240"}},{"code_sha256_prefix":"d3d85aa9528f0ea0","entry":"get_drop_metrics","repo":"shengranhu/adas","repo_kind":"official","path":"_drop/utils.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_drop/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d3d85aa9528f0ea0"}},{"code_sha256_prefix":"3386b2b248cf7d4b","entry":"get_reflexion_prompt","repo":"shengranhu/adas","repo_kind":"official","path":"_arc/arc_prompt.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_arc/arc_prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3386b2b248cf7d4b"}},{"code_sha256_prefix":"03e103c0289af528","entry":"get_reflexion_prompt","repo":"shengranhu/adas","repo_kind":"official","path":"_drop/drop_prompt.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_drop/drop_prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"03e103c0289af528"}},{"code_sha256_prefix":"223b46a8f9051398","entry":"list_to_string","repo":"shengranhu/adas","repo_kind":"official","path":"_arc/utils.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_arc/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"223b46a8f9051398"}},{"code_sha256_prefix":"7b1e70edd9eaffe7","entry":"load_questions","repo":"shengranhu/adas","repo_kind":"official","path":"_gpqa/utils.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_gpqa/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7b1e70edd9eaffe7"}},{"code_sha256_prefix":"4488e68c352c9d58","entry":"random_id","repo":"shengranhu/adas","repo_kind":"official","path":"_arc/utils.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_arc/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4488e68c352c9d58"}},{"code_sha256_prefix":"e303009857c65536","entry":"get_prompt","repo":"shengranhu/adas","repo_kind":"official","path":"_arc/arc_prompt.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_arc/arc_prompt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e303009857c65536"}},{"code_sha256_prefix":"7c697edb0437e5f5","entry":"get_prompt","repo":"shengranhu/adas","repo_kind":"official","path":"_drop/drop_prompt.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_drop/drop_prompt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7c697edb0437e5f5"}},{"code_sha256_prefix":"505fd0fb8ba0fca7","entry":"get_prompt","repo":"shengranhu/adas","repo_kind":"official","path":"_gpqa/gpqa_prompt.py","file_url":"https://github.com/shengranhu/adas/blob/HEAD/_gpqa/gpqa_prompt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"505fd0fb8ba0fca7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}