{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/meta-prompting-enhancing-language-models-with","title":"Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding","arxiv_id":"2401.12954","date":"2024-01-23","proceeding":null,"authors":["Mirac Suzgun","Adam Tauman Kalai"],"abstract":"We introduce meta-prompting, an effective scaffolding technique designed to enhance the functionality of language models (LMs). This approach transforms a single LM into a multi-faceted conductor, adept at managing and integrating multiple independent LM queries. By employing high-level instructions, meta-prompting guides the LM to break down complex tasks into smaller, more manageable subtasks. These subtasks are then handled by distinct \"expert\" instances of the same LM, each operating under specific, tailored instructions. Central to this process is the LM itself, in its role as the conductor, which ensures seamless communication and effective integration of the outputs from these expert models. It additionally employs its inherent critical thinking and robust verification processes to refine and authenticate the end result. This collaborative prompting approach empowers a single LM to simultaneously act as a comprehensive orchestrator and a panel of diverse experts, significantly enhancing its performance across a wide array of tasks. The zero-shot, task-agnostic nature of meta-prompting greatly simplifies user interaction by obviating the need for detailed, task-specific instructions. Furthermore, our research demonstrates the seamless integration of external tools, such as a Python interpreter, into the meta-prompting framework, thereby broadening its applicability and utility. Through rigorous experimentation with GPT-4, we establish the superiority of meta-prompting over conventional scaffolding methods: When averaged across all tasks, including the Game of 24, Checkmate-in-One, and Python Programming Puzzles, meta-prompting, augmented with a Python interpreter functionality, surpasses standard prompting by 17.1%, expert (dynamic) prompting by 17.3%, and multipersona prompting by 15.2%.","url_abs":"https://arxiv.org/abs/2401.12954v1","url_pdf":"https://arxiv.org/pdf/2401.12954v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"meta-prompting-enhancing-language-models-with","repo_url":"https://github.com/suzgunmirac/meta-prompting","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"checkmate-in-one","task_name":"Checkmate In One"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.12954","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.12954"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/suzgunmirac/meta-prompting","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":6},"by_repo_kind":{"official":{"samples":7,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fb7a42d3ba64d976","entry":"execute_code_with_timeout","repo":"suzgunmirac/meta-prompting","repo_kind":"official","path":"utils/execute_code.py","file_url":"https://github.com/suzgunmirac/meta-prompting/blob/HEAD/utils/execute_code.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fb7a42d3ba64d976"}},{"code_sha256_prefix":"58e2d070ad8d11a2","entry":"clean_line","repo":"suzgunmirac/meta-prompting","repo_kind":"official","path":"utils/sonnet_eval.py","file_url":"https://github.com/suzgunmirac/meta-prompting/blob/HEAD/utils/sonnet_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58e2d070ad8d11a2"}},{"code_sha256_prefix":"66641b83b667189e","entry":"clean_output_for_GameOf24","repo":"suzgunmirac/meta-prompting","repo_kind":"official","path":"evaluate_outputs.py","file_url":"https://github.com/suzgunmirac/meta-prompting/blob/HEAD/evaluate_outputs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"66641b83b667189e"}},{"code_sha256_prefix":"e5e3e85a8f617a60","entry":"clean_output_for_arithmetic","repo":"suzgunmirac/meta-prompting","repo_kind":"official","path":"evaluate_outputs.py","file_url":"https://github.com/suzgunmirac/meta-prompting/blob/HEAD/evaluate_outputs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5e3e85a8f617a60"}},{"code_sha256_prefix":"5c4ffbd4430cff7d","entry":"clean_word","repo":"suzgunmirac/meta-prompting","repo_kind":"official","path":"utils/sonnet_eval.py","file_url":"https://github.com/suzgunmirac/meta-prompting/blob/HEAD/utils/sonnet_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c4ffbd4430cff7d"}},{"code_sha256_prefix":"58a6661987e8b91b","entry":"extract_answer","repo":"suzgunmirac/meta-prompting","repo_kind":"official","path":"evaluate_outputs.py","file_url":"https://github.com/suzgunmirac/meta-prompting/blob/HEAD/evaluate_outputs.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58a6661987e8b91b"}},{"code_sha256_prefix":"933a16fa4bc8a082","entry":"run_model","repo":"suzgunmirac/meta-prompting","repo_kind":"official","path":"run_experiments.py","file_url":"https://github.com/suzgunmirac/meta-prompting/blob/HEAD/run_experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"933a16fa4bc8a082"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}