{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/one-prompt-is-not-enough-automated","title":"One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts","arxiv_id":"2407.00256","date":"2024-06-28","proceeding":null,"authors":["Ruochen Wang","Sohyun An","Minhao Cheng","Tianyi Zhou","Sung Ju Hwang","Cho-Jui Hsieh"],"abstract":"Large Language Models (LLMs) exhibit strong generalization capabilities to novel tasks when prompted with language instructions and in-context demos. Since this ability sensitively depends on the quality of prompts, various methods have been explored to automate the instruction design. While these methods demonstrated promising results, they also restricted the searched prompt to one instruction. Such simplification significantly limits their capacity, as a single demo-free instruction might not be able to cover the entire complex problem space of the targeted task. To alleviate this issue, we adopt the Mixture-of-Expert paradigm and divide the problem space into a set of sub-regions; Each sub-region is governed by a specialized expert, equipped with both an instruction and a set of demos. A two-phase process is developed to construct the specialized expert for each region: (1) demo assignment: Inspired by the theoretical connection between in-context learning and kernel regression, we group demos into experts based on their semantic similarity; (2) instruction assignment: A region-based joint search of an instruction per expert complements the demos assigned to it, yielding a synergistic effect. The resulting method, codenamed Mixture-of-Prompts (MoP), achieves an average win rate of 81% against prior arts across several major benchmarks.","url_abs":"https://arxiv.org/abs/2407.00256v1","url_pdf":"https://arxiv.org/pdf/2407.00256v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"one-prompt-is-not-enough-automated","repo_url":"https://github.com/ruocwang/mixture-of-prompts","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"semantic-similarity","task_name":"Semantic Similarity"},{"task_slug":"semantic-textual-similarity","task_name":"Semantic Textual Similarity"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.00256","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.00256"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ruocwang/mixture-of-prompts","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"unverified":3},"by_repo_kind":{"official":{"samples":7,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6a34bf650bbf498c","entry":"add_demos_to_prompt","repo":"ruocwang/mixture-of-prompts","repo_kind":"official","path":"mop/src/pe_manager.py","file_url":"https://github.com/ruocwang/mixture-of-prompts/blob/HEAD/mop/src/pe_manager.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6a34bf650bbf498c"}},{"code_sha256_prefix":"f890ea0b93a37532","entry":"compute_exp_stats","repo":"ruocwang/mixture-of-prompts","repo_kind":"official","path":"mop/src/exp_utils.py","file_url":"https://github.com/ruocwang/mixture-of-prompts/blob/HEAD/mop/src/exp_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f890ea0b93a37532"}},{"code_sha256_prefix":"9c4133378318a325","entry":"kMeansRes","repo":"ruocwang/mixture-of-prompts","repo_kind":"official","path":"mop/src/algo/mixture_of_experts/utils.py","file_url":"https://github.com/ruocwang/mixture-of-prompts/blob/HEAD/mop/src/algo/mixture_of_experts/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c4133378318a325"}},{"code_sha256_prefix":"12ac50dad1160c09","entry":"set_all_seed","repo":"ruocwang/mixture-of-prompts","repo_kind":"official","path":"mop/src/misc.py","file_url":"https://github.com/ruocwang/mixture-of-prompts/blob/HEAD/mop/src/misc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"12ac50dad1160c09"}},{"code_sha256_prefix":"7f2494772490fb0a","entry":"chooseBestKforKMeansParallel","repo":"ruocwang/mixture-of-prompts","repo_kind":"official","path":"mop/src/algo/mixture_of_experts/utils.py","file_url":"https://github.com/ruocwang/mixture-of-prompts/blob/HEAD/mop/src/algo/mixture_of_experts/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7f2494772490fb0a"}},{"code_sha256_prefix":"0ed5636df814298e","entry":"pick_gpu_lowest_memory","repo":"ruocwang/mixture-of-prompts","repo_kind":"official","path":"mop/src/exp_utils.py","file_url":"https://github.com/ruocwang/mixture-of-prompts/blob/HEAD/mop/src/exp_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ed5636df814298e"}},{"code_sha256_prefix":"25ca89c7cda28f21","entry":"save_image","repo":"ruocwang/mixture-of-prompts","repo_kind":"official","path":"mop/src/exp_utils.py","file_url":"https://github.com/ruocwang/mixture-of-prompts/blob/HEAD/mop/src/exp_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"25ca89c7cda28f21"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}