{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reordering-examples-helps-during-priming","title":"Reordering Examples Helps during Priming-based Few-Shot Learning","arxiv_id":"2106.01751","date":"2021-06-03","proceeding":"Findings (ACL) 2021 8","authors":["Sawan Kumar","Partha Talukdar"],"abstract":"The ability to learn from limited data, or few-shot learning, is a desirable and often critical requirement for NLP systems. While many existing methods do poorly at learning from a handful of examples, large pretrained language models have recently been shown to be efficient few-shot learners. One approach to few-shot learning, which does not require finetuning of model parameters, is to augment the language model's input with priming text which is typically constructed using task specific descriptions and examples. In this work, we further explore priming-based few-shot learning, with focus on using examples as prompts. We show that presenting examples in the right order is key for generalization. We introduce PERO (Prompting with Examples in the Right Order), where we formulate few-shot learning as search over the set of permutations of the training examples. We show that PERO can learn to generalize efficiently using as few as 10 examples, in contrast to existing approaches. While the newline token is a natural choice for separating the examples in the prompt, we show that learning a new separator token can potentially provide further gains in performance. We demonstrate the effectiveness of the proposed method on the tasks of sentiment classification, natural language inference and fact retrieval. Finally, we analyze the learned prompts to reveal novel insights, including the idea that two training examples in the right order alone can provide competitive performance for sentiment classification and natural language inference.","url_abs":"https://arxiv.org/abs/2106.01751v1","url_pdf":"https://arxiv.org/pdf/2106.01751v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reordering-examples-helps-during-priming","repo_url":"https://github.com/SawanKumar28/pero","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"sentiment-classification","task_name":"Sentiment Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.01751","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.01751"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SawanKumar28/pero","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/ucinlp/autoprompt","reach":null}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1},"found_in_text":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bfaef19b1a497d58","entry":"PredictWrapper","repo":"ucinlp/autoprompt","repo_kind":"found_in_text","path":"autoprompt/create_trigger.py","file_url":"https://github.com/ucinlp/autoprompt/blob/HEAD/autoprompt/create_trigger.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bfaef19b1a497d58"}},{"code_sha256_prefix":"cb904be0b5c6abc4","entry":"TriggerTemplatizer","repo":"ucinlp/autoprompt","repo_kind":"found_in_text","path":"autoprompt/create_trigger.py","file_url":"https://github.com/ucinlp/autoprompt/blob/HEAD/autoprompt/create_trigger.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cb904be0b5c6abc4"}},{"code_sha256_prefix":"c8fc624ca613854e","entry":"load_trigger_dataset","repo":"ucinlp/autoprompt","repo_kind":"found_in_text","path":"autoprompt/create_trigger.py","file_url":"https://github.com/ucinlp/autoprompt/blob/HEAD/autoprompt/create_trigger.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c8fc624ca613854e"}},{"code_sha256_prefix":"fa053ba6d77847c8","entry":"GA","repo":"SawanKumar28/pero","repo_kind":"official","path":"src/opt_utils.py","file_url":"https://github.com/SawanKumar28/pero/blob/HEAD/src/opt_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fa053ba6d77847c8"}},{"code_sha256_prefix":"ce83164ae06e5fae","entry":"PermutationHandler","repo":"SawanKumar28/pero","repo_kind":"official","path":"src/opt_utils.py","file_url":"https://github.com/SawanKumar28/pero/blob/HEAD/src/opt_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce83164ae06e5fae"}},{"code_sha256_prefix":"8bda67b843418308","entry":"SInstance","repo":"SawanKumar28/pero","repo_kind":"official","path":"src/opt_utils.py","file_url":"https://github.com/SawanKumar28/pero/blob/HEAD/src/opt_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8bda67b843418308"}},{"code_sha256_prefix":"de4d142f8cd18f36","entry":"run_model","repo":"ucinlp/autoprompt","repo_kind":"found_in_text","path":"autoprompt/create_trigger.py","file_url":"https://github.com/ucinlp/autoprompt/blob/HEAD/autoprompt/create_trigger.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"de4d142f8cd18f36"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}