{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/coverage-based-example-selection-for-in","title":"Coverage-based Example Selection for In-Context Learning","arxiv_id":"2305.14907","date":"2023-05-24","proceeding":null,"authors":["Shivanshu Gupta","Matt Gardner","Sameer Singh"],"abstract":"In-context learning (ICL), the ability of large language models to perform novel tasks by conditioning on a prompt with a few task examples, requires these examples to be informative about the test instance. The standard approach of independently ranking and selecting the most similar examples selects redundant examples while omitting important information. In this work, we show that BERTScore-Recall (BSR) selects better examples that demonstrate more of the salient aspects, e.g. reasoning patterns, of the test input. We further extend BSR and many standard metrics to easily optimizable set-level metrics, giving still better coverage of those salient aspects. On 15 datasets spanning 6 tasks and with 7 diverse LLMs, we show that (1) BSR is the superior metric for in-context example selection across the board, and (2) for compositional tasks, set selection using Set-BSR outperforms independent ranking by up to 17 points on average and, despite being training-free, surpasses methods that leverage task or LLM-specific training.","url_abs":"https://arxiv.org/abs/2305.14907v3","url_pdf":"https://arxiv.org/pdf/2305.14907v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"coverage-based-example-selection-for-in","repo_url":"https://github.com/shivanshu-gupta/icl-coverage","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"informativeness","task_name":"Informativeness"}],"methods":[{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.14907","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.14907"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shivanshu-gupta/icl-coverage","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7da8b3036d0762ab","entry":"bag_relevance","repo":"shivanshu-gupta/icl-coverage","repo_kind":"official","path":"src/selector/base.py","file_url":"https://github.com/shivanshu-gupta/icl-coverage/blob/HEAD/src/selector/base.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7da8b3036d0762ab"}},{"code_sha256_prefix":"a3c62fc1e4649b4a","entry":"cat_tensors","repo":"shivanshu-gupta/icl-coverage","repo_kind":"official","path":"src/selector/bertscore.py","file_url":"https://github.com/shivanshu-gupta/icl-coverage/blob/HEAD/src/selector/bertscore.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3c62fc1e4649b4a"}},{"code_sha256_prefix":"4390839db3bfd07d","entry":"get_combinations","repo":"shivanshu-gupta/icl-coverage","repo_kind":"official","path":"src/selector/base.py","file_url":"https://github.com/shivanshu-gupta/icl-coverage/blob/HEAD/src/selector/base.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4390839db3bfd07d"}},{"code_sha256_prefix":"bfd350fa27b2ed28","entry":"list_tensors","repo":"shivanshu-gupta/icl-coverage","repo_kind":"official","path":"src/selector/bertscore.py","file_url":"https://github.com/shivanshu-gupta/icl-coverage/blob/HEAD/src/selector/bertscore.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bfd350fa27b2ed28"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}