{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/compositional-exemplars-for-in-context","title":"Compositional Exemplars for In-context Learning","arxiv_id":"2302.05698","date":"2023-02-11","proceeding":null,"authors":["Jiacheng Ye","Zhiyong Wu","Jiangtao Feng","Tao Yu","Lingpeng Kong"],"abstract":"Large pretrained language models (LMs) have shown impressive In-Context Learning (ICL) ability, where the model learns to do an unseen task via a prompt consisting of input-output examples as the demonstration, without any parameter updates. The performance of ICL is highly dominated by the quality of the selected in-context examples. However, previous selection methods are mostly based on simple heuristics, leading to sub-optimal performance. In this work, we formulate in-context example selection as a subset selection problem. We propose CEIL (Compositional Exemplars for In-context Learning), which is instantiated by Determinantal Point Processes (DPPs) to model the interaction between the given input and in-context examples, and optimized through a carefully-designed contrastive learning objective to obtain preference from LMs. We validate CEIL on 12 classification and generation datasets from 7 distinct NLP tasks, including sentiment analysis, paraphrase detection, natural language inference, commonsense reasoning, open-domain question answering, code generation, and semantic parsing. Extensive experiments demonstrate not only the state-of-the-art performance but also the transferability and compositionality of CEIL, shedding new light on effective and efficient in-context learning. Our code is released at https://github.com/HKUNLP/icl-ceil.","url_abs":"https://arxiv.org/abs/2302.05698v3","url_pdf":"https://arxiv.org/pdf/2302.05698v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"compositional-exemplars-for-in-context","repo_url":"https://github.com/hkunlp/icl-ceil","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"code-generation","task_name":"Code Generation"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"open-domain-question-answering","task_name":"Open-Domain Question Answering"},{"task_slug":"point-processes","task_name":"Point Processes"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"semantic-parsing","task_name":"Semantic Parsing"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2302.05698","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2302.05698"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/HKUNLP/icl-ceil","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hkunlp/icl-ceil","reach":null}],"summary":{"ran":1,"ran_honours":1,"ran_draft_wrong":3,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"b18250859a814aa5","entry":"BiEncoderConfig","repo":"hkunlp/icl-ceil","repo_kind":"official","path":"src/models/biencoder.py","file_url":"https://github.com/hkunlp/icl-ceil/blob/HEAD/src/models/biencoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b18250859a814aa5"}},{"code_sha256_prefix":"6926ec521fecfeac","entry":"create_indices","repo":"hkunlp/icl-ceil","repo_kind":"official","path":"src/models/biencoder.py","file_url":"https://github.com/hkunlp/icl-ceil/blob/HEAD/src/models/biencoder.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6926ec521fecfeac"}},{"code_sha256_prefix":"77f69af078e09830","entry":"mean_pooling","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"77f69af078e09830"}},{"code_sha256_prefix":"6352da8da5ebf5d1","entry":"ranking_loss","repo":"hkunlp/icl-ceil","repo_kind":"official","path":"src/models/biencoder.py","file_url":"https://github.com/hkunlp/icl-ceil/blob/HEAD/src/models/biencoder.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6352da8da5ebf5d1"}},{"code_sha256_prefix":"3765584672d74545","entry":"run_api","repo":"HKUNLP/icl-ceil","repo_kind":"official","path":"src/models/api_client.py","file_url":"https://github.com/HKUNLP/icl-ceil/blob/HEAD/src/models/api_client.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3765584672d74545"}},{"code_sha256_prefix":"235b32ed3c02ddf7","entry":"BiEncoder","repo":"hkunlp/icl-ceil","repo_kind":"official","path":"src/models/biencoder.py","file_url":"https://github.com/hkunlp/icl-ceil/blob/HEAD/src/models/biencoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"235b32ed3c02ddf7"}},{"code_sha256_prefix":"72c214a0f7d521af","entry":"indexing","repo":"hkunlp/icl-ceil","repo_kind":"official","path":"src/models/biencoder.py","file_url":"https://github.com/hkunlp/icl-ceil/blob/HEAD/src/models/biencoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"72c214a0f7d521af"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}