{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/batch-icl-effective-efficient-and-order","title":"Batch-ICL: Effective, Efficient, and Order-Agnostic In-Context Learning","arxiv_id":"2401.06469","date":"2024-01-12","proceeding":null,"authors":["Kaiyi Zhang","Ang Lv","Yuhan Chen","Hansen Ha","Tao Xu","Rui Yan"],"abstract":"In this paper, by treating in-context learning (ICL) as a meta-optimization process, we explain why LLMs are sensitive to the order of ICL examples. This understanding leads us to the development of Batch-ICL, an effective, efficient, and order-agnostic inference algorithm for ICL. Differing from the standard N-shot learning approach, Batch-ICL employs $N$ separate 1-shot forward computations and aggregates the resulting meta-gradients. These aggregated meta-gradients are then applied to the forward computation of a zero-shot query to generate the final prediction. This batch processing approach renders the LLM agnostic to the order of ICL examples. Through extensive experiments and analysis, we demonstrate that Batch-ICL consistently outperforms most permutations of ICL examples. In some cases, it even exceeds the performance of the best order for standard ICL, all while reducing the computational resources required. Furthermore, we develop a novel variant of Batch-ICL featuring multiple \"epochs\" of meta-optimization. This variant implicitly explores permutations of ICL examples, further enhancing ICL performance.","url_abs":"https://arxiv.org/abs/2401.06469v3","url_pdf":"https://arxiv.org/pdf/2401.06469v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"batch-icl-effective-efficient-and-order","repo_url":"https://github.com/cardinalere/batch-icl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.06469","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.06469"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cardinalere/batch-icl","reach":{"status":"ok"}}],"summary":{"ran_fixture":2,"ran_honours":1,"ran":5,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":10,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"30d7eec482ebf6b1","entry":"repeat_kv","repo":"cardinalere/batch-icl","repo_kind":"official","path":"transformers/models/llama/modeling_llama.py","file_url":"https://github.com/cardinalere/batch-icl/blob/HEAD/transformers/models/llama/modeling_llama.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":2,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"30d7eec482ebf6b1"}},{"code_sha256_prefix":"d61c483a3c2b3156","entry":"apply_rotary_pos_emb","repo":"cardinalere/batch-icl","repo_kind":"official","path":"transformers/models/llama/modeling_llama.py","file_url":"https://github.com/cardinalere/batch-icl/blob/HEAD/transformers/models/llama/modeling_llama.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d61c483a3c2b3156"}},{"code_sha256_prefix":"f01688d0434b6835","entry":"compute_metric","repo":"cardinalere/batch-icl","repo_kind":"official","path":"Batch-ICL-multi_epoch.py","file_url":"https://github.com/cardinalere/batch-icl/blob/HEAD/Batch-ICL-multi_epoch.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f01688d0434b6835"}},{"code_sha256_prefix":"6ccd8b3dbcb91ca8","entry":"compute_metric","repo":"cardinalere/batch-icl","repo_kind":"official","path":"BatchICL_generation.py","file_url":"https://github.com/cardinalere/batch-icl/blob/HEAD/BatchICL_generation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6ccd8b3dbcb91ca8"}},{"code_sha256_prefix":"1cb9fbc9b1395b2c","entry":"compute_metric_SST2","repo":"cardinalere/batch-icl","repo_kind":"official","path":"BatchICL_generation.py","file_url":"https://github.com/cardinalere/batch-icl/blob/HEAD/BatchICL_generation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1cb9fbc9b1395b2c"}},{"code_sha256_prefix":"19fcefba27a9d7f6","entry":"compute_metric_layers","repo":"cardinalere/batch-icl","repo_kind":"official","path":"Batch-ICL-multi_epoch.py","file_url":"https://github.com/cardinalere/batch-icl/blob/HEAD/Batch-ICL-multi_epoch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19fcefba27a9d7f6"}},{"code_sha256_prefix":"57f56ea7eb7b02f1","entry":"compute_metric_layers","repo":"cardinalere/batch-icl","repo_kind":"official","path":"Batch-ICL.py","file_url":"https://github.com/cardinalere/batch-icl/blob/HEAD/Batch-ICL.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"57f56ea7eb7b02f1"}},{"code_sha256_prefix":"56d85b45bef33cdc","entry":"compute_metric_layers","repo":"cardinalere/batch-icl","repo_kind":"official","path":"Batch-ICL_best.py","file_url":"https://github.com/cardinalere/batch-icl/blob/HEAD/Batch-ICL_best.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"56d85b45bef33cdc"}},{"code_sha256_prefix":"6ab745408cb8648b","entry":"format_subject","repo":"cardinalere/batch-icl","repo_kind":"official","path":"Batch-ICL-multi_epoch.py","file_url":"https://github.com/cardinalere/batch-icl/blob/HEAD/Batch-ICL-multi_epoch.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6ab745408cb8648b"}},{"code_sha256_prefix":"b99eea6376d1e212","entry":"rotate_half","repo":"cardinalere/batch-icl","repo_kind":"official","path":"transformers/models/llama/modeling_llama.py","file_url":"https://github.com/cardinalere/batch-icl/blob/HEAD/transformers/models/llama/modeling_llama.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b99eea6376d1e212"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}