{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/naive-bayes-based-context-extension-for-large","title":"Naive Bayes-based Context Extension for Large Language Models","arxiv_id":"2403.17552","date":"2024-03-26","proceeding":null,"authors":["Jianlin Su","Murtadha Ahmed","Wenbo","Luo Ao","Mingren Zhu","Yunfeng Liu"],"abstract":"Large Language Models (LLMs) have shown promising in-context learning abilities. However, conventional In-Context Learning (ICL) approaches are often impeded by length limitations of transformer architecture, which pose challenges when attempting to effectively integrate supervision from a substantial number of demonstration examples. In this paper, we introduce a novel framework, called Naive Bayes-based Context Extension (NBCE), to enable existing LLMs to perform ICL with an increased number of demonstrations by significantly expanding their context size. Importantly, this expansion does not require fine-tuning or dependence on particular model architectures, all the while preserving linear efficiency. NBCE initially splits the context into equal-sized windows fitting the target LLM's maximum length. Then, it introduces a voting mechanism to select the most relevant window, regarded as the posterior context. Finally, it employs Bayes' theorem to generate the test task. Our experimental results demonstrate that NBCE substantially enhances performance, particularly as the number of demonstration examples increases, consistently outperforming alternative methods. The NBCE code will be made publicly accessible. The code NBCE is available at: https://github.com/amurtadha/NBCE-master","url_abs":"https://arxiv.org/abs/2403.17552v1","url_pdf":"https://arxiv.org/pdf/2403.17552v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"naive-bayes-based-context-extension-for-large","repo_url":"https://github.com/amurtadha/nbce-master","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.17552","atlas_url":"https://app.syntology.ai/?focus=2403.17552","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.17552"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/amurtadha/nbce-master","reach":null}],"summary":{"ran":1,"ran_draft_wrong":1,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"754b3cad5acdefd7","entry":"RestrictiveTokensLogitsProcessor","repo":"amurtadha/nbce-master","repo_kind":"official","path":"nbce_wrapper.py","file_url":"https://github.com/amurtadha/nbce-master/blob/HEAD/nbce_wrapper.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"754b3cad5acdefd7"}},{"code_sha256_prefix":"9afb86dab2f63284","entry":"combine_past_key_values","repo":"amurtadha/nbce-master","repo_kind":"official","path":"nbce_wrapper.py","file_url":"https://github.com/amurtadha/nbce-master/blob/HEAD/nbce_wrapper.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9afb86dab2f63284"}},{"code_sha256_prefix":"07fa19b84cc795d4","entry":"n_tokens_in_prompt","repo":"amurtadha/nbce-master","repo_kind":"official","path":"nbce_wrapper.py","file_url":"https://github.com/amurtadha/nbce-master/blob/HEAD/nbce_wrapper.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"07fa19b84cc795d4"}},{"code_sha256_prefix":"895d4184a7d25996","entry":"NBCEModelWrapper","repo":"amurtadha/nbce-master","repo_kind":"official","path":"nbce_wrapper.py","file_url":"https://github.com/amurtadha/nbce-master/blob/HEAD/nbce_wrapper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"895d4184a7d25996"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}