{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/distance-between-relevant-information-pieces","title":"Distance between Relevant Information Pieces Causes Bias in Long-Context LLMs","arxiv_id":"2410.14641","date":"2024-10-18","proceeding":null,"authors":["Runchu Tian","Yanghao Li","Yuepeng Fu","Siyang Deng","Qinyu Luo","Cheng Qian","Shuo Wang","Xin Cong","Zhong Zhang","Yesai Wu","Yankai Lin","Huadong Wang","Xiaojiang Liu"],"abstract":"Positional bias in large language models (LLMs) hinders their ability to effectively process long inputs. A prominent example is the \"lost in the middle\" phenomenon, where LLMs struggle to utilize relevant information situated in the middle of the input. While prior research primarily focuses on single pieces of relevant information, real-world applications often involve multiple relevant information pieces. To bridge this gap, we present LongPiBench, a benchmark designed to assess positional bias involving multiple pieces of relevant information. Thorough experiments are conducted with five commercial and six open-source models. These experiments reveal that while most current models are robust against the \"lost in the middle\" issue, there exist significant biases related to the spacing of relevant information pieces. These findings highlight the importance of evaluating and reducing positional biases to advance LLM's capabilities.","url_abs":"https://arxiv.org/abs/2410.14641v1","url_pdf":"https://arxiv.org/pdf/2410.14641v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"distance-between-relevant-information-pieces","repo_url":"https://github.com/Rachum-thu/LongPiBench","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.14641","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.14641"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Rachum-thu/LongPiBench","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d29263a372e88f4b","entry":"analyze_task_model","repo":"Rachum-thu/LongPiBench","repo_kind":"official","path":"visual/visualize.py","file_url":"https://github.com/Rachum-thu/LongPiBench/blob/HEAD/visual/visualize.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d29263a372e88f4b"}},{"code_sha256_prefix":"f92302698fc1cdeb","entry":"analyze_task_model","repo":"Rachum-thu/LongPiBench","repo_kind":"official","path":"visual/visualize_query.py","file_url":"https://github.com/Rachum-thu/LongPiBench/blob/HEAD/visual/visualize_query.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f92302698fc1cdeb"}},{"code_sha256_prefix":"98a52295395bbc3a","entry":"darken_color","repo":"Rachum-thu/LongPiBench","repo_kind":"official","path":"visual/visualize.py","file_url":"https://github.com/Rachum-thu/LongPiBench/blob/HEAD/visual/visualize.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98a52295395bbc3a"}},{"code_sha256_prefix":"11bf077f72f9d4c0","entry":"sanitize_filename","repo":"Rachum-thu/LongPiBench","repo_kind":"official","path":"visual/visualize.py","file_url":"https://github.com/Rachum-thu/LongPiBench/blob/HEAD/visual/visualize.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"11bf077f72f9d4c0"}},{"code_sha256_prefix":"e4196dba61412675","entry":"llm_generate","repo":"Rachum-thu/LongPiBench","repo_kind":"official","path":"src/llm/llm.py","file_url":"https://github.com/Rachum-thu/LongPiBench/blob/HEAD/src/llm/llm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4196dba61412675"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}