{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/retaining-key-information-under-high","title":"Retaining Key Information under High Compression Ratios: Query-Guided Compressor for LLMs","arxiv_id":"2406.02376","date":"2024-06-04","proceeding":null,"authors":["Zhiwei Cao","Qian Cao","Yu Lu","Ningxin Peng","Luyang Huang","Shanbo Cheng","Jinsong Su"],"abstract":"The growing popularity of Large Language Models has sparked interest in context compression for Large Language Models (LLMs). However, the performance of previous methods degrades dramatically as compression ratios increase, sometimes even falling to the closed-book level. This decline can be attributed to the loss of key information during the compression process. Our preliminary study supports this hypothesis, emphasizing the significance of retaining key information to maintain model performance under high compression ratios. As a result, we introduce Query-Guided Compressor (QGC), which leverages queries to guide the context compression process, effectively preserving key information within the compressed context. Additionally, we employ a dynamic compression strategy. We validate the effectiveness of our proposed QGC on the Question Answering task, including NaturalQuestions, TriviaQA, and HotpotQA datasets. Experimental results show that QGC can consistently perform well even at high compression ratios, which also offers significant benefits in terms of inference cost and throughput.","url_abs":"https://arxiv.org/abs/2406.02376v2","url_pdf":"https://arxiv.org/pdf/2406.02376v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"retaining-key-information-under-high","repo_url":"https://github.com/DeepLearnXMU/QGC","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"triviaqa","task_name":"TriviaQA"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.02376","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.02376"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DeepLearnXMU/QGC","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"cdb46b48bcf1ab1a","entry":"expand_mask","repo":"DeepLearnXMU/QGC","repo_kind":"official","path":"src/model.py","file_url":"https://github.com/DeepLearnXMU/QGC/blob/HEAD/src/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cdb46b48bcf1ab1a"}},{"code_sha256_prefix":"966b61662ce73fd3","entry":"CCGenerationMixin","repo":"DeepLearnXMU/QGC","repo_kind":"official","path":"src/model.py","file_url":"https://github.com/DeepLearnXMU/QGC/blob/HEAD/src/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"966b61662ce73fd3"}},{"code_sha256_prefix":"d565da870de577e7","entry":"ModelWithQGC","repo":"DeepLearnXMU/QGC","repo_kind":"official","path":"src/model.py","file_url":"https://github.com/DeepLearnXMU/QGC/blob/HEAD/src/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d565da870de577e7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}