{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2510-10618","title":"Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization","arxiv_id":"2510.10618","date":"2025-10-12","proceeding":"NeurIPS","authors":["Bowei He","Lihao Yin","Huiling Zhen","Shuqi Liu","Han Wu","Xiaokun Zhang","Mingxuan Yuan","Chen Ma"],"abstract":"Post-training compression has been a widely employed approach to scale down large language model (LLM) and facilitate efficient inference. In various proposed compression methods, including pruning and quantization, calibration data plays a vital role by informing the weight importance and activation dynamic ranges. However, how calibration data impacts the LLM capability after compression is less explored. Few of the existing works, though recognizing the significance of this study, only investigate the language modeling or commonsense reasoning performance degradation from limited angles, like the data sources or sample amounts. More systematic research is still needed to examine the impacts on different LLM capabilities in terms of compositional properties and domain correspondence of calibration data. In this work, we aim at bridging this gap and further analyze underlying influencing mechanisms from the activation pattern perspective. Especially, we explore the calibration data's impacts on high-level complex reasoning capabilities, like math problem solving and code generation. Delving into the underlying mechanism, we find that the representativeness and diversity in activation space more fundamentally determine the quality of calibration data. Finally, we propose a calibration data curation framework based on such observations and analysis, enhancing the performance of existing post-training compression methods on preserving critical LLM capabilities. Our code is provided in \\href{https://github.com/BokwaiHo/COLA.git}{Link}.","url_abs":"https://arxiv.org/abs/2510.10618","url_pdf":"https://arxiv.org/pdf/2510.10618","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2510.10618","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2510.10618"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/vllm-project/vllm","reach":null},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/EleutherAI/lm-evaluation-harness","reach":null},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/BokwaiHo/COLA","reach":{"status":"ok"}}],"summary":{"unverified":10},"by_repo_kind":{"found_in_text":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"03678da1c037f0ed","entry":"aggregate_layer_activations","repo":"BokwaiHo/COLA","repo_kind":"found_in_text","path":"cola/sample_selection.py","file_url":"https://github.com/BokwaiHo/COLA/blob/HEAD/cola/sample_selection.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"03678da1c037f0ed"}},{"code_sha256_prefix":"28c5f4153ed36a48","entry":"batch_process","repo":"BokwaiHo/COLA","repo_kind":"found_in_text","path":"cola/utils.py","file_url":"https://github.com/BokwaiHo/COLA/blob/HEAD/cola/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"28c5f4153ed36a48"}},{"code_sha256_prefix":"95e8391ce8043af8","entry":"compute_token_distribution_similarity","repo":"BokwaiHo/COLA","repo_kind":"found_in_text","path":"cola/dataset_selection.py","file_url":"https://github.com/BokwaiHo/COLA/blob/HEAD/cola/dataset_selection.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"95e8391ce8043af8"}},{"code_sha256_prefix":"111b2797d8d0faf2","entry":"enhance_format_with_reasoning","repo":"BokwaiHo/COLA","repo_kind":"found_in_text","path":"cola/dataset_processing.py","file_url":"https://github.com/BokwaiHo/COLA/blob/HEAD/cola/dataset_processing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"111b2797d8d0faf2"}},{"code_sha256_prefix":"0a94b5acfed62b30","entry":"extract_activations","repo":"BokwaiHo/COLA","repo_kind":"found_in_text","path":"cola/sample_selection.py","file_url":"https://github.com/BokwaiHo/COLA/blob/HEAD/cola/sample_selection.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0a94b5acfed62b30"}},{"code_sha256_prefix":"3d492573a8beef65","entry":"load_json","repo":"BokwaiHo/COLA","repo_kind":"found_in_text","path":"cola/utils.py","file_url":"https://github.com/BokwaiHo/COLA/blob/HEAD/cola/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3d492573a8beef65"}},{"code_sha256_prefix":"b798d93b8c6ec16f","entry":"optimize_sequence_length","repo":"BokwaiHo/COLA","repo_kind":"found_in_text","path":"cola/dataset_processing.py","file_url":"https://github.com/BokwaiHo/COLA/blob/HEAD/cola/dataset_processing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b798d93b8c6ec16f"}},{"code_sha256_prefix":"e69561f22f6e593e","entry":"random_projection","repo":"BokwaiHo/COLA","repo_kind":"found_in_text","path":"cola/sample_selection.py","file_url":"https://github.com/BokwaiHo/COLA/blob/HEAD/cola/sample_selection.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e69561f22f6e593e"}},{"code_sha256_prefix":"beccba5bfacc2638","entry":"setup_logger","repo":"BokwaiHo/COLA","repo_kind":"found_in_text","path":"cola/utils.py","file_url":"https://github.com/BokwaiHo/COLA/blob/HEAD/cola/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"beccba5bfacc2638"}},{"code_sha256_prefix":"9079bb08f9ff7bae","entry":"tokenize_text","repo":"BokwaiHo/COLA","repo_kind":"found_in_text","path":"cola/dataset_processing.py","file_url":"https://github.com/BokwaiHo/COLA/blob/HEAD/cola/dataset_processing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9079bb08f9ff7bae"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_api"},"syntology_extracted_results":null}