{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/model-preserving-adaptive-rounding","title":"Model-Preserving Adaptive Rounding","arxiv_id":"2505.22988","date":"2025-05-29","proceeding":null,"authors":["Albert Tseng","Zhaofeng Sun","Christopher De Sa"],"abstract":"The main goal of post-training quantization (PTQ) is to produced a compressed model whose output distribution is as close to the original model's as possible. To do this tractably, almost all LLM PTQ algorithms quantize linear layers by independently minimizing the immediate activation error. However, this localized objective ignores the effect of subsequent layers, so reducing it does not necessarily give a closer model. In this work, we introduce Yet Another Quantization Algorithm (YAQA), an adaptive rounding algorithm that uses Kronecker-factored approximations of each linear layer's Hessian with respect to the \\textit{full model} KL divergence. YAQA consists of two components: Kronecker-factored sketches of the full layerwise Hessian that can be tractably computed for hundred-billion parameter LLMs, and a quantizer-independent rounding algorithm that uses these sketches and comes with theoretical guarantees. Across a wide range of models and quantizers, YAQA empirically reduces the KL divergence to the original model by $\\approx 30\\%$ while achieving state of the art performance on downstream tasks.","url_abs":"https://arxiv.org/abs/2505.22988v1","url_pdf":"https://arxiv.org/pdf/2505.22988v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"model-preserving-adaptive-rounding","repo_url":"https://github.com/cornell-relaxml/yaqa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"model-preserving-adaptive-rounding","repo_url":"https://github.com/snu-mllab/guidedquant","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"quantization","task_name":"Quantization"},{"task_slug":"model","task_name":"model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2505.22988","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.22988"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Cornell-RelaxML/yaqa-quantization","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cornell-relaxml/yaqa","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/snu-mllab/guidedquant","reach":null}],"summary":{"ran_fixture":1,"ran_violates":1,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":3,"ran":1,"repositories":1},"found_in_text":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"56d26021b8f5c3d6","entry":"LDLQ_2hess","repo":"cornell-relaxml/yaqa","repo_kind":"official","path":"lib/algo/ldlq.py","file_url":"https://github.com/cornell-relaxml/yaqa/blob/HEAD/lib/algo/ldlq.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"56d26021b8f5c3d6"}},{"code_sha256_prefix":"379ac2a276b17fe0","entry":"check_exist","repo":"Cornell-RelaxML/yaqa-quantization","repo_kind":"found_in_text","path":"quantize_llama/quantize_finetune_llama.py","file_url":"https://github.com/Cornell-RelaxML/yaqa-quantization/blob/HEAD/quantize_llama/quantize_finetune_llama.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"379ac2a276b17fe0"}},{"code_sha256_prefix":"ea6e2956fb238d26","entry":"objective_function","repo":"snu-mllab/guidedquant","repo_kind":"listed","path":"any_precision/quantization/layerwise_quantize.py","file_url":"https://github.com/snu-mllab/guidedquant/blob/HEAD/any_precision/quantization/layerwise_quantize.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ea6e2956fb238d26"}},{"code_sha256_prefix":"250433daab43f15e","entry":"train_least_squares","repo":"snu-mllab/guidedquant","repo_kind":"listed","path":"any_precision/quantization/layerwise_quantize.py","file_url":"https://github.com/snu-mllab/guidedquant/blob/HEAD/any_precision/quantization/layerwise_quantize.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"250433daab43f15e"}},{"code_sha256_prefix":"b613cb051ae53a82","entry":"update_C","repo":"snu-mllab/guidedquant","repo_kind":"listed","path":"any_precision/quantization/layerwise_quantize.py","file_url":"https://github.com/snu-mllab/guidedquant/blob/HEAD/any_precision/quantization/layerwise_quantize.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b613cb051ae53a82"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}