{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hero-hessian-enhanced-robust-optimization-for","title":"HERO: Hessian-Enhanced Robust Optimization for Unifying and Improving Generalization and Quantization Performance","arxiv_id":"2111.11986","date":"2021-11-23","proceeding":null,"authors":["Huanrui Yang","Xiaoxuan Yang","Neil Zhenqiang Gong","Yiran Chen"],"abstract":"With the recent demand of deploying neural network models on mobile and edge devices, it is desired to improve the model's generalizability on unseen testing data, as well as enhance the model's robustness under fixed-point quantization for efficient deployment. Minimizing the training loss, however, provides few guarantees on the generalization and quantization performance. In this work, we fulfill the need of improving generalization and quantization performance simultaneously by theoretically unifying them under the framework of improving the model's robustness against bounded weight perturbation and minimizing the eigenvalues of the Hessian matrix with respect to model weights. We therefore propose HERO, a Hessian-enhanced robust optimization method, to minimize the Hessian eigenvalues through a gradient-based training process, simultaneously improving the generalization and quantization performance. HERO enables up to a 3.8% gain on test accuracy, up to 30% higher accuracy under 80% training label perturbation, and the best post-training quantization accuracy across a wide range of precision, including a >10% accuracy improvement over SGD-trained models for common model architectures on various datasets.","url_abs":"https://arxiv.org/abs/2111.11986v1","url_pdf":"https://arxiv.org/pdf/2111.11986v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hero-hessian-enhanced-robust-optimization-for","repo_url":"https://github.com/Xiaoxuan-Yang/HERO","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"quantization","task_name":"Quantization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.11986","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.11986"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Xiaoxuan-Yang/HERO","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"ran":1,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"Xiaoxuan-Yang/HERO","repo_kind":"official","path":"models/cifar/preresnet.py","file_url":"https://github.com/Xiaoxuan-Yang/HERO/blob/HEAD/models/cifar/preresnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"ac62432dc5134b0d","entry":"make_layers","repo":"Xiaoxuan-Yang/HERO","repo_kind":"official","path":"models/cifar/vgg.py","file_url":"https://github.com/Xiaoxuan-Yang/HERO/blob/HEAD/models/cifar/vgg.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ac62432dc5134b0d"}},{"code_sha256_prefix":"50afa2863ffd9fde","entry":"plot_overlap","repo":"Xiaoxuan-Yang/HERO","repo_kind":"official","path":"utils/logger.py","file_url":"https://github.com/Xiaoxuan-Yang/HERO/blob/HEAD/utils/logger.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50afa2863ffd9fde"}},{"code_sha256_prefix":"1d6e2edb8832d7b0","entry":"get_mean_and_std","repo":"Xiaoxuan-Yang/HERO","repo_kind":"official","path":"utils/misc.py","file_url":"https://github.com/Xiaoxuan-Yang/HERO/blob/HEAD/utils/misc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1d6e2edb8832d7b0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}