{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/understanding-hessian-alignment-for-domain","title":"Understanding Hessian Alignment for Domain Generalization","arxiv_id":"2308.11778","date":"2023-08-22","proceeding":"ICCV 2023 1","authors":["Sobhan Hemati","Guojun Zhang","Amir Estiri","Xi Chen"],"abstract":"Out-of-distribution (OOD) generalization is a critical ability for deep learning models in many real-world scenarios including healthcare and autonomous vehicles. Recently, different techniques have been proposed to improve OOD generalization. Among these methods, gradient-based regularizers have shown promising performance compared with other competitors. Despite this success, our understanding of the role of Hessian and gradient alignment in domain generalization is still limited. To address this shortcoming, we analyze the role of the classifier's head Hessian matrix and gradient in domain generalization using recent OOD theory of transferability. Theoretically, we show that spectral norm between the classifier's head Hessian matrices across domains is an upper bound of the transfer measure, a notion of distance between target and source domains. Furthermore, we analyze all the attributes that get aligned when we encourage similarity between Hessians and gradients. Our analysis explains the success of many regularizers like CORAL, IRM, V-REx, Fish, IGA, and Fishr as they regularize part of the classifier's head Hessian and/or gradient. Finally, we propose two simple yet effective methods to match the classifier's head Hessians and gradients in an efficient way, based on the Hessian Gradient Product (HGP) and Hutchinson's method (Hutchinson), and without directly calculating Hessians. We validate the OOD generalization ability of proposed methods in different scenarios, including transferability, severe correlation shift, label shift and diversity shift. Our results show that Hessian alignment methods achieve promising performance on various OOD benchmarks. The code is available at \\url{https://github.com/huawei-noah/Federated-Learning/tree/main/HessianAlignment}.","url_abs":"https://arxiv.org/abs/2308.11778v1","url_pdf":"https://arxiv.org/pdf/2308.11778v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"understanding-hessian-alignment-for-domain","repo_url":"https://github.com/huawei-noah/federated-learning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"autonomous-vehicles","task_name":"Autonomous Vehicles"},{"task_slug":"domain-generalization","task_name":"Domain Generalization"},{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[{"method_slug":"coral","method_name":"CORAL"},{"method_slug":"fishr","method_name":"Fishr"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.11778","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.11778"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huawei-noah/federated-learning","reach":{"status":"ok"}}],"summary":{"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"803f9afd7567729b","entry":"model_to_params","repo":"huawei-noah/federated-learning","repo_kind":"official","path":"Federated_Domain_Generalization/flsuite/utils/aggregate.py","file_url":"https://github.com/huawei-noah/federated-learning/blob/HEAD/Federated_Domain_Generalization/flsuite/utils/aggregate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"803f9afd7567729b"}},{"code_sha256_prefix":"a516982d7d0e59d8","entry":"round_list","repo":"huawei-noah/federated-learning","repo_kind":"official","path":"Federated_Domain_Generalization/flsuite/utils/io.py","file_url":"https://github.com/huawei-noah/federated-learning/blob/HEAD/Federated_Domain_Generalization/flsuite/utils/io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a516982d7d0e59d8"}},{"code_sha256_prefix":"b3e1a487e5e5e317","entry":"scale_model","repo":"huawei-noah/federated-learning","repo_kind":"official","path":"Federated_Domain_Generalization/flsuite/utils/aggregate.py","file_url":"https://github.com/huawei-noah/federated-learning/blob/HEAD/Federated_Domain_Generalization/flsuite/utils/aggregate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3e1a487e5e5e317"}},{"code_sha256_prefix":"38dfa34209c0ccab","entry":"zero_model","repo":"huawei-noah/federated-learning","repo_kind":"official","path":"Federated_Domain_Generalization/flsuite/utils/aggregate.py","file_url":"https://github.com/huawei-noah/federated-learning/blob/HEAD/Federated_Domain_Generalization/flsuite/utils/aggregate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"38dfa34209c0ccab"}},{"code_sha256_prefix":"e2125b1ac02e5435","entry":"fed_irm","repo":"huawei-noah/federated-learning","repo_kind":"official","path":"Federated_Domain_Generalization/flsuite/algs/fed_irm.py","file_url":"https://github.com/huawei-noah/federated-learning/blob/HEAD/Federated_Domain_Generalization/flsuite/algs/fed_irm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e2125b1ac02e5435"}},{"code_sha256_prefix":"329a55ee595b4d29","entry":"model_loader","repo":"huawei-noah/federated-learning","repo_kind":"official","path":"Federated_Domain_Generalization/flsuite/models/utils.py","file_url":"https://github.com/huawei-noah/federated-learning/blob/HEAD/Federated_Domain_Generalization/flsuite/models/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"329a55ee595b4d29"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}