{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unveiling-the-hessian-s-connection-to-the","title":"Unveiling the Hessian's Connection to the Decision Boundary","arxiv_id":"2306.07104","date":"2023-06-12","proceeding":null,"authors":["Mahalakshmi Sabanayagam","Freya Behrens","Urte Adomaityte","Anna Dawid"],"abstract":"Understanding the properties of well-generalizing minima is at the heart of deep learning research. On the one hand, the generalization of neural networks has been connected to the decision boundary complexity, which is hard to study in the high-dimensional input space. Conversely, the flatness of a minimum has become a controversial proxy for generalization. In this work, we provide the missing link between the two approaches and show that the Hessian top eigenvectors characterize the decision boundary learned by the neural network. Notably, the number of outliers in the Hessian spectrum is proportional to the complexity of the decision boundary. Based on this finding, we provide a new and straightforward approach to studying the complexity of a high-dimensional decision boundary; show that this connection naturally inspires a new generalization measure; and finally, we develop a novel margin estimation technique which, in combination with the generalization measure, precisely identifies minima with simple wide-margin boundaries. Overall, this analysis establishes the connection between the Hessian and the decision boundary and provides a new method to identify minima with simple wide-margin decision boundaries.","url_abs":"https://arxiv.org/abs/2306.07104v1","url_pdf":"https://arxiv.org/pdf/2306.07104v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unveiling-the-hessian-s-connection-to-the","repo_url":"https://github.com/shmoo137/hessian-and-decision-boundary","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2306.07104","atlas_url":"https://app.syntology.ai/?focus=2306.07104","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.07104"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shmoo137/hessian-and-decision-boundary","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cb366256fe0d4206","entry":"accuracy","repo":"shmoo137/hessian-and-decision-boundary","repo_kind":"official","path":"src/training.py","file_url":"https://github.com/shmoo137/hessian-and-decision-boundary/blob/HEAD/src/training.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb366256fe0d4206"}},{"code_sha256_prefix":"d0b1de8085ebc61c","entry":"compute_neural_collapse","repo":"shmoo137/hessian-and-decision-boundary","repo_kind":"official","path":"src/hessian/neural_collapse.py","file_url":"https://github.com/shmoo137/hessian-and-decision-boundary/blob/HEAD/src/hessian/neural_collapse.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d0b1de8085ebc61c"}},{"code_sha256_prefix":"6d99bb336790eff2","entry":"dataToTorch","repo":"shmoo137/hessian-and-decision-boundary","repo_kind":"official","path":"src/datasets.py","file_url":"https://github.com/shmoo137/hessian-and-decision-boundary/blob/HEAD/src/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d99bb336790eff2"}},{"code_sha256_prefix":"19e9553f4c760ccd","entry":"data_to_torch","repo":"shmoo137/hessian-and-decision-boundary","repo_kind":"official","path":"src/datasets.py","file_url":"https://github.com/shmoo137/hessian-and-decision-boundary/blob/HEAD/src/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"19e9553f4c760ccd"}},{"code_sha256_prefix":"1dd22cb57e2ffb34","entry":"fun","repo":"shmoo137/hessian-and-decision-boundary","repo_kind":"official","path":"src/datasets.py","file_url":"https://github.com/shmoo137/hessian-and-decision-boundary/blob/HEAD/src/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1dd22cb57e2ffb34"}},{"code_sha256_prefix":"752f3d0a418577ee","entry":"generalization_measure","repo":"shmoo137/hessian-and-decision-boundary","repo_kind":"official","path":"src/hessian/generalization.py","file_url":"https://github.com/shmoo137/hessian-and-decision-boundary/blob/HEAD/src/hessian/generalization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"752f3d0a418577ee"}},{"code_sha256_prefix":"d9264d752d5f1dc0","entry":"generate_grid_data","repo":"shmoo137/hessian-and-decision-boundary","repo_kind":"official","path":"plotting/intro_explain_plot.py","file_url":"https://github.com/shmoo137/hessian-and-decision-boundary/blob/HEAD/plotting/intro_explain_plot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9264d752d5f1dc0"}},{"code_sha256_prefix":"e75b906968eecf75","entry":"hook_feature","repo":"shmoo137/hessian-and-decision-boundary","repo_kind":"official","path":"src/hessian/neural_collapse.py","file_url":"https://github.com/shmoo137/hessian-and-decision-boundary/blob/HEAD/src/hessian/neural_collapse.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e75b906968eecf75"}},{"code_sha256_prefix":"8ce8679f6cf56fe4","entry":"model_l2_norm","repo":"shmoo137/hessian-and-decision-boundary","repo_kind":"official","path":"src/architectures.py","file_url":"https://github.com/shmoo137/hessian-and-decision-boundary/blob/HEAD/src/architectures.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8ce8679f6cf56fe4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}