{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-universal-class-of-sharpness-aware","title":"A Universal Class of Sharpness-Aware Minimization Algorithms","arxiv_id":"2406.03682","date":"2024-06-06","proceeding":null,"authors":["Behrooz Tahmasebi","Ashkan Soleymani","Dara Bahri","Stefanie Jegelka","Patrick Jaillet"],"abstract":"Recently, there has been a surge in interest in developing optimization algorithms for overparameterized models as achieving generalization is believed to require algorithms with suitable biases. This interest centers on minimizing sharpness of the original loss function; the Sharpness-Aware Minimization (SAM) algorithm has proven effective. However, most literature only considers a few sharpness measures, such as the maximum eigenvalue or trace of the training loss Hessian, which may not yield meaningful insights for non-convex optimization scenarios like neural networks. Additionally, many sharpness measures are sensitive to parameter invariances in neural networks, magnifying significantly under rescaling parameters. Motivated by these challenges, we introduce a new class of sharpness measures in this paper, leading to new sharpness-aware objective functions. We prove that these measures are \\textit{universally expressive}, allowing any function of the training loss Hessian matrix to be represented by appropriate hyperparameters. Furthermore, we show that the proposed objective functions explicitly bias towards minimizing their corresponding sharpness measures, and how they allow meaningful applications to models with parameter invariances (such as scale-invariances). Finally, as instances of our proposed general framework, we present \\textit{Frob-SAM} and \\textit{Det-SAM}, which are specifically designed to minimize the Frobenius norm and the determinant of the Hessian of the training loss, respectively. We also demonstrate the advantages of our general framework through extensive experiments.","url_abs":"https://arxiv.org/abs/2406.03682v2","url_pdf":"https://arxiv.org/pdf/2406.03682v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-universal-class-of-sharpness-aware","repo_url":"https://github.com/dbahri/universal_sam","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"sharpness-aware-minimization","method_name":"Sharpness-Aware Minimization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.03682","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.03682"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/dbahri/universal_sam","reach":null}],"summary":{"ran":3,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9f74929ef23622be","entry":"FROSAM","repo":"dbahri/universal_sam","repo_kind":"official","path":"solver/sam.py","file_url":"https://github.com/dbahri/universal_sam/blob/HEAD/solver/sam.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f74929ef23622be"}},{"code_sha256_prefix":"b5c45c26158cac1e","entry":"Registry","repo":"dbahri/universal_sam","repo_kind":"official","path":"solver/sam.py","file_url":"https://github.com/dbahri/universal_sam/blob/HEAD/solver/sam.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b5c45c26158cac1e"}},{"code_sha256_prefix":"be3e23f766be2625","entry":"_called_with_args","repo":"dbahri/universal_sam","repo_kind":"official","path":"solver/sam.py","file_url":"https://github.com/dbahri/universal_sam/blob/HEAD/solver/sam.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"be3e23f766be2625"}},{"code_sha256_prefix":"71fb39740e62a991","entry":"_get_args_from_config","repo":"dbahri/universal_sam","repo_kind":"official","path":"solver/sam.py","file_url":"https://github.com/dbahri/universal_sam/blob/HEAD/solver/sam.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"71fb39740e62a991"}},{"code_sha256_prefix":"21d027bd501c4c4f","entry":"_sample_gaussian","repo":"dbahri/universal_sam","repo_kind":"official","path":"solver/sam.py","file_url":"https://github.com/dbahri/universal_sam/blob/HEAD/solver/sam.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"21d027bd501c4c4f"}},{"code_sha256_prefix":"c8391702d5318ce9","entry":"configurable","repo":"dbahri/universal_sam","repo_kind":"official","path":"solver/sam.py","file_url":"https://github.com/dbahri/universal_sam/blob/HEAD/solver/sam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8391702d5318ce9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}