{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/asymmetric-valleys-beyond-sharp-and-flat","title":"Asymmetric Valleys: Beyond Sharp and Flat Local Minima","arxiv_id":"1902.00744","date":"2019-02-02","proceeding":"NeurIPS 2019 12","authors":["Haowei He","Gao Huang","Yang Yuan"],"abstract":"Despite the non-convex nature of their loss functions, deep neural networks\nare known to generalize well when optimized with stochastic gradient descent\n(SGD). Recent work conjectures that SGD with proper configuration is able to\nfind wide and flat local minima, which have been proposed to be associated with\ngood generalization performance. In this paper, we observe that local minima of\nmodern deep networks are more than being flat or sharp. Specifically, at a\nlocal minimum there exist many asymmetric directions such that the loss\nincreases abruptly along one side, and slowly along the opposite side--we\nformally define such minima as asymmetric valleys. Under mild assumptions, we\nprove that for asymmetric valleys, a solution biased towards the flat side\ngeneralizes better than the exact minimizer. Further, we show that simply\naveraging the weights along the SGD trajectory gives rise to such biased\nsolutions implicitly. This provides a theoretical explanation for the\nintriguing phenomenon observed by Izmailov et al. (2018). In addition, we\nempirically find that batch normalization (BN) appears to be a major cause for\nasymmetric valleys.","url_abs":"http://arxiv.org/abs/1902.00744v2","url_pdf":"http://arxiv.org/pdf/1902.00744v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"asymmetric-valleys-beyond-sharp-and-flat","repo_url":"https://github.com/962086838/code-for-Asymmetric-Valley","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"sgd","method_name":"SGD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1902.00744","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1902.00744"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/962086838/code-for-Asymmetric-Valley","reach":null}],"summary":{"ran_honours":2,"unverified":2},"by_repo_kind":{"listed":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"80ba171bb3f25f9a","entry":"parameters_to_vector","repo":"962086838/code-for-Asymmetric-Valley","repo_kind":"listed","path":"logistic_regression_2params.py","file_url":"https://github.com/962086838/code-for-Asymmetric-Valley/blob/HEAD/logistic_regression_2params.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"80ba171bb3f25f9a"}},{"code_sha256_prefix":"deabc35581daae80","entry":"target_transform","repo":"962086838/code-for-Asymmetric-Valley","repo_kind":"listed","path":"sgd_after_swa.py","file_url":"https://github.com/962086838/code-for-Asymmetric-Valley/blob/HEAD/sgd_after_swa.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"deabc35581daae80"}},{"code_sha256_prefix":"e157f24f49e210fb","entry":"find_asym","repo":"962086838/code-for-Asymmetric-Valley","repo_kind":"listed","path":"logistic_regression_2params.py","file_url":"https://github.com/962086838/code-for-Asymmetric-Valley/blob/HEAD/logistic_regression_2params.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e157f24f49e210fb"}},{"code_sha256_prefix":"6d7175c60d08e872","entry":"schedule","repo":"962086838/code-for-Asymmetric-Valley","repo_kind":"listed","path":"train2.py","file_url":"https://github.com/962086838/code-for-Asymmetric-Valley/blob/HEAD/train2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6d7175c60d08e872"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}