{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/implicit-regularization-in-deep-learning","title":"Implicit Regularization in Deep Learning","arxiv_id":"1709.01953","date":"2017-09-06","proceeding":null,"authors":["Behnam Neyshabur"],"abstract":"In an attempt to better understand generalization in deep learning, we study\nseveral possible explanations. We show that implicit regularization induced by\nthe optimization method is playing a key role in generalization and success of\ndeep learning models. Motivated by this view, we study how different complexity\nmeasures can ensure generalization and explain how optimization algorithms can\nimplicitly regularize complexity measures. We empirically investigate the\nability of these measures to explain different observed phenomena in deep\nlearning. We further study the invariances in neural networks, suggest\ncomplexity measures and optimization algorithms that have similar invariances\nto those in neural networks and evaluate them on a number of learning tasks.","url_abs":"http://arxiv.org/abs/1709.01953v2","url_pdf":"http://arxiv.org/pdf/1709.01953v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"implicit-regularization-in-deep-learning","repo_url":"https://github.com/bneyshabur/generalization-bounds","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1709.01953","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1709.01953"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bneyshabur/generalization-bounds","reach":null}],"summary":{"ran_honours":2,"ran_draft_wrong":1},"by_repo_kind":{},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"675d71c0943faf48","entry":"calc_measure","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"675d71c0943faf48"}},{"code_sha256_prefix":"42c393718dceb132","entry":"norm","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"42c393718dceb132"}},{"code_sha256_prefix":"4cfa0f04d54564b0","entry":"reparam","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"4cfa0f04d54564b0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}