{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-bilevel-learning","title":"Deep Bilevel Learning","arxiv_id":"1809.01465","date":"2018-09-05","proceeding":"ECCV 2018 9","authors":["Simon Jenni","Paolo Favaro"],"abstract":"We present a novel regularization approach to train neural networks that\nenjoys better generalization and test error than standard stochastic gradient\ndescent. Our approach is based on the principles of cross-validation, where a\nvalidation set is used to limit the model overfitting. We formulate such\nprinciples as a bilevel optimization problem. This formulation allows us to\ndefine the optimization of a cost on the validation set subject to another\noptimization on the training set. The overfitting is controlled by introducing\nweights on each mini-batch in the training set and by choosing their values so\nthat they minimize the error on the validation set. In practice, these weights\ndefine mini-batch learning rates in a gradient descent update equation that\nfavor gradients with better generalization capabilities. Because of its\nsimplicity, this approach can be integrated with other regularization methods\nand training schemes. We evaluate extensively our proposed algorithm on several\nneural network architectures and datasets, and find that it consistently\nimproves the generalization of the model, especially when labels are noisy.","url_abs":"http://arxiv.org/abs/1809.01465v1","url_pdf":"http://arxiv.org/pdf/1809.01465v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-bilevel-learning","repo_url":"https://github.com/sjenni/DeepBilevel","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"bilevel-optimization","task_name":"Bilevel Optimization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1809.01465","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1809.01465"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sjenni/DeepBilevel","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c9376dbf970ce49f","entry":"average_gradients","repo":"sjenni/DeepBilevel","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/sjenni/DeepBilevel/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c9376dbf970ce49f"}},{"code_sha256_prefix":"26d38c080c069fd3","entry":"montage_tf","repo":"sjenni/DeepBilevel","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/sjenni/DeepBilevel/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26d38c080c069fd3"}},{"code_sha256_prefix":"e44e316ae9f9f180","entry":"remove_missing","repo":"sjenni/DeepBilevel","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/sjenni/DeepBilevel/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e44e316ae9f9f180"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}