{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-penalty-based-bilevel-gradient-descent","title":"On Penalty-based Bilevel Gradient Descent Method","arxiv_id":"2302.05185","date":"2023-02-10","proceeding":null,"authors":["Han Shen","Quan Xiao","Tianyi Chen"],"abstract":"Bilevel optimization enjoys a wide range of applications in emerging machine learning and signal processing problems such as hyper-parameter optimization, image reconstruction, meta-learning, adversarial training, and reinforcement learning. However, bilevel optimization problems are traditionally known to be difficult to solve. Recent progress on bilevel algorithms mainly focuses on bilevel optimization problems through the lens of the implicit-gradient method, where the lower-level objective is either strongly convex or unconstrained. In this work, we tackle a challenging class of bilevel problems through the lens of the penalty method. We show that under certain conditions, the penalty reformulation recovers the (local) solutions of the original bilevel problem. Further, we propose the penalty-based bilevel gradient descent (PBGD) algorithm and establish its finite-time convergence for the constrained bilevel problem with lower-level constraints yet without lower-level strong convexity. Experiments on synthetic and real datasets showcase the efficiency of the proposed PBGD algorithm.","url_abs":"https://arxiv.org/abs/2302.05185v5","url_pdf":"https://arxiv.org/pdf/2302.05185v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-penalty-based-bilevel-gradient-descent","repo_url":"https://github.com/hanshen95/penalized-bilevel-gradient-descent","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"bilevel-optimization","task_name":"Bilevel Optimization"},{"task_slug":"image-reconstruction","task_name":"Image Reconstruction"},{"task_slug":"meta-learning","task_name":"Meta-Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2302.05185","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2302.05185"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hanshen95/penalized-bilevel-gradient-descent","reach":null}],"summary":{"ran_fixture":1,"ran_violates":3,"ran_honours":1},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"0e0f3de9c07cf33d","entry":"compute_acc","repo":"hanshen95/penalized-bilevel-gradient-descent","repo_kind":"official","path":"V-PBGD/data-hyper-cleaning/data_hyper_clean.py","file_url":"https://github.com/hanshen95/penalized-bilevel-gradient-descent/blob/HEAD/V-PBGD/data-hyper-cleaning/data_hyper_clean.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0e0f3de9c07cf33d"}},{"code_sha256_prefix":"6b2c8a3979d0712e","entry":"dg","repo":"hanshen95/penalized-bilevel-gradient-descent","repo_kind":"official","path":"V-PBGD/toy/toy.py","file_url":"https://github.com/hanshen95/penalized-bilevel-gradient-descent/blob/HEAD/V-PBGD/toy/toy.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6b2c8a3979d0712e"}},{"code_sha256_prefix":"7e299e6776b649a1","entry":"f","repo":"hanshen95/penalized-bilevel-gradient-descent","repo_kind":"official","path":"V-PBGD/toy/toy.py","file_url":"https://github.com/hanshen95/penalized-bilevel-gradient-descent/blob/HEAD/V-PBGD/toy/toy.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7e299e6776b649a1"}},{"code_sha256_prefix":"aaae9088694d9425","entry":"g","repo":"hanshen95/penalized-bilevel-gradient-descent","repo_kind":"official","path":"V-PBGD/toy/toy.py","file_url":"https://github.com/hanshen95/penalized-bilevel-gradient-descent/blob/HEAD/V-PBGD/toy/toy.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aaae9088694d9425"}},{"code_sha256_prefix":"7529f6f33361fe23","entry":"loss_L2","repo":"hanshen95/penalized-bilevel-gradient-descent","repo_kind":"official","path":"V-PBGD/data-hyper-cleaning/data_hyper_clean.py","file_url":"https://github.com/hanshen95/penalized-bilevel-gradient-descent/blob/HEAD/V-PBGD/data-hyper-cleaning/data_hyper_clean.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7529f6f33361fe23"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}