{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-gradient-based-bilevel-optimization","title":"Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond","arxiv_id":"2110.00455","date":"2021-10-01","proceeding":"NeurIPS 2021 12","authors":["Risheng Liu","Yaohua Liu","Shangzhi Zeng","Jin Zhang"],"abstract":"In recent years, Bi-Level Optimization (BLO) techniques have received extensive attentions from both learning and vision communities. A variety of BLO models in complex and practical tasks are of non-convex follower structure in nature (a.k.a., without Lower-Level Convexity, LLC for short). However, this challenging class of BLOs is lack of developments on both efficient solution strategies and solid theoretical guarantees. In this work, we propose a new algorithmic framework, named Initialization Auxiliary and Pessimistic Trajectory Truncated Gradient Method (IAPTT-GM), to partially address the above issues. In particular, by introducing an auxiliary as initialization to guide the optimization dynamics and designing a pessimistic trajectory truncation operation, we construct a reliable approximate version of the original BLO in the absence of LLC hypothesis. Our theoretical investigations establish the convergence of solutions returned by IAPTT-GM towards those of the original BLO without LLC. As an additional bonus, we also theoretically justify the quality of our IAPTT-GM embedded with Nesterov's accelerated dynamics under LLC. The experimental results confirm both the convergence of our algorithm without LLC, and the theoretical findings under LLC.","url_abs":"https://arxiv.org/abs/2110.00455v2","url_pdf":"https://arxiv.org/pdf/2110.00455v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-gradient-based-bilevel-optimization","repo_url":"https://github.com/vis-opt-group/iaptt-gm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"bilevel-optimization","task_name":"Bilevel Optimization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2110.00455","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.00455"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vis-opt-group/iaptt-gm","reach":null}],"summary":{"ran_draft_wrong":3,"ran_honours":2},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"67b4c00b3b034518","entry":"lim","repo":"vis-opt-group/iaptt-gm","repo_kind":"official","path":"experiment/Numerical.py","file_url":"https://github.com/vis-opt-group/iaptt-gm/blob/HEAD/experiment/Numerical.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67b4c00b3b034518"}},{"code_sha256_prefix":"bf56e57b52492a87","entry":"acc","repo":"vis-opt-group/iaptt-gm","repo_kind":"official","path":"experiment/Few_shot.py","file_url":"https://github.com/vis-opt-group/iaptt-gm/blob/HEAD/experiment/Few_shot.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf56e57b52492a87"}},{"code_sha256_prefix":"0d3ebc6ebabdfaa4","entry":"bias_reg_f","repo":"vis-opt-group/iaptt-gm","repo_kind":"official","path":"experiment/Few_shot.py","file_url":"https://github.com/vis-opt-group/iaptt-gm/blob/HEAD/experiment/Few_shot.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0d3ebc6ebabdfaa4"}},{"code_sha256_prefix":"a83cc4f021dc0c89","entry":"p_norm_reg","repo":"vis-opt-group/iaptt-gm","repo_kind":"official","path":"experiment/Few_shot.py","file_url":"https://github.com/vis-opt-group/iaptt-gm/blob/HEAD/experiment/Few_shot.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a83cc4f021dc0c89"}},{"code_sha256_prefix":"3a2fd59bcd150d70","entry":"proj","repo":"vis-opt-group/iaptt-gm","repo_kind":"official","path":"experiment/Numerical.py","file_url":"https://github.com/vis-opt-group/iaptt-gm/blob/HEAD/experiment/Numerical.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a2fd59bcd150d70"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}