{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/averaged-method-of-multipliers-for-bi-level","title":"Averaged Method of Multipliers for Bi-Level Optimization without Lower-Level Strong Convexity","arxiv_id":"2302.03407","date":"2023-02-07","proceeding":null,"authors":["Risheng Liu","Yaohua Liu","Wei Yao","Shangzhi Zeng","Jin Zhang"],"abstract":"Gradient methods have become mainstream techniques for Bi-Level Optimization (BLO) in learning fields. The validity of existing works heavily rely on either a restrictive Lower-Level Strong Convexity (LLSC) condition or on solving a series of approximation subproblems with high accuracy or both. In this work, by averaging the upper and lower level objectives, we propose a single loop Bi-level Averaged Method of Multipliers (sl-BAMM) for BLO that is simple yet efficient for large-scale BLO and gets rid of the limited LLSC restriction. We further provide non-asymptotic convergence analysis of sl-BAMM towards KKT stationary points, and the comparative advantage of our analysis lies in the absence of strong gradient boundedness assumption, which is always required by others. Thus our theory safely captures a wider variety of applications in deep learning, especially where the upper-level objective is quadratic w.r.t. the lower-level variable. Experimental results demonstrate the superiority of our method.","url_abs":"https://arxiv.org/abs/2302.03407v2","url_pdf":"https://arxiv.org/pdf/2302.03407v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"averaged-method-of-multipliers-for-bi-level","repo_url":"https://github.com/vis-opt-group/sl-bamm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2302.03407","atlas_url":"https://app.syntology.ai/?focus=2302.03407","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2302.03407"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/vis-opt-group/sl-BAMM","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vis-opt-group/sl-bamm","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":3,"ran_draft_wrong":4,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"3ed1406329714c3f","entry":"cat_list_to_tensor","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"3ed1406329714c3f"}},{"code_sha256_prefix":"ccd4881b531f738e","entry":"cg","repo":"vis-opt-group/sl-bamm","repo_kind":"official","path":"hypergrad/CG_torch.py","file_url":"https://github.com/vis-opt-group/sl-bamm/blob/HEAD/hypergrad/CG_torch.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ccd4881b531f738e"}},{"code_sha256_prefix":"cd2cd81580a4b699","entry":"fixed_point","repo":"vis-opt-group/sl-bamm","repo_kind":"official","path":"hypergrad/hypergradients.py","file_url":"https://github.com/vis-opt-group/sl-bamm/blob/HEAD/hypergrad/hypergradients.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd2cd81580a4b699"}},{"code_sha256_prefix":"718723909e4de80c","entry":"get_outer_gradients","repo":"vis-opt-group/sl-bamm","repo_kind":"official","path":"hypergrad/hypergradients.py","file_url":"https://github.com/vis-opt-group/sl-bamm/blob/HEAD/hypergrad/hypergradients.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"718723909e4de80c"}},{"code_sha256_prefix":"1b9c1e465ef0029f","entry":"grad_unused_zero","repo":"vis-opt-group/sl-bamm","repo_kind":"official","path":"hypergrad/hypergradients.py","file_url":"https://github.com/vis-opt-group/sl-bamm/blob/HEAD/hypergrad/hypergradients.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1b9c1e465ef0029f"}},{"code_sha256_prefix":"452ac773b1804bc4","entry":"loss_L1","repo":"vis-opt-group/sl-bamm","repo_kind":"official","path":"merely_convex/strategy3.py","file_url":"https://github.com/vis-opt-group/sl-bamm/blob/HEAD/merely_convex/strategy3.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"452ac773b1804bc4"}},{"code_sha256_prefix":"1592bf271df76b62","entry":"loss_L2","repo":"vis-opt-group/sl-bamm","repo_kind":"official","path":"merely_convex/strategy3.py","file_url":"https://github.com/vis-opt-group/sl-bamm/blob/HEAD/merely_convex/strategy3.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1592bf271df76b62"}},{"code_sha256_prefix":"43e10b662538de12","entry":"update_tensor_grads","repo":"vis-opt-group/sl-bamm","repo_kind":"official","path":"hypergrad/hypergradients.py","file_url":"https://github.com/vis-opt-group/sl-bamm/blob/HEAD/hypergrad/hypergradients.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43e10b662538de12"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}