{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/asynchronous-distributed-bilevel-optimization","title":"Asynchronous Distributed Bilevel Optimization","arxiv_id":"2212.10048","date":"2022-12-20","proceeding":null,"authors":["Yang Jiao","Kai Yang","Tiancheng Wu","Dongjin Song","Chengtao Jian"],"abstract":"Bilevel optimization plays an essential role in many machine learning tasks, ranging from hyperparameter optimization to meta-learning. Existing studies on bilevel optimization, however, focus on either centralized or synchronous distributed setting. The centralized bilevel optimization approaches require collecting massive amount of data to a single server, which inevitably incur significant communication expenses and may give rise to data privacy risks. Synchronous distributed bilevel optimization algorithms, on the other hand, often face the straggler problem and will immediately stop working if a few workers fail to respond. As a remedy, we propose Asynchronous Distributed Bilevel Optimization (ADBO) algorithm. The proposed ADBO can tackle bilevel optimization problems with both nonconvex upper-level and lower-level objective functions, and its convergence is theoretically guaranteed. Furthermore, it is revealed through theoretic analysis that the iteration complexity of ADBO to obtain the $\\epsilon$-stationary point is upper bounded by $\\mathcal{O}(\\frac{1}{{{\\epsilon ^2}}})$. Thorough empirical studies on public datasets have been conducted to elucidate the effectiveness and efficiency of the proposed ADBO.","url_abs":"https://arxiv.org/abs/2212.10048v3","url_pdf":"https://arxiv.org/pdf/2212.10048v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"asynchronous-distributed-bilevel-optimization","repo_url":"https://github.com/iclr23submission6251/adbo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"bilevel-optimization","task_name":"Bilevel Optimization"},{"task_slug":"hyperparameter-optimization","task_name":"Hyperparameter Optimization"},{"task_slug":"meta-learning","task_name":"Meta-Learning"}],"methods":[{"method_slug":"fail","method_name":"fail"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2212.10048","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.10048"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/iclr23submission6251/adbo","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bb279b17181bca6a","entry":"ecd","repo":"iclr23submission6251/adbo","repo_kind":"official","path":"hyperclean-mnist/cp-main-async/cp-main-async.py","file_url":"https://github.com/iclr23submission6251/adbo/blob/HEAD/hyperclean-mnist/cp-main-async/cp-main-async.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bb279b17181bca6a"}},{"code_sha256_prefix":"d4dd97c3a435df86","entry":"dcd","repo":"iclr23submission6251/adbo","repo_kind":"official","path":"hyperclean-mnist/cp-main-async/cp-main-async.py","file_url":"https://github.com/iclr23submission6251/adbo/blob/HEAD/hyperclean-mnist/cp-main-async/cp-main-async.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d4dd97c3a435df86"}},{"code_sha256_prefix":"6da8ccf488ef51b1","entry":"pack_tensors","repo":"iclr23submission6251/adbo","repo_kind":"official","path":"hyperclean-mnist/cp-main-async/cp-main-async.py","file_url":"https://github.com/iclr23submission6251/adbo/blob/HEAD/hyperclean-mnist/cp-main-async/cp-main-async.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6da8ccf488ef51b1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}