{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bome-bilevel-optimization-made-easy-a-simple","title":"BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach","arxiv_id":"2209.08709","date":"2022-09-19","proceeding":null,"authors":["Mao Ye","Bo Liu","Stephen Wright","Peter Stone","Qiang Liu"],"abstract":"Bilevel optimization (BO) is useful for solving a variety of important machine learning problems including but not limited to hyperparameter optimization, meta-learning, continual learning, and reinforcement learning. Conventional BO methods need to differentiate through the low-level optimization process with implicit differentiation, which requires expensive calculations related to the Hessian matrix. There has been a recent quest for first-order methods for BO, but the methods proposed to date tend to be complicated and impractical for large-scale deep learning applications. In this work, we propose a simple first-order BO algorithm that depends only on first-order gradient information, requires no implicit differentiation, and is practical and efficient for large-scale non-convex functions in deep learning. We provide non-asymptotic convergence analysis of the proposed method to stationary points for non-convex objectives and present empirical results that show its superior practical performance.","url_abs":"https://arxiv.org/abs/2209.08709v1","url_pdf":"https://arxiv.org/pdf/2209.08709v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bome-bilevel-optimization-made-easy-a-simple","repo_url":"https://github.com/cranial-xix/bome","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"bilevel-optimization","task_name":"Bilevel Optimization"},{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"hyperparameter-optimization","task_name":"Hyperparameter Optimization"},{"task_slug":"meta-learning","task_name":"Meta-Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2209.08709","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}