{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-29108","title":"Efficient Bilevel Optimization with KFAC-Based Hypergradients","arxiv_id":"2603.29108","date":"2026-03-31","proceeding":null,"authors":["Disen Liao","Felix Dangel","Yaoliang Yu"],"abstract":"Bilevel optimization (BO) is widely applicable to many machine learning problems. Scaling BO, however, requires repeatedly computing hypergradients, which involves solving inverse Hessian-vector products (IHVPs). In practice, these operations are often approximated using crude surrogates such as one-step gradient unrolling or identity/short Neumann expansions, which discard curvature information. We build on implicit function theorem-based algorithms and propose to incorporate Kronecker-factored approximate curvature (KFAC), yielding curvature-aware hypergradients with a better performance efficiency trade-off than Conjugate Gradient (CG) or Neumann methods and consistently outperforming unrolling. We evaluate this approach across diverse tasks, including meta-learning and AI safety problems. On models up to BERT, we show that curvature information is valuable at scale, and KFAC can provide it with only modest memory and runtime overhead. Our implementation is available at https://github.com/liaodisen/NeuralBo.","url_abs":"https://arxiv.org/abs/2603.29108","url_pdf":"https://arxiv.org/pdf/2603.29108","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2603.29108","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.29108"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Cranial-XIX/BOME","reach":{"status":"ok"}},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/xjtushujun/Meta-weight-net_class-imbalance","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":4,"ran":2,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"8b3ab75b3af344bf","entry":"exact","repo":"Cranial-XIX/BOME","repo_kind":"found_in_text","path":"hpo/hypergrad/hypergradients.py","file_url":"https://github.com/Cranial-XIX/BOME/blob/HEAD/hpo/hypergrad/hypergradients.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8b3ab75b3af344bf"}},{"code_sha256_prefix":"a2f6a25906e664c5","entry":"to_var","repo":"xjtushujun/Meta-weight-net_class-imbalance","repo_kind":"found_in_text","path":"resnet.py","file_url":"https://github.com/xjtushujun/Meta-weight-net_class-imbalance/blob/HEAD/resnet.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a2f6a25906e664c5"}},{"code_sha256_prefix":"8f873dc2f47b8f1c","entry":"build_dataset","repo":"xjtushujun/Meta-weight-net_class-imbalance","repo_kind":"found_in_text","path":"data_utils.py","file_url":"https://github.com/xjtushujun/Meta-weight-net_class-imbalance/blob/HEAD/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8f873dc2f47b8f1c"}},{"code_sha256_prefix":"cae4b2624bd8ac72","entry":"get_img_num_per_cls","repo":"xjtushujun/Meta-weight-net_class-imbalance","repo_kind":"found_in_text","path":"data_utils.py","file_url":"https://github.com/xjtushujun/Meta-weight-net_class-imbalance/blob/HEAD/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cae4b2624bd8ac72"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}