{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/truncated-back-propagation-for-bilevel","title":"Truncated Back-propagation for Bilevel Optimization","arxiv_id":"1810.10667","date":"2018-10-25","proceeding":null,"authors":["Amirreza Shaban","Ching-An Cheng","Nathan Hatch","Byron Boots"],"abstract":"Bilevel optimization has been recently revisited for designing and analyzing\nalgorithms in hyperparameter tuning and meta learning tasks. However, due to\nits nested structure, evaluating exact gradients for high-dimensional problems\nis computationally challenging. One heuristic to circumvent this difficulty is\nto use the approximate gradient given by performing truncated back-propagation\nthrough the iterative optimization procedure that solves the lower-level\nproblem. Although promising empirical performance has been reported, its\ntheoretical properties are still unclear. In this paper, we analyze the\nproperties of this family of approximate gradients and establish sufficient\nconditions for convergence. We validate this on several hyperparameter tuning\nand meta learning tasks. We find that optimization with the approximate\ngradient computed using few-step back-propagation often performs comparably to\noptimization with the exact gradient, while requiring far less memory and half\nthe computation time.","url_abs":"http://arxiv.org/abs/1810.10667v2","url_pdf":"http://arxiv.org/pdf/1810.10667v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"truncated-back-propagation-for-bilevel","repo_url":"https://github.com/lucfra/FAR-HO","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"truncated-back-propagation-for-bilevel","repo_url":"https://github.com/nhatch/nhatch.github.io","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"bilevel-optimization","task_name":"Bilevel Optimization"},{"task_slug":"meta-learning","task_name":"Meta-Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1810.10667","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1810.10667"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nhatch/nhatch.github.io","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lucfra/FAR-HO","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e422c2d2070be100","entry":"as_tuple_or_list","repo":"lucfra/FAR-HO","repo_kind":"listed","path":"far_ho/utils.py","file_url":"https://github.com/lucfra/FAR-HO/blob/HEAD/far_ho/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e422c2d2070be100"}},{"code_sha256_prefix":"d7ba131401399f13","entry":"flatten_list","repo":"lucfra/FAR-HO","repo_kind":"listed","path":"far_ho/utils.py","file_url":"https://github.com/lucfra/FAR-HO/blob/HEAD/far_ho/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d7ba131401399f13"}},{"code_sha256_prefix":"89d864087a53ce95","entry":"merge_two_dicts","repo":"lucfra/FAR-HO","repo_kind":"listed","path":"far_ho/utils.py","file_url":"https://github.com/lucfra/FAR-HO/blob/HEAD/far_ho/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89d864087a53ce95"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}