{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/optimizer-amalgamation-1","title":"Optimizer Amalgamation","arxiv_id":"2203.06474","date":"2022-03-12","proceeding":"ICLR 2022 4","authors":["Tianshu Huang","Tianlong Chen","Sijia Liu","Shiyu Chang","Lisa Amini","Zhangyang Wang"],"abstract":"Selecting an appropriate optimizer for a given problem is of major interest for researchers and practitioners. Many analytical optimizers have been proposed using a variety of theoretical and empirical approaches; however, none can offer a universal advantage over other competitive optimizers. We are thus motivated to study a new problem named Optimizer Amalgamation: how can we best combine a pool of \"teacher\" optimizers into a single \"student\" optimizer that can have stronger problem-specific performance? In this paper, we draw inspiration from the field of \"learning to optimize\" to use a learnable amalgamation target. First, we define three differentiable amalgamation mechanisms to amalgamate a pool of analytical optimizers by gradient descent. Then, in order to reduce variance of the amalgamation process, we also explore methods to stabilize the amalgamation process by perturbing the amalgamation target. Finally, we present experiments showing the superiority of our amalgamated optimizer compared to its amalgamated components and learning to optimize baselines, and the efficacy of our variance reducing perturbations. Our code and pre-trained models are publicly available at http://github.com/VITA-Group/OptimizerAmalgamation.","url_abs":"https://arxiv.org/abs/2203.06474v2","url_pdf":"https://arxiv.org/pdf/2203.06474v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"optimizer-amalgamation-1","repo_url":"https://github.com/vita-group/optimizeramalgamation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.06474","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.06474"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/VITA-Group/OptimizerAmalgamation","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vita-group/optimizeramalgamation","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":7,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"82ca546c2aece10b","entry":"BaseResult","repo":"vita-group/optimizeramalgamation","repo_kind":"official","path":"plot/strategy.py","file_url":"https://github.com/vita-group/optimizeramalgamation/blob/HEAD/plot/strategy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82ca546c2aece10b"}},{"code_sha256_prefix":"996638765da81b5f","entry":"CurriculumResult","repo":"vita-group/optimizeramalgamation","repo_kind":"official","path":"plot/strategy.py","file_url":"https://github.com/vita-group/optimizeramalgamation/blob/HEAD/plot/strategy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"996638765da81b5f"}},{"code_sha256_prefix":"75b0d8f04509767c","entry":"RepeatResult","repo":"vita-group/optimizeramalgamation","repo_kind":"official","path":"plot/strategy.py","file_url":"https://github.com/vita-group/optimizeramalgamation/blob/HEAD/plot/strategy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75b0d8f04509767c"}},{"code_sha256_prefix":"9c4d0eeaadec8ea7","entry":"_read_json","repo":"vita-group/optimizeramalgamation","repo_kind":"official","path":"plot/strategy.py","file_url":"https://github.com/vita-group/optimizeramalgamation/blob/HEAD/plot/strategy.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c4d0eeaadec8ea7"}},{"code_sha256_prefix":"93f1552a3a329ce8","entry":"ReplicateResults","repo":"vita-group/optimizeramalgamation","repo_kind":"official","path":"plot/strategy.py","file_url":"https://github.com/vita-group/optimizeramalgamation/blob/HEAD/plot/strategy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93f1552a3a329ce8"}},{"code_sha256_prefix":"660595544730cf2b","entry":"_npload","repo":"vita-group/optimizeramalgamation","repo_kind":"official","path":"plot/strategy.py","file_url":"https://github.com/vita-group/optimizeramalgamation/blob/HEAD/plot/strategy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"660595544730cf2b"}},{"code_sha256_prefix":"1bd6c21a92c0ab8c","entry":"get_container","repo":"vita-group/optimizeramalgamation","repo_kind":"official","path":"plot/strategy.py","file_url":"https://github.com/vita-group/optimizeramalgamation/blob/HEAD/plot/strategy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1bd6c21a92c0ab8c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}