{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ether-efficient-finetuning-of-large-scale","title":"ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections","arxiv_id":"2405.20271","date":"2024-05-30","proceeding":null,"authors":["Massimo Bini","Karsten Roth","Zeynep Akata","Anna Khoreva"],"abstract":"Parameter-efficient finetuning (PEFT) has become ubiquitous to adapt foundation models to downstream task requirements while retaining their generalization ability. However, the amount of additionally introduced parameters and compute for successful adaptation and hyperparameter searches can explode quickly, especially when deployed at scale to serve numerous individual requests. To ensure effective, parameter-efficient, and hyperparameter-robust adaptation, we propose the ETHER transformation family, which performs Efficient fineTuning via HypErplane Reflections. By design, ETHER transformations require a minimal number of parameters, are less likely to deteriorate model performance, and exhibit robustness to hyperparameter and learning rate choices. In particular, we introduce ETHER and its relaxation ETHER+, which match or outperform existing PEFT methods with significantly fewer parameters ($\\sim$$10$-$100$ times lower than LoRA or OFT) across multiple image synthesis and natural language tasks without exhaustive hyperparameter tuning. Finally, we investigate the recent emphasis on Hyperspherical Energy retention for adaptation and raise questions on its practical utility. The code is available at https://github.com/mwbini/ether.","url_abs":"https://arxiv.org/abs/2405.20271v2","url_pdf":"https://arxiv.org/pdf/2405.20271v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ether-efficient-finetuning-of-large-scale","repo_url":"https://github.com/mwbini/ether","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.20271","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.20271"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/mwbini/ether","reach":null}],"summary":{"ran":2,"ran_violates":1,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f2ca5c994fea9c12","entry":"ETHERLayer","repo":"mwbini/ether","repo_kind":"official","path":"ether-instruct/lit_gpt/ether.py","file_url":"https://github.com/mwbini/ether/blob/HEAD/ether-instruct/lit_gpt/ether.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f2ca5c994fea9c12"}},{"code_sha256_prefix":"9189460045ab03f9","entry":"ETHERLinear","repo":"mwbini/ether","repo_kind":"official","path":"ether-instruct/lit_gpt/ether.py","file_url":"https://github.com/mwbini/ether/blob/HEAD/ether-instruct/lit_gpt/ether.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9189460045ab03f9"}},{"code_sha256_prefix":"fe9ea76c40408be5","entry":"ether_filter","repo":"mwbini/ether","repo_kind":"official","path":"ether-instruct/lit_gpt/ether.py","file_url":"https://github.com/mwbini/ether/blob/HEAD/ether-instruct/lit_gpt/ether.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe9ea76c40408be5"}},{"code_sha256_prefix":"1416cca657944035","entry":"get_lr_scheduler","repo":"mwbini/ether","repo_kind":"official","path":"ether-instruct/finetune/ether.py","file_url":"https://github.com/mwbini/ether/blob/HEAD/ether-instruct/finetune/ether.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1416cca657944035"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}