{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2602-00722","title":"Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation","arxiv_id":"2602.00722","date":"2026-01-31","proceeding":"ICML","authors":["Hao Gu","Mao-Lin Luo","Zi-Hao Zhou","Han-Chen Zhang","Min-Ling Zhang","Tong Wei"],"abstract":"Parameter-efficient continual learning aims to adapt pre-trained models to sequential tasks without forgetting previously acquired knowledge. Most existing approaches treat continual learning as avoiding interference with past updates, rather than considering what properties make the current task-specific update naturally preserve previously acquired knowledge. From a knowledge-decomposition perspective, we observe that low-rank adaptations exhibit highly imbalanced singular value spectra: a few dominant components absorb most of the adaptation energy, thereby (i) more likely to disrupt previously acquired knowledge and (ii) making the update more vulnerable to interference from subsequent tasks. To enable explicit balance among components, we decouple the magnitude of the task update from its directional structure and formulate it as a constrained optimization problem on a restricted Stiefel manifold. We address this problem using a projected first-order method compatible with standard deep-learning optimizers used in vision-language models. Our method mitigates both backward and forward forgetting, consistently outperforming continual learning baselines. The implementation code is available at https://github.com/haodotgu/EBLoRA.","url_abs":"https://arxiv.org/abs/2602.00722","url_pdf":"https://arxiv.org/pdf/2602.00722","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2602.00722","atlas_url":"https://app.syntology.ai/?focus=2602.00722","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2602.00722"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/haodotgu/EBLoRA","reach":null}],"summary":{"ran":1,"ran_violates":4,"ran_honours":2,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f2240d43dba2b3d0","entry":"RiemannianAdamW","repo":"haodotgu/EBLoRA","repo_kind":"found_in_text","path":"llava/train/riemannian_adam.py","file_url":"https://github.com/haodotgu/EBLoRA/blob/HEAD/llava/train/riemannian_adam.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f2240d43dba2b3d0"}},{"code_sha256_prefix":"68db4636d010c513","entry":"euclidean2riemannian_U","repo":"haodotgu/EBLoRA","repo_kind":"found_in_text","path":"llava/train/riemannian_adam.py","file_url":"https://github.com/haodotgu/EBLoRA/blob/HEAD/llava/train/riemannian_adam.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"68db4636d010c513"}},{"code_sha256_prefix":"55350aadc942d740","entry":"euclidean2riemannian_Vt","repo":"haodotgu/EBLoRA","repo_kind":"found_in_text","path":"llava/train/riemannian_adam.py","file_url":"https://github.com/haodotgu/EBLoRA/blob/HEAD/llava/train/riemannian_adam.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"55350aadc942d740"}},{"code_sha256_prefix":"2a0933175e363abe","entry":"polar_retraction_U","repo":"haodotgu/EBLoRA","repo_kind":"found_in_text","path":"llava/train/riemannian_adam.py","file_url":"https://github.com/haodotgu/EBLoRA/blob/HEAD/llava/train/riemannian_adam.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2a0933175e363abe"}},{"code_sha256_prefix":"0921dc421f2d3073","entry":"polar_retraction_Vt","repo":"haodotgu/EBLoRA","repo_kind":"found_in_text","path":"llava/train/riemannian_adam.py","file_url":"https://github.com/haodotgu/EBLoRA/blob/HEAD/llava/train/riemannian_adam.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0921dc421f2d3073"}},{"code_sha256_prefix":"68db4da3e9514261","entry":"polar_uf","repo":"haodotgu/EBLoRA","repo_kind":"found_in_text","path":"llava/train/riemannian_adam.py","file_url":"https://github.com/haodotgu/EBLoRA/blob/HEAD/llava/train/riemannian_adam.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"68db4da3e9514261"}},{"code_sha256_prefix":"88022d42405baaea","entry":"tangent_project_U","repo":"haodotgu/EBLoRA","repo_kind":"found_in_text","path":"llava/train/riemannian_adam.py","file_url":"https://github.com/haodotgu/EBLoRA/blob/HEAD/llava/train/riemannian_adam.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"88022d42405baaea"}},{"code_sha256_prefix":"5b4bb8ce66457224","entry":"tangent_project_Vt","repo":"haodotgu/EBLoRA","repo_kind":"found_in_text","path":"llava/train/riemannian_adam.py","file_url":"https://github.com/haodotgu/EBLoRA/blob/HEAD/llava/train/riemannian_adam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5b4bb8ce66457224"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}