{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/inflora-interference-free-low-rank-adaptation","title":"InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning","arxiv_id":"2404.00228","date":"2024-03-30","proceeding":"CVPR 2024 1","authors":["Yan-Shuo Liang","Wu-Jun Li"],"abstract":"Continual learning requires the model to learn multiple tasks sequentially. In continual learning, the model should possess the ability to maintain its performance on old tasks (stability) and the ability to adapt to new tasks continuously (plasticity). Recently, parameter-efficient fine-tuning (PEFT), which involves freezing a pre-trained model and injecting a small number of learnable parameters to adapt to downstream tasks, has gained increasing popularity in continual learning. Although existing continual learning methods based on PEFT have demonstrated superior performance compared to those not based on PEFT, most of them do not consider how to eliminate the interference of the new task on the old tasks, which inhibits the model from making a good trade-off between stability and plasticity. In this work, we propose a new PEFT method, called interference-free low-rank adaptation (InfLoRA), for continual learning. InfLoRA injects a small number of parameters to reparameterize the pre-trained weights and shows that fine-tuning these injected parameters is equivalent to fine-tuning the pre-trained weights within a subspace. Furthermore, InfLoRA designs this subspace to eliminate the interference of the new task on the old tasks, making a good trade-off between stability and plasticity. Experimental results show that InfLoRA outperforms existing state-of-the-art continual learning methods on multiple datasets.","url_abs":"https://arxiv.org/abs/2404.00228v3","url_pdf":"https://arxiv.org/pdf/2404.00228v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"inflora-interference-free-low-rank-adaptation","repo_url":"https://github.com/liangyanshuo/InfLoRA","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2404.00228","atlas_url":"https://app.syntology.ai/?focus=2404.00228","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.00228"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/liangyanshuo/InfLoRA","reach":null}],"summary":{"ran":1,"ran_draft_wrong":1,"ran_fixture":1,"unverified":7},"by_repo_kind":{"official":{"samples":10,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7fc74661bb506da7","entry":"Attention_LoRA","repo":"liangyanshuo/InfLoRA","repo_kind":"official","path":"methods/inflora.py","file_url":"https://github.com/liangyanshuo/InfLoRA/blob/HEAD/methods/inflora.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7fc74661bb506da7"}},{"code_sha256_prefix":"4eeb738096393d87","entry":"checkpoint_filter_fn","repo":"liangyanshuo/InfLoRA","repo_kind":"official","path":"methods/inflora.py","file_url":"https://github.com/liangyanshuo/InfLoRA/blob/HEAD/methods/inflora.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4eeb738096393d87"}},{"code_sha256_prefix":"19b12af6590323f6","entry":"resize_pos_embed","repo":"liangyanshuo/InfLoRA","repo_kind":"official","path":"methods/inflora.py","file_url":"https://github.com/liangyanshuo/InfLoRA/blob/HEAD/methods/inflora.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"19b12af6590323f6"}},{"code_sha256_prefix":"222edc61eeca890a","entry":"BaseLearner","repo":"liangyanshuo/InfLoRA","repo_kind":"official","path":"methods/inflora.py","file_url":"https://github.com/liangyanshuo/InfLoRA/blob/HEAD/methods/inflora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"222edc61eeca890a"}},{"code_sha256_prefix":"47dda0c70839da58","entry":"Block","repo":"liangyanshuo/InfLoRA","repo_kind":"official","path":"methods/inflora.py","file_url":"https://github.com/liangyanshuo/InfLoRA/blob/HEAD/methods/inflora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"47dda0c70839da58"}},{"code_sha256_prefix":"1b3bf1e887a758b1","entry":"InfLoRA","repo":"liangyanshuo/InfLoRA","repo_kind":"official","path":"methods/inflora.py","file_url":"https://github.com/liangyanshuo/InfLoRA/blob/HEAD/methods/inflora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1b3bf1e887a758b1"}},{"code_sha256_prefix":"ecad0e19092b3a07","entry":"SiNet","repo":"liangyanshuo/InfLoRA","repo_kind":"official","path":"methods/inflora.py","file_url":"https://github.com/liangyanshuo/InfLoRA/blob/HEAD/methods/inflora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ecad0e19092b3a07"}},{"code_sha256_prefix":"84a2cb412dfdc495","entry":"ViT_lora_co","repo":"liangyanshuo/InfLoRA","repo_kind":"official","path":"methods/inflora.py","file_url":"https://github.com/liangyanshuo/InfLoRA/blob/HEAD/methods/inflora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"84a2cb412dfdc495"}},{"code_sha256_prefix":"20b9b6a55783a6e5","entry":"_create_vision_transformer","repo":"liangyanshuo/InfLoRA","repo_kind":"official","path":"methods/inflora.py","file_url":"https://github.com/liangyanshuo/InfLoRA/blob/HEAD/methods/inflora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"20b9b6a55783a6e5"}},{"code_sha256_prefix":"1753ff7e1c2b9442","entry":"_load_weights","repo":"liangyanshuo/InfLoRA","repo_kind":"official","path":"methods/inflora.py","file_url":"https://github.com/liangyanshuo/InfLoRA/blob/HEAD/methods/inflora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1753ff7e1c2b9442"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}