{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/flora-low-rank-core-space-for-n-dimension","title":"FLoRA: Low-Rank Core Space for N-dimension","arxiv_id":"2405.14739","date":"2024-05-23","proceeding":null,"authors":["Chongjie Si","Xuehui Wang","Xue Yang","Zhengqin Xu","Qingyun Li","Jifeng Dai","Yu Qiao","Xiaokang Yang","Wei Shen"],"abstract":"Adapting pre-trained foundation models for various downstream tasks has been prevalent in artificial intelligence. Due to the vast number of tasks and high costs, adjusting all parameters becomes unfeasible. To mitigate this, several fine-tuning techniques have been developed to update the pre-trained model weights in a more resource-efficient manner, such as through low-rank adjustments. Yet, almost all of these methods focus on linear weights, neglecting the intricacies of parameter spaces in higher dimensions like 4D. Alternatively, some methods can be adapted for high-dimensional parameter space by compressing changes in the original space into two dimensions and then employing low-rank matrix decomposition. However, these approaches destructs the structural integrity of the involved high-dimensional spaces. To tackle the diversity of dimensional spaces across different foundation models and provide a more precise representation of the changes within these spaces, this paper introduces a generalized parameter-efficient fine-tuning framework, FLoRA, designed for various dimensional parameter space. Specifically, utilizing Tucker decomposition, FLoRA asserts that changes in each dimensional parameter space are based on a low-rank core space which maintains the consistent topological structure with the original space. It then models the changes through this core space alongside corresponding weights to reconstruct alterations in the original space. FLoRA effectively preserves the structural integrity of the change of original N-dimensional parameter space, meanwhile decomposes it via low-rank tensor decomposition. Extensive experiments on computer vision, natural language processing and multi-modal tasks validate FLoRA's effectiveness. Codes are available at https://github.com/SJTU-DeepVisionLab/FLoRA.","url_abs":"https://arxiv.org/abs/2405.14739v1","url_pdf":"https://arxiv.org/pdf/2405.14739v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"flora-low-rank-core-space-for-n-dimension","repo_url":"https://github.com/sjtu-deepvisionlab/flora","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"tensor-decomposition","task_name":"Tensor Decomposition"},{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[{"method_slug":"focus","method_name":"Focus"},{"method_slug":"tucker","method_name":"TuckER"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.14739","atlas_url":"https://app.syntology.ai/?focus=2405.14739","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.14739"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sjtu-deepvisionlab/flora","reach":null}],"summary":{"ran":4,"unverified":4},"by_repo_kind":{"official":{"samples":8,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"a5f23dee099716f5","entry":"Conv2D","repo":"sjtu-deepvisionlab/flora","repo_kind":"official","path":"flora/flora.py","file_url":"https://github.com/sjtu-deepvisionlab/flora/blob/HEAD/flora/flora.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a5f23dee099716f5"}},{"code_sha256_prefix":"48519da9516c180c","entry":"Conv3D","repo":"sjtu-deepvisionlab/flora","repo_kind":"official","path":"flora/flora.py","file_url":"https://github.com/sjtu-deepvisionlab/flora/blob/HEAD/flora/flora.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"48519da9516c180c"}},{"code_sha256_prefix":"aaffa03c07ffd3cb","entry":"FLoRALayer","repo":"sjtu-deepvisionlab/flora","repo_kind":"official","path":"flora/flora.py","file_url":"https://github.com/sjtu-deepvisionlab/flora/blob/HEAD/flora/flora.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aaffa03c07ffd3cb"}},{"code_sha256_prefix":"544fc7c000e169b1","entry":"Linear","repo":"sjtu-deepvisionlab/flora","repo_kind":"official","path":"flora/flora.py","file_url":"https://github.com/sjtu-deepvisionlab/flora/blob/HEAD/flora/flora.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"544fc7c000e169b1"}},{"code_sha256_prefix":"5a62f6b034765d23","entry":"BaseFLoRA","repo":"sjtu-deepvisionlab/flora","repo_kind":"official","path":"flora/flora.py","file_url":"https://github.com/sjtu-deepvisionlab/flora/blob/HEAD/flora/flora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5a62f6b034765d23"}},{"code_sha256_prefix":"30cc2aeb15114c79","entry":"Embedding","repo":"sjtu-deepvisionlab/flora","repo_kind":"official","path":"flora/flora.py","file_url":"https://github.com/sjtu-deepvisionlab/flora/blob/HEAD/flora/flora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"30cc2aeb15114c79"}},{"code_sha256_prefix":"4b97335bd613d5e2","entry":"FLoRA","repo":"sjtu-deepvisionlab/flora","repo_kind":"official","path":"flora/flora.py","file_url":"https://github.com/sjtu-deepvisionlab/flora/blob/HEAD/flora/flora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4b97335bd613d5e2"}},{"code_sha256_prefix":"3626c2258def5d66","entry":"define_flora_block","repo":"sjtu-deepvisionlab/flora","repo_kind":"official","path":"flora/flora.py","file_url":"https://github.com/sjtu-deepvisionlab/flora/blob/HEAD/flora/flora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3626c2258def5d66"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}