{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/densely-connected-parameter-efficient-tuning","title":"Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation","arxiv_id":"2501.08580","date":"2025-01-15","proceeding":null,"authors":["Jiaqi Huang","Zunnan Xu","Ting Liu","Yong liu","Haonan Han","Kehong Yuan","Xiu Li"],"abstract":"In the domain of computer vision, Parameter-Efficient Tuning (PET) is increasingly replacing the traditional paradigm of pre-training followed by full fine-tuning. PET is particularly favored for its effectiveness in large foundation models, as it streamlines transfer learning costs and optimizes hardware utilization. However, the current PET methods are mainly designed for single-modal optimization. While some pioneering studies have undertaken preliminary explorations, they still remain at the level of aligned encoders (e.g., CLIP) and lack exploration of misaligned encoders. These methods show sub-optimal performance with misaligned encoders, as they fail to effectively align the multimodal features during fine-tuning. In this paper, we introduce DETRIS, a parameter-efficient tuning framework designed to enhance low-rank visual feature propagation by establishing dense interconnections between each layer and all preceding layers, which enables effective cross-modal feature interaction and adaptation to misaligned encoders. We also suggest using text adapters to improve textual features. Our simple yet efficient approach greatly surpasses state-of-the-art methods with 0.9% to 1.8% backbone parameter updates, evaluated on challenging benchmarks. Our project is available at \\url{https://github.com/jiaqihuang01/DETRIS}.","url_abs":"https://arxiv.org/abs/2501.08580v1","url_pdf":"https://arxiv.org/pdf/2501.08580v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"densely-connected-parameter-efficient-tuning","repo_url":"https://github.com/jiaqihuang01/detris","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"referring-expression-segmentation","task_name":"Referring Expression Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/referring-expression-segmentation-on-refcoco-6","task":"Referring Expression Segmentation","dataset":"RefCOCO","model":"DETRIS","rank_in_archive_order":1,"of":4,"metrics":{"IoU":"81.0"},"uses_additional_data":false},{"leaderboard":"/sota/referring-expression-segmentation-on-refcoco-8","task":"Referring Expression Segmentation","dataset":"RefCOCO testA","model":"DETRIS","rank_in_archive_order":6,"of":13,"metrics":{"Overall IoU":"81.9"},"uses_additional_data":false},{"leaderboard":"/sota/referring-expression-segmentation-on-refcoco-9","task":"Referring Expression Segmentation","dataset":"RefCOCO testB","model":"DETRIS","rank_in_archive_order":5,"of":13,"metrics":{"Overall IoU":"79.0"},"uses_additional_data":false},{"leaderboard":"/sota/referring-expression-segmentation-on-refcoco-5","task":"Referring Expression Segmentation","dataset":"RefCOCO+ test B","model":"DETRIS","rank_in_archive_order":4,"of":30,"metrics":{"Overall IoU":"70.2"},"uses_additional_data":false},{"leaderboard":"/sota/referring-expression-segmentation-on-refcoco-4","task":"Referring Expression Segmentation","dataset":"RefCOCO+ testA","model":"DETRIS","rank_in_archive_order":5,"of":30,"metrics":{"Overall IoU":"78.6"},"uses_additional_data":false},{"leaderboard":"/sota/referring-expression-segmentation-on-refcoco-3","task":"Referring Expression Segmentation","dataset":"RefCOCO+ val","model":"DETRIS","rank_in_archive_order":5,"of":33,"metrics":{"Overall IoU":"75.2"},"uses_additional_data":false},{"leaderboard":"/sota/referring-expression-segmentation-on-refcocog-1","task":"Referring Expression Segmentation","dataset":"RefCOCOg-test","model":"DETRIS","rank_in_archive_order":7,"of":18,"metrics":{"Overall IoU":"75.3"},"uses_additional_data":false},{"leaderboard":"/sota/referring-expression-segmentation-on-refcocog","task":"Referring Expression Segmentation","dataset":"RefCOCOg-val","model":"DETRIS","rank_in_archive_order":7,"of":23,"metrics":{"Overall IoU":"74.6"},"uses_additional_data":false},{"leaderboard":"/sota/referring-expression-segmentation-on-refcoco","task":"Referring Expression Segmentation","dataset":"RefCoCo val","model":"DETRIS","rank_in_archive_order":9,"of":37,"metrics":{"Overall IoU":"81.0"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2501.08580","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.08580"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiaqihuang01/detris","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":4,"ran":3,"unverified":10},"by_repo_kind":{"official":{"samples":17,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"98f385d847636a3e","entry":"basic_clean","repo":"jiaqihuang01/detris","repo_kind":"official","path":"utils/simple_tokenizer.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/utils/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98f385d847636a3e"}},{"code_sha256_prefix":"dbfe3357dc25ca5f","entry":"conv_layer","repo":"jiaqihuang01/detris","repo_kind":"official","path":"model/layers.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/model/layers.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dbfe3357dc25ca5f"}},{"code_sha256_prefix":"d919ae32e5e4e616","entry":"get_pairs","repo":"jiaqihuang01/detris","repo_kind":"official","path":"utils/simple_tokenizer.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/utils/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d919ae32e5e4e616"}},{"code_sha256_prefix":"539cbfa1328eb51b","entry":"init_random_seed","repo":"jiaqihuang01/detris","repo_kind":"official","path":"utils/misc.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/utils/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"539cbfa1328eb51b"}},{"code_sha256_prefix":"89f9fd49ddec76ba","entry":"linear_layer","repo":"jiaqihuang01/detris","repo_kind":"official","path":"model/layers.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/model/layers.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89f9fd49ddec76ba"}},{"code_sha256_prefix":"78241a45492d6667","entry":"trainMetricGPU","repo":"jiaqihuang01/detris","repo_kind":"official","path":"utils/misc.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/utils/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78241a45492d6667"}},{"code_sha256_prefix":"9542161e9640b858","entry":"whitespace_clean","repo":"jiaqihuang01/detris","repo_kind":"official","path":"utils/simple_tokenizer.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/utils/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9542161e9640b858"}},{"code_sha256_prefix":"98ab2f374bc768a9","entry":"adapt_position_encoding","repo":"jiaqihuang01/detris","repo_kind":"official","path":"model/clip.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/model/clip.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98ab2f374bc768a9"}},{"code_sha256_prefix":"5d16d4d4fc573ac7","entry":"add_residual","repo":"jiaqihuang01/detris","repo_kind":"official","path":"model/dinov2/layers/block.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/model/dinov2/layers/block.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5d16d4d4fc573ac7"}},{"code_sha256_prefix":"fd39a00b70046a6a","entry":"build_model","repo":"jiaqihuang01/detris","repo_kind":"official","path":"model/clip.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/model/clip.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fd39a00b70046a6a"}},{"code_sha256_prefix":"73cecca9f3575f09","entry":"concat_all_gather","repo":"jiaqihuang01/detris","repo_kind":"official","path":"utils/misc.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/utils/misc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"73cecca9f3575f09"}},{"code_sha256_prefix":"db55fb2ff4ac93bb","entry":"deconv_layer","repo":"jiaqihuang01/detris","repo_kind":"official","path":"model/layers.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/model/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db55fb2ff4ac93bb"}},{"code_sha256_prefix":"85f7ffc01945bb72","entry":"drop_add_residual_stochastic_depth","repo":"jiaqihuang01/detris","repo_kind":"official","path":"model/dinov2/layers/block.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/model/dinov2/layers/block.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"85f7ffc01945bb72"}},{"code_sha256_prefix":"5be3610fa1fee19e","entry":"get_branges_scales","repo":"jiaqihuang01/detris","repo_kind":"official","path":"model/dinov2/layers/block.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/model/dinov2/layers/block.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5be3610fa1fee19e"}},{"code_sha256_prefix":"cc23416cff396c39","entry":"load_cfg_from_cfg_file","repo":"jiaqihuang01/detris","repo_kind":"official","path":"utils/config.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/utils/config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc23416cff396c39"}},{"code_sha256_prefix":"b4c154b6f0568b6b","entry":"loads_pyarrow","repo":"jiaqihuang01/detris","repo_kind":"official","path":"utils/dataset.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/utils/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b4c154b6f0568b6b"}},{"code_sha256_prefix":"022883c7911679b3","entry":"merge_cfg_from_list","repo":"jiaqihuang01/detris","repo_kind":"official","path":"utils/config.py","file_url":"https://github.com/jiaqihuang01/detris/blob/HEAD/utils/config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"022883c7911679b3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}