{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cat-sam-conditional-tuning-network-for-few","title":"CAT-SAM: Conditional Tuning for Few-Shot Adaptation of Segment Anything Model","arxiv_id":"2402.03631","date":"2024-02-06","proceeding":null,"authors":["Aoran Xiao","Weihao Xuan","Heli Qi","Yun Xing","Ruijie Ren","Xiaoqin Zhang","Ling Shao","Shijian Lu"],"abstract":"The recent Segment Anything Model (SAM) has demonstrated remarkable zero-shot capability and flexible geometric prompting in general image segmentation. However, SAM often struggles when handling various unconventional images, such as aerial, medical, and non-RGB images. This paper presents CAT-SAM, a ConditionAl Tuning network that adapts SAM toward various unconventional target tasks with just few-shot target samples. CAT-SAM freezes the entire SAM and adapts its mask decoder and image encoder simultaneously with a small number of learnable parameters. The core design is a prompt bridge structure that enables decoder-conditioned joint tuning of the heavyweight image encoder and the lightweight mask decoder. The bridging maps the prompt token of the mask decoder to the image encoder, fostering synergic adaptation of the encoder and the decoder with mutual benefits. We develop two representative tuning strategies for the image encoder which leads to two CAT-SAM variants: one injecting learnable prompt tokens in the input space and the other inserting lightweight adapter networks. Extensive experiments over 11 unconventional tasks show that both CAT-SAM variants achieve superior target segmentation performance consistently even under the very challenging one-shot adaptation setup. Project page: https://xiaoaoran.github.io/projects/CAT-SAM","url_abs":"https://arxiv.org/abs/2402.03631v3","url_pdf":"https://arxiv.org/pdf/2402.03631v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cat-sam-conditional-tuning-network-for-few","repo_url":"https://github.com/weihao1115/cat-sam","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"adapter","method_name":"Adapter"},{"method_slug":"sam","method_name":"SAM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.03631","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.03631"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/weihao1115/cat-sam","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"unverified":4},"by_repo_kind":{"official":{"samples":9,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ab8c035313260667","entry":"find_objects_from_mask","repo":"weihao1115/cat-sam","repo_kind":"official","path":"cat_sam/datasets/misc.py","file_url":"https://github.com/weihao1115/cat-sam/blob/HEAD/cat_sam/datasets/misc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab8c035313260667"}},{"code_sha256_prefix":"249675ff48bbd428","entry":"find_random_points_in_objects","repo":"weihao1115/cat-sam","repo_kind":"official","path":"cat_sam/datasets/misc.py","file_url":"https://github.com/weihao1115/cat-sam/blob/HEAD/cat_sam/datasets/misc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"249675ff48bbd428"}},{"code_sha256_prefix":"da50a8d659217d5d","entry":"get_json_dict_from_dir","repo":"weihao1115/cat-sam","repo_kind":"official","path":"cat_sam/datasets/misc.py","file_url":"https://github.com/weihao1115/cat-sam/blob/HEAD/cat_sam/datasets/misc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"da50a8d659217d5d"}},{"code_sha256_prefix":"fb3b97e28b328e7c","entry":"to_numpy","repo":"weihao1115/cat-sam","repo_kind":"official","path":"cat_sam/utils/evaluators.py","file_url":"https://github.com/weihao1115/cat-sam/blob/HEAD/cat_sam/utils/evaluators.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fb3b97e28b328e7c"}},{"code_sha256_prefix":"9c42c0260c3c9229","entry":"trunc_normal_","repo":"weihao1115/cat-sam","repo_kind":"official","path":"cat_sam/models/module_lib.py","file_url":"https://github.com/weihao1115/cat-sam/blob/HEAD/cat_sam/models/module_lib.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c42c0260c3c9229"}},{"code_sha256_prefix":"f9bc5f31ce61cee8","entry":"build_sam_vit_b","repo":"weihao1115/cat-sam","repo_kind":"official","path":"cat_sam/models/segment_anything_ext/build_sam.py","file_url":"https://github.com/weihao1115/cat-sam/blob/HEAD/cat_sam/models/segment_anything_ext/build_sam.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9bc5f31ce61cee8"}},{"code_sha256_prefix":"6b77c1f11fff3ed6","entry":"build_sam_vit_h","repo":"weihao1115/cat-sam","repo_kind":"official","path":"cat_sam/models/segment_anything_ext/build_sam.py","file_url":"https://github.com/weihao1115/cat-sam/blob/HEAD/cat_sam/models/segment_anything_ext/build_sam.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b77c1f11fff3ed6"}},{"code_sha256_prefix":"3f8890e695469246","entry":"build_sam_vit_l","repo":"weihao1115/cat-sam","repo_kind":"official","path":"cat_sam/models/segment_anything_ext/build_sam.py","file_url":"https://github.com/weihao1115/cat-sam/blob/HEAD/cat_sam/models/segment_anything_ext/build_sam.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3f8890e695469246"}},{"code_sha256_prefix":"87f16a36e0c0923b","entry":"to_2tuple","repo":"weihao1115/cat-sam","repo_kind":"official","path":"cat_sam/models/module_lib.py","file_url":"https://github.com/weihao1115/cat-sam/blob/HEAD/cat_sam/models/module_lib.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"87f16a36e0c0923b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}