{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/relax-image-specific-prompt-requirement-in","title":"Relax Image-Specific Prompt Requirement in SAM: A Single Generic Prompt for Segmenting Camouflaged Objects","arxiv_id":"2312.07374","date":"2023-12-12","proceeding":null,"authors":["Jian Hu","Jiayi Lin","Weitong Cai","Shaogang Gong"],"abstract":"Camouflaged object detection (COD) approaches heavily rely on pixel-level annotated datasets. Weakly-supervised COD (WSCOD) approaches use sparse annotations like scribbles or points to reduce annotation effort, but this can lead to decreased accuracy. The Segment Anything Model (SAM) shows remarkable segmentation ability with sparse prompts like points. However, manual prompt is not always feasible, as it may not be accessible in real-world application. Additionally, it only provides localization information instead of semantic one, which can intrinsically cause ambiguity in interpreting the targets. In this work, we aim to eliminate the need for manual prompt. The key idea is to employ Cross-modal Chains of Thought Prompting (CCTP) to reason visual prompts using the semantic information given by a generic text prompt. To that end, we introduce a test-time adaptation per-instance mechanism called Generalizable SAM (GenSAM) to automatically enerate and optimize visual prompts the generic task prompt for WSCOD. In particular, CCTP maps a single generic text prompt onto image-specific consensus foreground and background heatmaps using vision-language models, acquiring reliable visual prompts. Moreover, to test-time adapt the visual prompts, we further propose Progressive Mask Generation (PMG) to iteratively reweight the input image, guiding the model to focus on the targets in a coarse-to-fine manner. Crucially, all network parameters are fixed, avoiding the need for additional training. Experiments demonstrate the superiority of GenSAM. Experiments on three benchmarks demonstrate that GenSAM outperforms point supervision approaches and achieves comparable results to scribble supervision ones, solely relying on general task descriptions as prompts. our codes is in: https://lwpyh.github.io/GenSAM/.","url_abs":"https://arxiv.org/abs/2312.07374v3","url_pdf":"https://arxiv.org/pdf/2312.07374v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"relax-image-specific-prompt-requirement-in","repo_url":"https://github.com/jyLin8100/GenSAM","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"camouflaged-object-segmentation-with-a-single","task_name":"Camouflaged Object Segmentation with a Single Task-generic Prompt"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"test-time-adaptation","task_name":"Test-time Adaptation"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"gensam","method_name":"GenSAM"},{"method_slug":"sam","method_name":"SAM"}],"datasets_introduced":[],"methods_introduced":[{"slug":"gensam","name":"GenSAM","full_name":"Generalizable SAM"}],"results":[{"leaderboard":"/sota/camouflaged-object-segmentation-with-a-single-1","task":"Camouflaged Object Segmentation with a Single Task-generic Prompt","dataset":"CAMO","model":"GenSAM","rank_in_archive_order":3,"of":3,"metrics":{"E_{\\phi}":"0.775","F_{\\beta}":"0.659","MAE":"0.113","S_{\\alpha}":"0.719"},"uses_additional_data":false},{"leaderboard":"/sota/camouflaged-object-segmentation-with-a-single-2","task":"Camouflaged Object Segmentation with a Single Task-generic Prompt","dataset":"COD10K","model":"GenSAM","rank_in_archive_order":3,"of":3,"metrics":{"E_{\\phi}":"0.838","F_{\\beta}":"0.681","MAE":"0.067","S_{\\alpha}":"0.775"},"uses_additional_data":false},{"leaderboard":"/sota/camouflaged-object-segmentation-with-a-single","task":"Camouflaged Object Segmentation with a Single Task-generic Prompt","dataset":"Chameleon","model":"GenSAM","rank_in_archive_order":2,"of":2,"metrics":{"E_{\\phi}":"0.807","F_{\\beta}":"0.68","MAE":"0.09","S_{\\alpha}":"0.764"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2312.07374","atlas_url":"https://app.syntology.ai/?focus=2312.07374","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.07374"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jyLin8100/GenSAM","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"ran":5,"unverified":5},"by_repo_kind":{"official":{"samples":13,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"98f385d847636a3e","entry":"basic_clean","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98f385d847636a3e"}},{"code_sha256_prefix":"8b40bb3f5347ccd5","entry":"compute_num_params","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"utils.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b40bb3f5347ccd5"}},{"code_sha256_prefix":"dfa1030744f2ab1a","entry":"fuse_mask","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"utils_model.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/utils_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dfa1030744f2ab1a"}},{"code_sha256_prefix":"d919ae32e5e4e616","entry":"get_pairs","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d919ae32e5e4e616"}},{"code_sha256_prefix":"7c448639360af3b4","entry":"resize_fn","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"datasets/wrappers.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/datasets/wrappers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7c448639360af3b4"}},{"code_sha256_prefix":"14982a5b13861f94","entry":"time_text","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"utils.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14982a5b13861f94"}},{"code_sha256_prefix":"68f0f6a323be1015","entry":"to_mask","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"datasets/wrappers.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/datasets/wrappers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68f0f6a323be1015"}},{"code_sha256_prefix":"9542161e9640b858","entry":"whitespace_clean","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9542161e9640b858"}},{"code_sha256_prefix":"1a165219e3845d5a","entry":"build_model","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"clip/build_model.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/clip/build_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a165219e3845d5a"}},{"code_sha256_prefix":"dc03e85a3091152d","entry":"load","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"clip/clip.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/clip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc03e85a3091152d"}},{"code_sha256_prefix":"7fdb65394809a6a8","entry":"make","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"datasets/datasets.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/datasets/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7fdb65394809a6a8"}},{"code_sha256_prefix":"9687bf7a922f098a","entry":"register","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"datasets/datasets.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/datasets/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9687bf7a922f098a"}},{"code_sha256_prefix":"7ab8ae87f1f425de","entry":"set_save_path","repo":"jyLin8100/GenSAM","repo_kind":"official","path":"utils.py","file_url":"https://github.com/jyLin8100/GenSAM/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ab8ae87f1f425de"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}