{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/frozen-clip-a-strong-backbone-for-weakly-1","title":"Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentation","arxiv_id":"2406.11189","date":"2024-06-17","proceeding":"CVPR 2024 1","authors":["Bingfeng Zhang","Siyue Yu","Yunchao Wei","Yao Zhao","Jimin Xiao"],"abstract":"Weakly supervised semantic segmentation has witnessed great achievements with image-level labels. Several recent approaches use the CLIP model to generate pseudo labels for training an individual segmentation model, while there is no attempt to apply the CLIP model as the backbone to directly segment objects with image-level labels. In this paper, we propose WeCLIP, a CLIP-based single-stage pipeline, for weakly supervised semantic segmentation. Specifically, the frozen CLIP model is applied as the backbone for semantic feature extraction, and a new decoder is designed to interpret extracted semantic features for final prediction. Meanwhile, we utilize the above frozen backbone to generate pseudo labels for training the decoder. Such labels cannot be optimized during training. We then propose a refinement module (RFM) to rectify them dynamically. Our architecture enforces the proposed decoder and RFM to benefit from each other to boost the final performance. Extensive experiments show that our approach significantly outperforms other approaches with less training cost. Additionally, our WeCLIP also obtains promising results for fully supervised settings. The code is available at https://github.com/zbf1991/WeCLIP.","url_abs":"https://arxiv.org/abs/2406.11189v1","url_pdf":"https://arxiv.org/pdf/2406.11189v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"frozen-clip-a-strong-backbone-for-weakly-1","repo_url":"https://github.com/zbf1991/weclip","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"weakly-supervised-semantic-segmentation-1","task_name":"Weakly supervised Semantic Segmentation"},{"task_slug":"weakly-supervised-semantic-segmentation","task_name":"Weakly-Supervised Semantic Segmentation"}],"methods":[{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2406.11189","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.11189"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zbf1991/weclip","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/zbf1991/WeCLIP","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"unverified":8},"by_repo_kind":{"official":{"samples":9,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"5bd180a934a83565","entry":"reshape_transform","repo":"zbf1991/WeCLIP","repo_kind":"official","path":"WeCLIP+/WeCLIP_Plus/model_attn_aff_coco.py","file_url":"https://github.com/zbf1991/WeCLIP/blob/HEAD/WeCLIP%2B/WeCLIP_Plus/model_attn_aff_coco.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5bd180a934a83565"}},{"code_sha256_prefix":"1318e10e47a11362","entry":"dice_loss","repo":"zbf1991/WeCLIP","repo_kind":"official","path":"WeCLIP+/WeCLIP_Plus/dice_loss.py","file_url":"https://github.com/zbf1991/WeCLIP/blob/HEAD/WeCLIP%2B/WeCLIP_Plus/dice_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1318e10e47a11362"}},{"code_sha256_prefix":"ab1a564327ec7b27","entry":"get_aff_loss","repo":"zbf1991/WeCLIP","repo_kind":"official","path":"WeCLIP+/utils/losses.py","file_url":"https://github.com/zbf1991/WeCLIP/blob/HEAD/WeCLIP%2B/utils/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab1a564327ec7b27"}},{"code_sha256_prefix":"c5adf2da855149f1","entry":"get_energy_loss","repo":"zbf1991/WeCLIP","repo_kind":"official","path":"WeCLIP+/utils/losses.py","file_url":"https://github.com/zbf1991/WeCLIP/blob/HEAD/WeCLIP%2B/utils/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5adf2da855149f1"}},{"code_sha256_prefix":"9fe5c9cbe1f4399e","entry":"get_seg_loss","repo":"zbf1991/WeCLIP","repo_kind":"official","path":"WeCLIP+/utils/losses.py","file_url":"https://github.com/zbf1991/WeCLIP/blob/HEAD/WeCLIP%2B/utils/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9fe5c9cbe1f4399e"}},{"code_sha256_prefix":"b7949bedd44e211a","entry":"mmd_rbf","repo":"zbf1991/WeCLIP","repo_kind":"official","path":"WeCLIP+/utils/mmdloss.py","file_url":"https://github.com/zbf1991/WeCLIP/blob/HEAD/WeCLIP%2B/utils/mmdloss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b7949bedd44e211a"}},{"code_sha256_prefix":"34cdaf8dc22ba489","entry":"rbf_kernel","repo":"zbf1991/WeCLIP","repo_kind":"official","path":"WeCLIP+/utils/mmdloss.py","file_url":"https://github.com/zbf1991/WeCLIP/blob/HEAD/WeCLIP%2B/utils/mmdloss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"34cdaf8dc22ba489"}},{"code_sha256_prefix":"a112dc051428bbc2","entry":"replace_layer_recursive","repo":"zbf1991/WeCLIP","repo_kind":"official","path":"WeCLIP+/pytorch_grad_cam/ablation_cam_multilayer.py","file_url":"https://github.com/zbf1991/WeCLIP/blob/HEAD/WeCLIP%2B/pytorch_grad_cam/ablation_cam_multilayer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a112dc051428bbc2"}},{"code_sha256_prefix":"335f0f9cd2d9678c","entry":"upsample_pos_emb","repo":"zbf1991/WeCLIP","repo_kind":"official","path":"WeCLIP+/clip/model.py","file_url":"https://github.com/zbf1991/WeCLIP/blob/HEAD/WeCLIP%2B/clip/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"335f0f9cd2d9678c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}