{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cascade-clip-cascaded-vision-language","title":"Cascade-CLIP: Cascaded Vision-Language Embeddings Alignment for Zero-Shot Semantic Segmentation","arxiv_id":"2406.00670","date":"2024-06-02","proceeding":null,"authors":["Yunheng Li","Zhongyu Li","Quansheng Zeng","Qibin Hou","Ming-Ming Cheng"],"abstract":"Pre-trained vision-language models, e.g., CLIP, have been successfully applied to zero-shot semantic segmentation. Existing CLIP-based approaches primarily utilize visual features from the last layer to align with text embeddings, while they neglect the crucial information in intermediate layers that contain rich object details. However, we find that directly aggregating the multi-level visual features weakens the zero-shot ability for novel classes. The large differences between the visual features from different layers make these features hard to align well with the text embeddings. We resolve this problem by introducing a series of independent decoders to align the multi-level visual features with the text embeddings in a cascaded way, forming a novel but simple framework named Cascade-CLIP. Our Cascade-CLIP is flexible and can be easily applied to existing zero-shot semantic segmentation methods. Experimental results show that our simple Cascade-CLIP achieves superior zero-shot performance on segmentation benchmarks, like COCO-Stuff, Pascal-VOC, and Pascal-Context. Our code is available at: https://github.com/HVision-NKU/Cascade-CLIP","url_abs":"https://arxiv.org/abs/2406.00670v2","url_pdf":"https://arxiv.org/pdf/2406.00670v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cascade-clip-cascaded-vision-language","repo_url":"https://github.com/hvision-nku/cascade-clip","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"zero-shot-semantic-segmentation","task_name":"Zero-Shot Semantic Segmentation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.00670","atlas_url":"https://app.syntology.ai/?focus=2406.00670","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.00670"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hvision-nku/cascade-clip","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"864ff812c98727df","entry":"cosine_margin_loss","repo":"hvision-nku/cascade-clip","repo_kind":"official","path":"models/losses/criterion.py","file_url":"https://github.com/hvision-nku/cascade-clip/blob/HEAD/models/losses/criterion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"864ff812c98727df"}},{"code_sha256_prefix":"016dba4163da28df","entry":"dice_loss","repo":"hvision-nku/cascade-clip","repo_kind":"official","path":"models/losses/criterion.py","file_url":"https://github.com/hvision-nku/cascade-clip/blob/HEAD/models/losses/criterion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"016dba4163da28df"}},{"code_sha256_prefix":"39f08d24e0c8725c","entry":"gaussian","repo":"hvision-nku/cascade-clip","repo_kind":"official","path":"models/decode_heads/decode_seg.py","file_url":"https://github.com/hvision-nku/cascade-clip/blob/HEAD/models/decode_heads/decode_seg.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39f08d24e0c8725c"}},{"code_sha256_prefix":"7816c6d65ab55d8f","entry":"get_angles","repo":"hvision-nku/cascade-clip","repo_kind":"official","path":"models/decode_heads/utils.py","file_url":"https://github.com/hvision-nku/cascade-clip/blob/HEAD/models/decode_heads/utils.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7816c6d65ab55d8f"}},{"code_sha256_prefix":"da3acf7173bd84f0","entry":"positional_encoding","repo":"hvision-nku/cascade-clip","repo_kind":"official","path":"models/decode_heads/utils.py","file_url":"https://github.com/hvision-nku/cascade-clip/blob/HEAD/models/decode_heads/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"da3acf7173bd84f0"}},{"code_sha256_prefix":"2675e89c94ae08bf","entry":"sigmoid_focal_loss","repo":"hvision-nku/cascade-clip","repo_kind":"official","path":"models/losses/criterion.py","file_url":"https://github.com/hvision-nku/cascade-clip/blob/HEAD/models/losses/criterion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2675e89c94ae08bf"}},{"code_sha256_prefix":"58cc9ff3bf75e753","entry":"nested_tensor_from_tensor_list","repo":"hvision-nku/cascade-clip","repo_kind":"official","path":"models/losses/misc.py","file_url":"https://github.com/hvision-nku/cascade-clip/blob/HEAD/models/losses/misc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58cc9ff3bf75e753"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}