{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diffuse-attend-and-segment-unsupervised-zero","title":"Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusion","arxiv_id":"2308.12469","date":"2023-08-23","proceeding":null,"authors":["Junjiao Tian","Lavisha Aggarwal","Andrea Colaco","Zsolt Kira","Mar Gonzalez-Franco"],"abstract":"Producing quality segmentation masks for images is a fundamental problem in computer vision. Recent research has explored large-scale supervised training to enable zero-shot segmentation on virtually any image style and unsupervised training to enable segmentation without dense annotations. However, constructing a model capable of segmenting anything in a zero-shot manner without any annotations is still challenging. In this paper, we propose to utilize the self-attention layers in stable diffusion models to achieve this goal because the pre-trained stable diffusion model has learned inherent concepts of objects within its attention layers. Specifically, we introduce a simple yet effective iterative merging process based on measuring KL divergence among attention maps to merge them into valid segmentation masks. The proposed method does not require any training or language dependency to extract quality segmentation for any images. On COCO-Stuff-27, our method surpasses the prior unsupervised zero-shot SOTA method by an absolute 26% in pixel accuracy and 17% in mean IoU. The project page is at \\url{https://sites.google.com/view/diffseg/home}.","url_abs":"https://arxiv.org/abs/2308.12469v3","url_pdf":"https://arxiv.org/pdf/2308.12469v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"diffuse-attend-and-segment-unsupervised-zero","repo_url":"https://github.com/google/diffseg","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"zero-shot-segmentation","task_name":"Zero Shot Segmentation"},{"task_slug":null,"task_name":"valid"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/semantic-segmentation-on-coco-stuff-27","task":"Semantic Segmentation","dataset":"COCO-Stuff-27","model":"DiffSeg (512)","rank_in_archive_order":1,"of":1,"metrics":{"Pixel Accuracy":"72.5","mIoU":"43.6"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cityscapes-2","task":"Semantic Segmentation","dataset":"Cityscapes","model":"DiffSeg (512)","rank_in_archive_order":2,"of":2,"metrics":{"Pixel Accuracy":"76","mIoU":"21.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.12469","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.12469"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google/diffseg","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"023a352094863b2c","entry":"find_edges","repo":"google/diffseg","repo_kind":"official","path":"diffseg/utils.py","file_url":"https://github.com/google/diffseg/blob/HEAD/diffseg/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"023a352094863b2c"}},{"code_sha256_prefix":"7c271596415d695e","entry":"hungarian_matching","repo":"google/diffseg","repo_kind":"official","path":"diffseg/utils.py","file_url":"https://github.com/google/diffseg/blob/HEAD/diffseg/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7c271596415d695e"}},{"code_sha256_prefix":"145bd246f65e64ca","entry":"process_image","repo":"google/diffseg","repo_kind":"official","path":"diffseg/utils.py","file_url":"https://github.com/google/diffseg/blob/HEAD/diffseg/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"145bd246f65e64ca"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}