{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/conceptattention-diffusion-transformers-learn","title":"ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features","arxiv_id":"2502.04320","date":"2025-02-06","proceeding":null,"authors":["Alec Helbling","Tuna Han Salih Meral","Ben Hoover","Pinar Yanardag","Duen Horng Chau"],"abstract":"Do the rich representations of multi-modal diffusion transformers (DiTs) exhibit unique properties that enhance their interpretability? We introduce ConceptAttention, a novel method that leverages the expressive power of DiT attention layers to generate high-quality saliency maps that precisely locate textual concepts within images. Without requiring additional training, ConceptAttention repurposes the parameters of DiT attention layers to produce highly contextualized concept embeddings, contributing the major discovery that performing linear projections in the output space of DiT attention layers yields significantly sharper saliency maps compared to commonly used cross-attention mechanisms. Remarkably, ConceptAttention even achieves state-of-the-art performance on zero-shot image segmentation benchmarks, outperforming 11 other zero-shot interpretability methods on the ImageNet-Segmentation dataset and on a single-class subset of PascalVOC. Our work contributes the first evidence that the representations of multi-modal DiT models like Flux are highly transferable to vision tasks like segmentation, even outperforming multi-modal foundation models like CLIP.","url_abs":"https://arxiv.org/abs/2502.04320v1","url_pdf":"https://arxiv.org/pdf/2502.04320v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"conceptattention-diffusion-transformers-learn","repo_url":"https://github.com/helblazer811/ConceptAttention","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.04320","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.04320"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/helblazer811/ConceptAttention","reach":null}],"summary":{"ran":4,"unverified":5},"by_repo_kind":{"official":{"samples":9,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"68e13022e7ff1c3e","entry":"Modulation","repo":"helblazer811/ConceptAttention","repo_kind":"official","path":"concept_attention/flux/dit_block.py","file_url":"https://github.com/helblazer811/ConceptAttention/blob/HEAD/concept_attention/flux/dit_block.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"68e13022e7ff1c3e"}},{"code_sha256_prefix":"a9abef6f4da5afe3","entry":"ModulationOut","repo":"helblazer811/ConceptAttention","repo_kind":"official","path":"concept_attention/flux/dit_block.py","file_url":"https://github.com/helblazer811/ConceptAttention/blob/HEAD/concept_attention/flux/dit_block.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a9abef6f4da5afe3"}},{"code_sha256_prefix":"2f43209f61e8219e","entry":"QKNorm","repo":"helblazer811/ConceptAttention","repo_kind":"official","path":"concept_attention/flux/dit_block.py","file_url":"https://github.com/helblazer811/ConceptAttention/blob/HEAD/concept_attention/flux/dit_block.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2f43209f61e8219e"}},{"code_sha256_prefix":"5c52ab157dc1c15c","entry":"RMSNorm","repo":"helblazer811/ConceptAttention","repo_kind":"official","path":"concept_attention/flux/dit_block.py","file_url":"https://github.com/helblazer811/ConceptAttention/blob/HEAD/concept_attention/flux/dit_block.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5c52ab157dc1c15c"}},{"code_sha256_prefix":"54f6ec9f0d9af990","entry":"ModifiedDoubleStreamBlock","repo":"helblazer811/ConceptAttention","repo_kind":"official","path":"concept_attention/flux/dit_block.py","file_url":"https://github.com/helblazer811/ConceptAttention/blob/HEAD/concept_attention/flux/dit_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"54f6ec9f0d9af990"}},{"code_sha256_prefix":"44b8130635f7bada","entry":"SelfAttention","repo":"helblazer811/ConceptAttention","repo_kind":"official","path":"concept_attention/flux/dit_block.py","file_url":"https://github.com/helblazer811/ConceptAttention/blob/HEAD/concept_attention/flux/dit_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"44b8130635f7bada"}},{"code_sha256_prefix":"7f1b2506d32e33e3","entry":"apply_rope","repo":"helblazer811/ConceptAttention","repo_kind":"official","path":"concept_attention/flux/dit_block.py","file_url":"https://github.com/helblazer811/ConceptAttention/blob/HEAD/concept_attention/flux/dit_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7f1b2506d32e33e3"}},{"code_sha256_prefix":"6d1fbe9be366c5ae","entry":"attention","repo":"helblazer811/ConceptAttention","repo_kind":"official","path":"concept_attention/flux/dit_block.py","file_url":"https://github.com/helblazer811/ConceptAttention/blob/HEAD/concept_attention/flux/dit_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6d1fbe9be366c5ae"}},{"code_sha256_prefix":"1af01c699046b8d4","entry":"scaled_dot_product_attention","repo":"helblazer811/ConceptAttention","repo_kind":"official","path":"concept_attention/flux/dit_block.py","file_url":"https://github.com/helblazer811/ConceptAttention/blob/HEAD/concept_attention/flux/dit_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1af01c699046b8d4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}