{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/coe-chain-of-explanation-via-automatic-visual","title":"CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantification","arxiv_id":"2503.15234","date":"2025-03-19","proceeding":"CVPR 2025 1","authors":["Wenlong Yu","Qilong Wang","Chuang Liu","Dong Li","QinGhua Hu"],"abstract":"Explainability is a critical factor influencing the wide deployment of deep vision models (DVMs). Concept-based post-hoc explanation methods can provide both global and local insights into model decisions. However, current methods in this field face challenges in that they are inflexible to automatically construct accurate and sufficient linguistic explanations for global concepts and local circuits. Particularly, the intrinsic polysemanticity in semantic Visual Concepts (VCs) impedes the interpretability of concepts and DVMs, which is underestimated severely. In this paper, we propose a Chain-of-Explanation (CoE) approach to address these issues. Specifically, CoE automates the decoding and description of VCs to construct global concept explanation datasets. Further, to alleviate the effect of polysemanticity on model explainability, we design a concept polysemanticity disentanglement and filtering mechanism to distinguish the most contextually relevant concept atoms. Besides, a Concept Polysemanticity Entropy (CPE), as a measure of model interpretability, is formulated to quantify the degree of concept uncertainty. The modeling of deterministic concepts is upgraded to uncertain concept atom distributions. Finally, CoE automatically enables linguistic local explanations of the decision-making process of DVMs by tracing the concept circuit. GPT-4o and human-based experiments demonstrate the effectiveness of CPE and the superiority of CoE, achieving an average absolute improvement of 36% in terms of explainability scores.","url_abs":"https://arxiv.org/abs/2503.15234v1","url_pdf":"https://arxiv.org/pdf/2503.15234v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"coe-chain-of-explanation-via-automatic-visual","repo_url":"https://github.com/YuWLong666/CoE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"disentanglement","task_name":"Disentanglement"}],"methods":[{"method_slug":"cpe","method_name":"CPE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.15234","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.15234"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/YuWLong666/CoE","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"ran":3,"ran_draft_wrong":2,"ran_fixture":1,"unverified":9},"by_repo_kind":{"official":{"samples":15,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"aa6da43d05cfaf65","entry":"all_gather","repo":"YuWLong666/CoE","repo_kind":"official","path":"util/misc.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/util/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"aa6da43d05cfaf65"}},{"code_sha256_prefix":"57e95166ade5f25c","entry":"build_transform","repo":"YuWLong666/CoE","repo_kind":"official","path":"models/internvl.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/models/internvl.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"57e95166ade5f25c"}},{"code_sha256_prefix":"d610b5eabe0c9db1","entry":"dynamic_preprocess","repo":"YuWLong666/CoE","repo_kind":"official","path":"models/internvl.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/models/internvl.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"d610b5eabe0c9db1"}},{"code_sha256_prefix":"f41cb1a19b154297","entry":"encode_image","repo":"YuWLong666/CoE","repo_kind":"official","path":"closeai.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/closeai.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"f41cb1a19b154297"}},{"code_sha256_prefix":"bd77f5f8067f18e9","entry":"find_closest_aspect_ratio","repo":"YuWLong666/CoE","repo_kind":"official","path":"models/internvl.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/models/internvl.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"bd77f5f8067f18e9"}},{"code_sha256_prefix":"05992505df9ef0cd","entry":"reduce_dict","repo":"YuWLong666/CoE","repo_kind":"official","path":"util/misc.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/util/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"05992505df9ef0cd"}},{"code_sha256_prefix":"5636ff5ce8a9e959","entry":"basicset_logger","repo":"YuWLong666/CoE","repo_kind":"official","path":"util/loggerset.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/util/loggerset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"5636ff5ce8a9e959"}},{"code_sha256_prefix":"f4007627e1cb8260","entry":"check_folder","repo":"YuWLong666/CoE","repo_kind":"official","path":"polysemantic_eval.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/polysemantic_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"f4007627e1cb8260"}},{"code_sha256_prefix":"d29adfeb0ea0baf0","entry":"context_entails_response","repo":"YuWLong666/CoE","repo_kind":"official","path":"semantic_entropy.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/semantic_entropy.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"d29adfeb0ea0baf0"}},{"code_sha256_prefix":"0a889e98f9adc412","entry":"entropy_loss_func","repo":"YuWLong666/CoE","repo_kind":"official","path":"util/misc.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/util/misc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"0a889e98f9adc412"}},{"code_sha256_prefix":"a628613dbdd55286","entry":"get_args_parser","repo":"YuWLong666/CoE","repo_kind":"official","path":"parser.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"a628613dbdd55286"}},{"code_sha256_prefix":"43fb79dfb73bea95","entry":"get_response","repo":"YuWLong666/CoE","repo_kind":"official","path":"closeai.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/closeai.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"43fb79dfb73bea95"}},{"code_sha256_prefix":"0635b0e8c158742e","entry":"get_semantic_ids","repo":"YuWLong666/CoE","repo_kind":"official","path":"semantic_entropy.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/semantic_entropy.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"0635b0e8c158742e"}},{"code_sha256_prefix":"1ac9c618cee6223f","entry":"logsumexp_by_id","repo":"YuWLong666/CoE","repo_kind":"official","path":"semantic_entropy.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/semantic_entropy.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"1ac9c618cee6223f"}},{"code_sha256_prefix":"e53736b8f5970f9c","entry":"read_csv","repo":"YuWLong666/CoE","repo_kind":"official","path":"util/plot_utils.py","file_url":"https://github.com/YuWLong666/CoE/blob/HEAD/util/plot_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e53736b8f5970f9c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}