{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/corrupting-neuron-explanations-of-deep-visual-1","title":"Corrupting Neuron Explanations of Deep Visual Features","arxiv_id":"2310.16332","date":"2023-10-25","proceeding":"ICCV 2023 1","authors":["Divyansh Srivastava","Tuomas Oikarinen","Tsui-Wei Weng"],"abstract":"The inability of DNNs to explain their black-box behavior has led to a recent surge of explainability methods. However, there are growing concerns that these explainability methods are not robust and trustworthy. In this work, we perform the first robustness analysis of Neuron Explanation Methods under a unified pipeline and show that these explanations can be significantly corrupted by random noises and well-designed perturbations added to their probing data. We find that even adding small random noise with a standard deviation of 0.02 can already change the assigned concepts of up to 28% neurons in the deeper layers. Furthermore, we devise a novel corruption algorithm and show that our algorithm can manipulate the explanation of more than 80% neurons by poisoning less than 10% of probing data. This raises the concern of trusting Neuron Explanation Methods in real-life safety and fairness critical applications.","url_abs":"https://arxiv.org/abs/2310.16332v1","url_pdf":"https://arxiv.org/pdf/2310.16332v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"corrupting-neuron-explanations-of-deep-visual-1","repo_url":"https://github.com/Trustworthy-ML-Lab/corrupting_neuron_explanations","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.16332","atlas_url":"https://app.syntology.ai/?focus=2310.16332","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.16332"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Trustworthy-ML-Lab/corrupting_neuron_explanations","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c5abe011eafabd71","entry":"decode_packed_segmentation_rgb","repo":"Trustworthy-ML-Lab/corrupting_neuron_explanations","repo_kind":"official","path":"network-dissection/loader/data_loader.py","file_url":"https://github.com/Trustworthy-ML-Lab/corrupting_neuron_explanations/blob/HEAD/network-dissection/loader/data_loader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c5abe011eafabd71"}},{"code_sha256_prefix":"c8b1a48d7aa8bf6a","entry":"load_csv","repo":"Trustworthy-ML-Lab/corrupting_neuron_explanations","repo_kind":"official","path":"network-dissection/loader/data_loader.py","file_url":"https://github.com/Trustworthy-ML-Lab/corrupting_neuron_explanations/blob/HEAD/network-dissection/loader/data_loader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8b1a48d7aa8bf6a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}