{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-robust-fidelity-for-evaluating","title":"Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks","arxiv_id":"2310.01820","date":"2023-10-03","proceeding":null,"authors":["Xu Zheng","Farhad Shirani","Tianchun Wang","Wei Cheng","Zhuomin Chen","Haifeng Chen","Hua Wei","Dongsheng Luo"],"abstract":"Graph Neural Networks (GNNs) are neural models that leverage the dependency structure in graphical data via message passing among the graph nodes. GNNs have emerged as pivotal architectures in analyzing graph-structured data, and their expansive application in sensitive domains requires a comprehensive understanding of their decision-making processes -- necessitating a framework for GNN explainability. An explanation function for GNNs takes a pre-trained GNN along with a graph as input, to produce a `sufficient statistic' subgraph with respect to the graph label. A main challenge in studying GNN explainability is to provide fidelity measures that evaluate the performance of these explanation functions. This paper studies this foundational challenge, spotlighting the inherent limitations of prevailing fidelity metrics, including $Fid_+$, $Fid_-$, and $Fid_\\Delta$. Specifically, a formal, information-theoretic definition of explainability is introduced and it is shown that existing metrics often fail to align with this definition across various statistical scenarios. The reason is due to potential distribution shifts when subgraphs are removed in computing these fidelity measures. Subsequently, a robust class of fidelity measures are introduced, and it is shown analytically that they are resilient to distribution shift issues and are applicable in a wide range of scenarios. Extensive empirical analysis on both synthetic and real datasets are provided to illustrate that the proposed metrics are more coherent with gold standard metrics. The source code is available at https://trustai4s-lab.github.io/fidelity.","url_abs":"https://arxiv.org/abs/2310.01820v2","url_pdf":"https://arxiv.org/pdf/2310.01820v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-robust-fidelity-for-evaluating","repo_url":"https://github.com/AslanDing/Fidelity","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.01820","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.01820"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AslanDing/Fidelity","reach":{"status":"ok"}}],"summary":{"ran":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"2a2c78c4cd9756e1","entry":"get_gin","repo":"AslanDing/Fidelity","repo_kind":"official","path":"ExplanationEvaluation/models/PG_paper.py","file_url":"https://github.com/AslanDing/Fidelity/blob/HEAD/ExplanationEvaluation/models/PG_paper.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a2c78c4cd9756e1"}},{"code_sha256_prefix":"a838ece468d28a75","entry":"get_pretrained_path","repo":"AslanDing/Fidelity","repo_kind":"official","path":"ExplanationEvaluation/models/model_selector.py","file_url":"https://github.com/AslanDing/Fidelity/blob/HEAD/ExplanationEvaluation/models/model_selector.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a838ece468d28a75"}},{"code_sha256_prefix":"b73159e0a84d0f40","entry":"get_pretrained_path_extend","repo":"AslanDing/Fidelity","repo_kind":"official","path":"ExplanationEvaluation/models/model_selector.py","file_url":"https://github.com/AslanDing/Fidelity/blob/HEAD/ExplanationEvaluation/models/model_selector.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b73159e0a84d0f40"}},{"code_sha256_prefix":"eee7bf973fd50e47","entry":"visulaized","repo":"AslanDing/Fidelity","repo_kind":"official","path":"generate_edit_distance_random.py","file_url":"https://github.com/AslanDing/Fidelity/blob/HEAD/generate_edit_distance_random.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eee7bf973fd50e47"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}