{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/be-confident-towards-trustworthy-graph-neural","title":"Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration","arxiv_id":"2109.14285","date":"2021-09-29","proceeding":"NeurIPS 2021 12","authors":["Xiao Wang","Hongrui Liu","Chuan Shi","Cheng Yang"],"abstract":"Despite Graph Neural Networks (GNNs) have achieved remarkable accuracy, whether the results are trustworthy is still unexplored. Previous studies suggest that many modern neural networks are over-confident on the predictions, however, surprisingly, we discover that GNNs are primarily in the opposite direction, i.e., GNNs are under-confident. Therefore, the confidence calibration for GNNs is highly desired. In this paper, we propose a novel trustworthy GNN model by designing a topology-aware post-hoc calibration function. Specifically, we first verify that the confidence distribution in a graph has homophily property, and this finding inspires us to design a calibration GNN model (CaGCN) to learn the calibration function. CaGCN is able to obtain a unique transformation from logits of GNNs to the calibrated confidence for each node, meanwhile, such transformation is able to preserve the order between classes, satisfying the accuracy-preserving property. Moreover, we apply the calibration GNN to self-training framework, showing that more trustworthy pseudo labels can be obtained with the calibrated confidence and further improve the performance. Extensive experiments demonstrate the effectiveness of our proposed model in terms of both calibration and accuracy.","url_abs":"https://arxiv.org/abs/2109.14285v3","url_pdf":"https://arxiv.org/pdf/2109.14285v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"be-confident-towards-trustworthy-graph-neural","repo_url":"https://github.com/BUPT-GAMMA/CaGCN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"be-confident-towards-trustworthy-graph-neural","repo_url":"https://github.com/TaurusTaurus-Rui/CaGCN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2109.14285","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.14285"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Davidham3/deeper_insights_into_GCNs","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/tkipf/pygcn","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/Diego999/pyGAT","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TaurusTaurus-Rui/CaGCN","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/BUPT-GAMMA/CaGCN","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"ran_violates":1,"unverified":11},"by_repo_kind":{"listed":{"samples":8,"ran":1,"repositories":2},"found_in_text":{"samples":7,"ran":3,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3ab88254add4c192","entry":"encode_onehot","repo":"tkipf/pygcn","repo_kind":"found_in_text","path":"pygcn/utils.py","file_url":"https://github.com/tkipf/pygcn/blob/HEAD/pygcn/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3ab88254add4c192"}},{"code_sha256_prefix":"aa9c29936dee40a7","entry":"normalize","repo":"tkipf/pygcn","repo_kind":"found_in_text","path":"pygcn/utils.py","file_url":"https://github.com/tkipf/pygcn/blob/HEAD/pygcn/utils.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa9c29936dee40a7"}},{"code_sha256_prefix":"d45f434d25f2e870","entry":"normalize_adj","repo":"Diego999/pyGAT","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/Diego999/pyGAT/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d45f434d25f2e870"}},{"code_sha256_prefix":"21b0199200507ff0","entry":"normalize_adj","repo":"BUPT-GAMMA/CaGCN","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/BUPT-GAMMA/CaGCN/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"21b0199200507ff0"}},{"code_sha256_prefix":"49985109b7aff28c","entry":"brier_score_criterion","repo":"BUPT-GAMMA/CaGCN","repo_kind":"listed","path":"util_calibration.py","file_url":"https://github.com/BUPT-GAMMA/CaGCN/blob/HEAD/util_calibration.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"49985109b7aff28c"}},{"code_sha256_prefix":"f9445ddf8653224c","entry":"encode_onehot","repo":"Diego999/pyGAT","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/Diego999/pyGAT/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9445ddf8653224c"}},{"code_sha256_prefix":"bbb7de90271b2771","entry":"generate_pesudo_label","repo":"BUPT-GAMMA/CaGCN","repo_kind":"listed","path":"CaGCN.py","file_url":"https://github.com/BUPT-GAMMA/CaGCN/blob/HEAD/CaGCN.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bbb7de90271b2771"}},{"code_sha256_prefix":"a7da17a36ae1d68d","entry":"generate_pesudo_label","repo":"BUPT-GAMMA/CaGCN","repo_kind":"listed","path":"baseline/co_training.py","file_url":"https://github.com/BUPT-GAMMA/CaGCN/blob/HEAD/baseline/co_training.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a7da17a36ae1d68d"}},{"code_sha256_prefix":"39199ce92db5bcf5","entry":"generate_pesudo_label","repo":"TaurusTaurus-Rui/CaGCN","repo_kind":"listed","path":"baseline/self_training.py","file_url":"https://github.com/TaurusTaurus-Rui/CaGCN/blob/HEAD/baseline/self_training.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39199ce92db5bcf5"}},{"code_sha256_prefix":"f653820785535682","entry":"get_confidence","repo":"BUPT-GAMMA/CaGCN","repo_kind":"listed","path":"util_calibration.py","file_url":"https://github.com/BUPT-GAMMA/CaGCN/blob/HEAD/util_calibration.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f653820785535682"}},{"code_sha256_prefix":"d35fed350a45f9fc","entry":"irova_calibrate","repo":"BUPT-GAMMA/CaGCN","repo_kind":"listed","path":"util_calibration.py","file_url":"https://github.com/BUPT-GAMMA/CaGCN/blob/HEAD/util_calibration.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d35fed350a45f9fc"}},{"code_sha256_prefix":"d1ed93364d2ef7cb","entry":"load_data","repo":"Diego999/pyGAT","repo_kind":"found_in_text","path":"utils.py","file_url":"https://github.com/Diego999/pyGAT/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d1ed93364d2ef7cb"}},{"code_sha256_prefix":"0503d0e1f92b51c3","entry":"load_data","repo":"tkipf/pygcn","repo_kind":"found_in_text","path":"pygcn/utils.py","file_url":"https://github.com/tkipf/pygcn/blob/HEAD/pygcn/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0503d0e1f92b51c3"}},{"code_sha256_prefix":"bca7f86b2fb5bf67","entry":"make_dot","repo":"Diego999/pyGAT","repo_kind":"found_in_text","path":"visualize_graph.py","file_url":"https://github.com/Diego999/pyGAT/blob/HEAD/visualize_graph.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bca7f86b2fb5bf67"}},{"code_sha256_prefix":"934ca60e9bd12c56","entry":"preprocess_adj","repo":"BUPT-GAMMA/CaGCN","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/BUPT-GAMMA/CaGCN/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"934ca60e9bd12c56"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}