{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/quantifying-the-knowledge-in-gnns-for","title":"Quantifying the Knowledge in GNNs for Reliable Distillation into MLPs","arxiv_id":"2306.05628","date":"2023-06-09","proceeding":null,"authors":["Lirong Wu","Haitao Lin","Yufei Huang","Stan Z. Li"],"abstract":"To bridge the gaps between topology-aware Graph Neural Networks (GNNs) and inference-efficient Multi-Layer Perceptron (MLPs), GLNN proposes to distill knowledge from a well-trained teacher GNN into a student MLP. Despite their great progress, comparatively little work has been done to explore the reliability of different knowledge points (nodes) in GNNs, especially their roles played during distillation. In this paper, we first quantify the knowledge reliability in GNN by measuring the invariance of their information entropy to noise perturbations, from which we observe that different knowledge points (1) show different distillation speeds (temporally); (2) are differentially distributed in the graph (spatially). To achieve reliable distillation, we propose an effective approach, namely Knowledge-inspired Reliable Distillation (KRD), that models the probability of each node being an informative and reliable knowledge point, based on which we sample a set of additional reliable knowledge points as supervision for training student MLPs. Extensive experiments show that KRD improves over the vanilla MLPs by 12.62% and outperforms its corresponding teacher GNNs by 2.16% averaged over 7 datasets and 3 GNN architectures.","url_abs":"https://arxiv.org/abs/2306.05628v1","url_pdf":"https://arxiv.org/pdf/2306.05628v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"quantifying-the-knowledge-in-gnns-for","repo_url":"https://github.com/lirongwu/rkd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"quantifying-the-knowledge-in-gnns-for","repo_url":"https://github.com/rufaim/routing-by-memory","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.05628","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.05628"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lirongwu/rkd","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rufaim/routing-by-memory","reach":null}],"summary":{"ran":1,"ran_violates":1,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":7,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"77c12b26c7311bbd","entry":"is_binary_bag_of_words","repo":"lirongwu/rkd","repo_kind":"official","path":"src/data_preprocess.py","file_url":"https://github.com/lirongwu/rkd/blob/HEAD/src/data_preprocess.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77c12b26c7311bbd"}},{"code_sha256_prefix":"aa9c29936dee40a7","entry":"normalize","repo":"lirongwu/rkd","repo_kind":"official","path":"src/data_preprocess.py","file_url":"https://github.com/lirongwu/rkd/blob/HEAD/src/data_preprocess.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa9c29936dee40a7"}},{"code_sha256_prefix":"c1b14a7851940dca","entry":"to_binary_bag_of_words","repo":"lirongwu/rkd","repo_kind":"official","path":"src/data_preprocess.py","file_url":"https://github.com/lirongwu/rkd/blob/HEAD/src/data_preprocess.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c1b14a7851940dca"}},{"code_sha256_prefix":"1901d498eb16daad","entry":"evaluate","repo":"lirongwu/rkd","repo_kind":"official","path":"src/train_and_eval.py","file_url":"https://github.com/lirongwu/rkd/blob/HEAD/src/train_and_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1901d498eb16daad"}},{"code_sha256_prefix":"ba8659b1d271a02a","entry":"get_evaluator","repo":"lirongwu/rkd","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/lirongwu/rkd/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ba8659b1d271a02a"}},{"code_sha256_prefix":"29faf1d5e00e0643","entry":"train","repo":"lirongwu/rkd","repo_kind":"official","path":"src/train_and_eval.py","file_url":"https://github.com/lirongwu/rkd/blob/HEAD/src/train_and_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"29faf1d5e00e0643"}},{"code_sha256_prefix":"b83839d3fe369101","entry":"train_mini_batch","repo":"lirongwu/rkd","repo_kind":"official","path":"src/train_and_eval.py","file_url":"https://github.com/lirongwu/rkd/blob/HEAD/src/train_and_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b83839d3fe369101"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}