{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/graph-less-neural-networks-teaching-old-mlps-1","title":"Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation","arxiv_id":"2110.08727","date":"2021-10-17","proceeding":"ICLR 2022 4","authors":["Shichang Zhang","Yozen Liu","Yizhou Sun","Neil Shah"],"abstract":"Graph Neural Networks (GNNs) are popular for graph machine learning and have shown great results on wide node classification tasks. Yet, they are less popular for practical deployments in the industry owing to their scalability challenges incurred by data dependency. Namely, GNN inference depends on neighbor nodes multiple hops away from the target, and fetching them burdens latency-constrained applications. Existing inference acceleration methods like pruning and quantization can speed up GNNs by reducing Multiplication-and-ACcumulation (MAC) operations, but the improvements are limited given the data dependency is not resolved. Conversely, multi-layer perceptrons (MLPs) have no graph dependency and infer much faster than GNNs, even though they are less accurate than GNNs for node classification in general. Motivated by these complementary strengths and weaknesses, we bring GNNs and MLPs together via knowledge distillation (KD). Our work shows that the performance of MLPs can be improved by large margins with GNN KD. We call the distilled MLPs Graph-less Neural Networks (GLNNs) as they have no inference graph dependency. We show that GLNNs with competitive accuracy infer faster than GNNs by 146X-273X and faster than other acceleration methods by 14X-27X. Under a production setting involving both transductive and inductive predictions across 7 datasets, GLNN accuracies improve over stand-alone MLPs by 12.36% on average and match GNNs on 6/7 datasets. Comprehensive analysis shows when and why GLNNs can achieve competitive accuracies to GNNs and suggests GLNN as a handy choice for latency-constrained applications.","url_abs":"https://arxiv.org/abs/2110.08727v2","url_pdf":"https://arxiv.org/pdf/2110.08727v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"graph-less-neural-networks-teaching-old-mlps-1","repo_url":"https://github.com/snap-research/graphless-neural-networks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"node-classification","task_name":"Node Classification"},{"task_slug":"node-property-prediction","task_name":"Node Property Prediction"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/node-classification-on-amz-computers","task":"Node Classification","dataset":"AMZ Computers","model":"GLNN","rank_in_archive_order":5,"of":5,"metrics":{"Accuracy":"83.03± 1.87%"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-amz-photo","task":"Node Classification","dataset":"AMZ Photo","model":"GLNN","rank_in_archive_order":11,"of":14,"metrics":{"Accuracy":"92.11± 1.08%"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-citeseer","task":"Node Classification","dataset":"Citeseer","model":"GLNN","rank_in_archive_order":50,"of":71,"metrics":{"Accuracy":"71.77± 2.01"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-cora","task":"Node Classification","dataset":"Cora","model":"GLNN","rank_in_archive_order":64,"of":73,"metrics":{"Accuracy":"80.54± 1.35%"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-pubmed","task":"Node Classification","dataset":"Pubmed","model":"GLNN","rank_in_archive_order":63,"of":70,"metrics":{"Accuracy":"75.42 ± 2.31"},"uses_additional_data":false},{"leaderboard":"/sota/node-property-prediction-on-ogbn-arxiv","task":"Node Property Prediction","dataset":"ogbn-arxiv","model":"GLNN","rank_in_archive_order":82,"of":86,"metrics":{"Test Accuracy":"0.6346± 0.0045"},"uses_additional_data":false},{"leaderboard":"/sota/node-property-prediction-on-ogbn-products","task":"Node Property Prediction","dataset":"ogbn-products","model":"GLNN","rank_in_archive_order":61,"of":64,"metrics":{"Test Accuracy":"0.6886 ± 0.0046"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2110.08727","atlas_url":"https://app.syntology.ai/?focus=2110.08727","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.08727"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/snap-research/graphless-neural-networks","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"ran_violates":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"5b7ca3c80fc2a11a","entry":"get_training_config","repo":"snap-research/graphless-neural-networks","repo_kind":"official","path":"utils.py","file_url":"https://github.com/snap-research/graphless-neural-networks/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"5b7ca3c80fc2a11a"}},{"code_sha256_prefix":"ba7dbda11c083fd7","entry":"idx_split","repo":"snap-research/graphless-neural-networks","repo_kind":"official","path":"utils.py","file_url":"https://github.com/snap-research/graphless-neural-networks/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"ba7dbda11c083fd7"}},{"code_sha256_prefix":"77c12b26c7311bbd","entry":"is_binary_bag_of_words","repo":"snap-research/graphless-neural-networks","repo_kind":"official","path":"data_preprocess.py","file_url":"https://github.com/snap-research/graphless-neural-networks/blob/HEAD/data_preprocess.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"77c12b26c7311bbd"}},{"code_sha256_prefix":"aa9c29936dee40a7","entry":"normalize","repo":"snap-research/graphless-neural-networks","repo_kind":"official","path":"data_preprocess.py","file_url":"https://github.com/snap-research/graphless-neural-networks/blob/HEAD/data_preprocess.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"aa9c29936dee40a7"}},{"code_sha256_prefix":"c1b14a7851940dca","entry":"to_binary_bag_of_words","repo":"snap-research/graphless-neural-networks","repo_kind":"official","path":"data_preprocess.py","file_url":"https://github.com/snap-research/graphless-neural-networks/blob/HEAD/data_preprocess.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"c1b14a7851940dca"}},{"code_sha256_prefix":"2b547c0185564a58","entry":"train","repo":"snap-research/graphless-neural-networks","repo_kind":"official","path":"train_and_eval.py","file_url":"https://github.com/snap-research/graphless-neural-networks/blob/HEAD/train_and_eval.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"2b547c0185564a58"}},{"code_sha256_prefix":"69640025673d56e1","entry":"train_mini_batch","repo":"snap-research/graphless-neural-networks","repo_kind":"official","path":"train_and_eval.py","file_url":"https://github.com/snap-research/graphless-neural-networks/blob/HEAD/train_and_eval.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"69640025673d56e1"}},{"code_sha256_prefix":"ee0ff9821725414f","entry":"train_sage","repo":"snap-research/graphless-neural-networks","repo_kind":"official","path":"train_and_eval.py","file_url":"https://github.com/snap-research/graphless-neural-networks/blob/HEAD/train_and_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"ee0ff9821725414f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}