{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/nodeformer-a-scalable-graph-structure","title":"NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification","arxiv_id":"2306.08385","date":"2023-06-14","proceeding":null,"authors":["Qitian Wu","Wentao Zhao","Zenan Li","David Wipf","Junchi Yan"],"abstract":"Graph neural networks have been extensively studied for learning with inter-connected data. Despite this, recent evidence has revealed GNNs' deficiencies related to over-squashing, heterophily, handling long-range dependencies, edge incompleteness and particularly, the absence of graphs altogether. While a plausible solution is to learn new adaptive topology for message passing, issues concerning quadratic complexity hinder simultaneous guarantees for scalability and precision in large networks. In this paper, we introduce a novel all-pair message passing scheme for efficiently propagating node signals between arbitrary nodes, as an important building block for a pioneering Transformer-style network for node classification on large graphs, dubbed as \\textsc{NodeFormer}. Specifically, the efficient computation is enabled by a kernerlized Gumbel-Softmax operator that reduces the algorithmic complexity to linearity w.r.t. node numbers for learning latent graph structures from large, potentially fully-connected graphs in a differentiable manner. We also provide accompanying theory as justification for our design. Extensive experiments demonstrate the promising efficacy of the method in various tasks including node classification on graphs (with up to 2M nodes) and graph-enhanced applications (e.g., image classification) where input graphs are missing.","url_abs":"https://arxiv.org/abs/2306.08385v1","url_pdf":"https://arxiv.org/pdf/2306.08385v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"nodeformer-a-scalable-graph-structure","repo_url":"https://github.com/qitianwu/nodeformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"graph-structure-learning","task_name":"Graph structure learning"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"node-classification","task_name":"Node Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.08385","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.08385"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/qitianwu/NodeFormer","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qitianwu/nodeformer","reach":{"status":"ok"}}],"summary":{"ran_honours":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"214d66f90baa5770","entry":"create_products_of_givens_rotations","repo":"qitianwu/NodeFormer","repo_kind":"official","path":"nodeformer.py","file_url":"https://github.com/qitianwu/NodeFormer/blob/HEAD/nodeformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"214d66f90baa5770"}},{"code_sha256_prefix":"44fbc869db4685a6","entry":"create_projection_matrix","repo":"qitianwu/NodeFormer","repo_kind":"official","path":"nodeformer.py","file_url":"https://github.com/qitianwu/NodeFormer/blob/HEAD/nodeformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"44fbc869db4685a6"}},{"code_sha256_prefix":"9ec77c43b56275a8","entry":"relu_kernel_transformation","repo":"qitianwu/NodeFormer","repo_kind":"official","path":"nodeformer.py","file_url":"https://github.com/qitianwu/NodeFormer/blob/HEAD/nodeformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9ec77c43b56275a8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}