{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spikegraphormer-a-high-performance-graph","title":"SpikeGraphormer: A High-Performance Graph Transformer with Spiking Graph Attention","arxiv_id":"2403.15480","date":"2024-03-21","proceeding":null,"authors":["Yundong Sun","Dongjie Zhu","Yansong Wang","Zhaoshuo Tian","Ning Cao","Gregory O'Hared"],"abstract":"Recently, Graph Transformers have emerged as a promising solution to alleviate the inherent limitations of Graph Neural Networks (GNNs) and enhance graph representation performance. Unfortunately, Graph Transformers are computationally expensive due to the quadratic complexity inherent in self-attention when applied over large-scale graphs, especially for node tasks. In contrast, spiking neural networks (SNNs), with event-driven and binary spikes properties, can perform energy-efficient computation. In this work, we propose a novel insight into integrating SNNs with Graph Transformers and design a Spiking Graph Attention (SGA) module. The matrix multiplication is replaced by sparse addition and mask operations. The linear complexity enables all-pair node interactions on large-scale graphs with limited GPU memory. To our knowledge, our work is the first attempt to introduce SNNs into Graph Transformers. Furthermore, we design SpikeGraphormer, a Dual-branch architecture, combining a sparse GNN branch with our SGA-driven Graph Transformer branch, which can simultaneously perform all-pair node interactions and capture local neighborhoods. SpikeGraphormer consistently outperforms existing state-of-the-art approaches across various datasets and makes substantial improvements in training time, inference time, and GPU memory cost (10 ~ 20x lower than vanilla self-attention). It also performs well in cross-domain applications (image and text classification). We release our code at https://github.com/PHD-lanyu/SpikeGraphormer.","url_abs":"https://arxiv.org/abs/2403.15480v1","url_pdf":"https://arxiv.org/pdf/2403.15480v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spikegraphormer-a-high-performance-graph","repo_url":"https://github.com/phd-lanyu/spikegraphormer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"graph-attention","task_name":"Graph Attention"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"graph-transformer","method_name":"Graph Transformer"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"lapeigen","method_name":"LapEigen"},{"method_slug":"laplacian-pe","method_name":"Laplacian PE"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.15480","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.15480"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/phd-lanyu/spikegraphormer","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":5,"unverified":1},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"de5bdf76ff672322","entry":"class_rand_splits","repo":"phd-lanyu/spikegraphormer","repo_kind":"official","path":"large/data_utils.py","file_url":"https://github.com/phd-lanyu/spikegraphormer/blob/HEAD/large/data_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"de5bdf76ff672322"}},{"code_sha256_prefix":"e0d4c1372efb5382","entry":"class_rand_splits","repo":"phd-lanyu/spikegraphormer","repo_kind":"official","path":"medium/data_utils.py","file_url":"https://github.com/phd-lanyu/spikegraphormer/blob/HEAD/medium/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e0d4c1372efb5382"}},{"code_sha256_prefix":"ef6b5e211f3c0418","entry":"load_fixed_splits","repo":"phd-lanyu/spikegraphormer","repo_kind":"official","path":"large/data_utils.py","file_url":"https://github.com/phd-lanyu/spikegraphormer/blob/HEAD/large/data_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ef6b5e211f3c0418"}},{"code_sha256_prefix":"162abe8248541689","entry":"normalize_feat","repo":"phd-lanyu/spikegraphormer","repo_kind":"official","path":"medium/data_utils.py","file_url":"https://github.com/phd-lanyu/spikegraphormer/blob/HEAD/medium/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"162abe8248541689"}},{"code_sha256_prefix":"cf34dc0377c08698","entry":"rand_train_test_idx","repo":"phd-lanyu/spikegraphormer","repo_kind":"official","path":"medium/data_utils.py","file_url":"https://github.com/phd-lanyu/spikegraphormer/blob/HEAD/medium/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cf34dc0377c08698"}},{"code_sha256_prefix":"13234652741e2264","entry":"rand_train_test_idx","repo":"phd-lanyu/spikegraphormer","repo_kind":"official","path":"large/data_utils.py","file_url":"https://github.com/phd-lanyu/spikegraphormer/blob/HEAD/large/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"13234652741e2264"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}