{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/maxk-gnn-towards-theoretical-speed-limits-for","title":"MaxK-GNN: Extremely Fast GPU Kernel Design for Accelerating Graph Neural Networks Training","arxiv_id":"2312.08656","date":"2023-12-14","proceeding":null,"authors":["Hongwu Peng","Xi Xie","Kaustubh Shivdikar","MD Amit Hasan","Jiahui Zhao","Shaoyi Huang","Omer Khan","David Kaeli","Caiwen Ding"],"abstract":"In the acceleration of deep neural network training, the GPU has become the mainstream platform. GPUs face substantial challenges on GNNs, such as workload imbalance and memory access irregularities, leading to underutilized hardware. Existing solutions such as PyG, DGL with cuSPARSE, and GNNAdvisor frameworks partially address these challenges but memory traffic is still significant. We argue that drastic performance improvements can only be achieved by the vertical optimization of algorithm and system innovations, rather than treating the speedup optimization as an \"after-thought\" (i.e., (i) given a GNN algorithm, designing an accelerator, or (ii) given hardware, mainly optimizing the GNN algorithm). In this paper, we present MaxK-GNN, an advanced high-performance GPU training system integrating algorithm and system innovation. (i) We introduce the MaxK nonlinearity and provide a theoretical analysis of MaxK nonlinearity as a universal approximator, and present the Compressed Balanced Sparse Row (CBSR) format, designed to store the data and index of the feature matrix after nonlinearity; (ii) We design a coalescing enhanced forward computation with row-wise product-based SpGEMM Kernel using CBSR for input feature matrix fetching and strategic placement of a sparse output accumulation buffer in shared memory; (iii) We develop an optimized backward computation with outer product-based and SSpMM Kernel. We conduct extensive evaluations of MaxK-GNN and report the end-to-end system run-time. Experiments show that MaxK-GNN system could approach the theoretical speedup limit according to Amdahl's law. We achieve comparable accuracy to SOTA GNNs, but at a significantly increased speed: 3.22/4.24 times speedup (vs. theoretical limits, 5.52/7.27 times) on Reddit compared to DGL and GNNAdvisor implementations.","url_abs":"https://arxiv.org/abs/2312.08656v5","url_pdf":"https://arxiv.org/pdf/2312.08656v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"maxk-gnn-towards-theoretical-speed-limits-for","repo_url":"https://github.com/harveyp123/maxk-gnn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":null,"task_name":"GPU"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.08656","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.08656"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/harveyp123/maxk-gnn","reach":{"status":"ok"}}],"summary":{"ran":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"c049a4fbd57129f7","entry":"accuracy","repo":"harveyp123/maxk-gnn","repo_kind":"official","path":"utils/general_utils.py","file_url":"https://github.com/harveyp123/maxk-gnn/blob/HEAD/utils/general_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c049a4fbd57129f7"}},{"code_sha256_prefix":"0fb6732d5ca48437","entry":"get_logger","repo":"harveyp123/maxk-gnn","repo_kind":"official","path":"utils/general_utils.py","file_url":"https://github.com/harveyp123/maxk-gnn/blob/HEAD/utils/general_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0fb6732d5ca48437"}},{"code_sha256_prefix":"bf0bc305c2302fec","entry":"get_parser","repo":"harveyp123/maxk-gnn","repo_kind":"official","path":"utils/config.py","file_url":"https://github.com/harveyp123/maxk-gnn/blob/HEAD/utils/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bf0bc305c2302fec"}},{"code_sha256_prefix":"e743627634484e6d","entry":"param_size","repo":"harveyp123/maxk-gnn","repo_kind":"official","path":"utils/general_utils.py","file_url":"https://github.com/harveyp123/maxk-gnn/blob/HEAD/utils/general_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e743627634484e6d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}