{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-the-initialization-of-graph-neural","title":"On the Initialization of Graph Neural Networks","arxiv_id":"2312.02622","date":"2023-12-05","proceeding":null,"authors":["Jiahang Li","Yakun Song","Xiang Song","David Paul Wipf"],"abstract":"Graph Neural Networks (GNNs) have displayed considerable promise in graph representation learning across various applications. The core learning process requires the initialization of model weight matrices within each GNN layer, which is typically accomplished via classic initialization methods such as Xavier initialization. However, these methods were originally motivated to stabilize the variance of hidden embeddings and gradients across layers of Feedforward Neural Networks (FNNs) and Convolutional Neural Networks (CNNs) to avoid vanishing gradients and maintain steady information flow. In contrast, within the GNN context classical initializations disregard the impact of the input graph structure and message passing on variance. In this paper, we analyze the variance of forward and backward propagation across GNN layers and show that the variance instability of GNN initializations comes from the combined effect of the activation function, hidden dimension, graph structure and message passing. To better account for these influence factors, we propose a new initialization method for Variance Instability Reduction within GNN Optimization (Virgo), which naturally tends to equate forward and backward variances across successive layers. We conduct comprehensive experiments on 15 datasets to show that Virgo can lead to superior model performance and more stable variance at initialization on node classification, link prediction and graph classification tasks. Codes are in https://github.com/LspongebobJH/virgo_icml2023.","url_abs":"https://arxiv.org/abs/2312.02622v1","url_pdf":"https://arxiv.org/pdf/2312.02622v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-the-initialization-of-graph-neural","repo_url":"https://github.com/lspongebobjh/virgo_icml2023","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"graph-classification","task_name":"Graph Classification"},{"task_slug":"graph-representation-learning","task_name":"Graph Representation Learning"},{"task_slug":"link-prediction","task_name":"Link Prediction"},{"task_slug":"node-classification","task_name":"Node Classification"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.02622","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.02622"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lspongebobjh/virgo_icml2023","reach":null}],"summary":{"ran_draft_wrong":3,"ran_violates":1,"ran_honours":1},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"dc0809119c5ff309","entry":"difference","repo":"lspongebobjh/virgo_icml2023","repo_kind":"official","path":"utils.py","file_url":"https://github.com/lspongebobjh/virgo_icml2023/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc0809119c5ff309"}},{"code_sha256_prefix":"3391b7a2f0d87a26","entry":"evaluate","repo":"lspongebobjh/virgo_icml2023","repo_kind":"official","path":"comp1.py","file_url":"https://github.com/lspongebobjh/virgo_icml2023/blob/HEAD/comp1.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3391b7a2f0d87a26"}},{"code_sha256_prefix":"97c9e5145257422e","entry":"intersection","repo":"lspongebobjh/virgo_icml2023","repo_kind":"official","path":"utils.py","file_url":"https://github.com/lspongebobjh/virgo_icml2023/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"97c9e5145257422e"}},{"code_sha256_prefix":"07cd572b30fab1f9","entry":"to_uni_a","repo":"lspongebobjh/virgo_icml2023","repo_kind":"official","path":"utils.py","file_url":"https://github.com/lspongebobjh/virgo_icml2023/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"07cd572b30fab1f9"}},{"code_sha256_prefix":"de3d41d5550e1e6e","entry":"to_var","repo":"lspongebobjh/virgo_icml2023","repo_kind":"official","path":"utils_pyg.py","file_url":"https://github.com/lspongebobjh/virgo_icml2023/blob/HEAD/utils_pyg.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"de3d41d5550e1e6e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}