{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/old-can-be-gold-better-gradient-flow-can-make","title":"Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again","arxiv_id":"2210.08122","date":"2022-10-14","proceeding":null,"authors":["Ajay Jaiswal","Peihao Wang","Tianlong Chen","Justin F. Rousseau","Ying Ding","Zhangyang Wang"],"abstract":"Despite the enormous success of Graph Convolutional Networks (GCNs) in modeling graph-structured data, most of the current GCNs are shallow due to the notoriously challenging problems of over-smoothening and information squashing along with conventional difficulty caused by vanishing gradients and over-fitting. Previous works have been primarily focused on the study of over-smoothening and over-squashing phenomena in training deep GCNs. Surprisingly, in comparison with CNNs/RNNs, very limited attention has been given to understanding how healthy gradient flow can benefit the trainability of deep GCNs. In this paper, firstly, we provide a new perspective of gradient flow to understand the substandard performance of deep GCNs and hypothesize that by facilitating healthy gradient flow, we can significantly improve their trainability, as well as achieve state-of-the-art (SOTA) level performance from vanilla-GCNs. Next, we argue that blindly adopting the Glorot initialization for GCNs is not optimal, and derive a topology-aware isometric initialization scheme for vanilla-GCNs based on the principles of isometry. Additionally, contrary to ad-hoc addition of skip-connections, we propose to use gradient-guided dynamic rewiring of vanilla-GCNs} with skip connections. Our dynamic rewiring method uses the gradient flow within each layer during training to introduce on-demand skip-connections adaptively. We provide extensive empirical evidence across multiple datasets that our methods improve gradient flow in deep vanilla-GCNs and significantly boost their performance to comfortably compete and outperform many fancy state-of-the-art methods. Codes are available at: https://github.com/VITA-Group/GradientGCN.","url_abs":"https://arxiv.org/abs/2210.08122v1","url_pdf":"https://arxiv.org/pdf/2210.08122v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"old-can-be-gold-better-gradient-flow-can-make","repo_url":"https://github.com/vita-group/gradientgcn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2210.08122","atlas_url":"https://app.syntology.ai/?focus=2210.08122","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.08122"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/VITA-Group/GradientGCN","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vita-group/gradientgcn","reach":{"status":"ok"}}],"summary":{"ran_honours":2,"unverified":4},"by_repo_kind":{"official":{"samples":6,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"5701520c223695f3","entry":"energy","repo":"vita-group/gradientgcn","repo_kind":"official","path":"models/GCN.py","file_url":"https://github.com/vita-group/gradientgcn/blob/HEAD/models/GCN.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5701520c223695f3"}},{"code_sha256_prefix":"e359f7852b0eb48f","entry":"tonp","repo":"vita-group/gradientgcn","repo_kind":"official","path":"models/GCN.py","file_url":"https://github.com/vita-group/gradientgcn/blob/HEAD/models/GCN.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e359f7852b0eb48f"}},{"code_sha256_prefix":"c8ba021d6766b247","entry":"DeepGCN","repo":"vita-group/gradientgcn","repo_kind":"official","path":"rewire_model.py","file_url":"https://github.com/vita-group/gradientgcn/blob/HEAD/rewire_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c8ba021d6766b247"}},{"code_sha256_prefix":"dc41ad7f75682a6d","entry":"energy","repo":"vita-group/gradientgcn","repo_kind":"official","path":"rewire_model.py","file_url":"https://github.com/vita-group/gradientgcn/blob/HEAD/rewire_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc41ad7f75682a6d"}},{"code_sha256_prefix":"e0ef167d233644ef","entry":"get_laplacian_mat","repo":"vita-group/gradientgcn","repo_kind":"official","path":"rewire_model.py","file_url":"https://github.com/vita-group/gradientgcn/blob/HEAD/rewire_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e0ef167d233644ef"}},{"code_sha256_prefix":"7b524945ac9fc063","entry":"tonp","repo":"vita-group/gradientgcn","repo_kind":"official","path":"rewire_model.py","file_url":"https://github.com/vita-group/gradientgcn/blob/HEAD/rewire_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b524945ac9fc063"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}