{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/optimization-of-graph-neural-networks-with","title":"Optimization of Graph Neural Networks with Natural Gradient Descent","arxiv_id":"2008.09624","date":"2020-08-21","proceeding":null,"authors":["Mohammad Rasool Izadi","Yihao Fang","Robert Stevenson","Lizhen Lin"],"abstract":"In this work, we propose to employ information-geometric tools to optimize a graph neural network architecture such as the graph convolutional networks. More specifically, we develop optimization algorithms for the graph-based semi-supervised learning by employing the natural gradient information in the optimization process. This allows us to efficiently exploit the geometry of the underlying statistical model or parameter space for optimization and inference. To the best of our knowledge, this is the first work that has utilized the natural gradient for the optimization of graph neural networks that can be extended to other semi-supervised problems. Efficient computations algorithms are developed and extensive numerical studies are conducted to demonstrate the superior performance of our algorithms over existing algorithms such as ADAM and SGD.","url_abs":"https://arxiv.org/abs/2008.09624v1","url_pdf":"https://arxiv.org/pdf/2008.09624v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"optimization-of-graph-neural-networks-with","repo_url":"https://github.com/russellizadi/ssp","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"graph-neural-network","task_name":"Graph Neural Network"},{"task_slug":"node-classification","task_name":"Node Classification"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"graph-neural-network","method_name":"Graph Neural Network"},{"method_slug":"sgd","method_name":"SGD"},{"method_slug":"sgd-with-momentum","method_name":"SGD with Momentum"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/node-classification-on-citeseer-with-public","task":"Node Classification","dataset":"CiteSeer with Public Split: fixed 20 nodes per class","model":"SSP","rank_in_archive_order":8,"of":40,"metrics":{"Accuracy":"74.28 ± 0.67%"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-citeseer","task":"Node Classification","dataset":"Citeseer","model":"SSP","rank_in_archive_order":5,"of":71,"metrics":{"Accuracy":"80.52 ± 0.14"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-cora","task":"Node Classification","dataset":"Cora","model":"SSP","rank_in_archive_order":1,"of":73,"metrics":{"Accuracy":"90.16% ± 0.59%"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-cora-with-public-split","task":"Node Classification","dataset":"Cora with Public Split: fixed 20 nodes per class","model":"SSP","rank_in_archive_order":25,"of":36,"metrics":{"Accuracy":"82.84 ± 0.87%"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-pubmed-with-public","task":"Node Classification","dataset":"PubMed with Public Split: fixed 20 nodes per class","model":"SSP","rank_in_archive_order":16,"of":37,"metrics":{"Accuracy":"80.06 ± 0.34%"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-pubmed","task":"Node Classification","dataset":"Pubmed","model":"SSP","rank_in_archive_order":11,"of":70,"metrics":{"Accuracy":"89.36 ± 0.57"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2008.09624","atlas_url":"https://app.syntology.ai/?focus=2008.09624","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}