{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mixture-of-experts-meets-decoupled-message","title":"Mixture of Experts Meets Decoupled Message Passing: Towards General and Adaptive Node Classification","arxiv_id":"2412.08193","date":"2024-12-11","proceeding":null,"authors":["Xuanze Chen","Jiajun Zhou","Shanqing Yu","Qi Xuan"],"abstract":"Graph neural networks excel at graph representation learning but struggle with heterophilous data and long-range dependencies. And graph transformers address these issues through self-attention, yet face scalability and noise challenges on large-scale graphs. To overcome these limitations, we propose GNNMoE, a universal model architecture for node classification. This architecture flexibly combines fine-grained message-passing operations with a mixture-of-experts mechanism to build feature encoding blocks. Furthermore, by incorporating soft and hard gating layers to assign the most suitable expert networks to each node, we enhance the model's expressive power and adaptability to different graph types. In addition, we introduce adaptive residual connections and an enhanced FFN module into GNNMoE, further improving the expressiveness of node representation. Extensive experimental results demonstrate that GNNMoE performs exceptionally well across various types of graph data, effectively alleviating the over-smoothing issue and global noise, enhancing model robustness and adaptability, while also ensuring computational efficiency on large-scale graphs.","url_abs":"https://arxiv.org/abs/2412.08193v1","url_pdf":"https://arxiv.org/pdf/2412.08193v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mixture-of-experts-meets-decoupled-message","repo_url":"https://github.com/GISec-Team/GNNMoE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":"graph-representation-learning","task_name":"Graph Representation Learning"},{"task_slug":"mixture-of-experts","task_name":"Mixture-of-Experts"},{"task_slug":"node-classification","task_name":"Node Classification"},{"task_slug":"node-property-prediction","task_name":"Node Property Prediction"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/node-classification-on-actor","task":"Node Classification","dataset":"Actor","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":11,"of":62,"metrics":{"Accuracy":"37.97±1.01"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-actor","task":"Node Classification","dataset":"Actor","model":"GNNMoE(GAT-like P)","rank_in_archive_order":17,"of":62,"metrics":{"Accuracy":"37.76±0.98"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-actor","task":"Node Classification","dataset":"Actor","model":"GNNMoE(GCN-like P)","rank_in_archive_order":20,"of":62,"metrics":{"Accuracy":"37.59±1.36"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-amazon-computers-1","task":"Node Classification","dataset":"Amazon Computers","model":"GNNMoE(GCN-like P)","rank_in_archive_order":4,"of":12,"metrics":{"Accuracy":"92.17±0.50"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-amazon-computers-1","task":"Node Classification","dataset":"Amazon Computers","model":"GNNMoE(GAT-like P)","rank_in_archive_order":5,"of":12,"metrics":{"Accuracy":"91.98±0.46"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-amazon-computers-1","task":"Node Classification","dataset":"Amazon Computers","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":6,"of":12,"metrics":{"Accuracy":"91.85±0.39"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-amazon-photo-1","task":"Node Classification","dataset":"Amazon Photo","model":"GNNMoE(GCN-like P)","rank_in_archive_order":4,"of":11,"metrics":{"Accuracy":"95.81±0.41"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-amazon-photo-1","task":"Node Classification","dataset":"Amazon Photo","model":"GNNMoE(GAT-like P)","rank_in_archive_order":5,"of":11,"metrics":{"Accuracy":"95.71±0.37"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-amazon-photo-1","task":"Node Classification","dataset":"Amazon Photo","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":6,"of":11,"metrics":{"Accuracy":"95.46±0.24"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-chameleon-48-32-20","task":"Node Classification","dataset":"Chameleon (48%/32%/20% fixed splits)","model":"GNNMoE(GCN-like P)","rank_in_archive_order":2,"of":4,"metrics":{"Accuracy":"47.19±2.93"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-chameleon-48-32-20","task":"Node Classification","dataset":"Chameleon (48%/32%/20% fixed splits)","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":3,"of":4,"metrics":{"Accuracy":"45.73±3.19"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-chameleon-48-32-20","task":"Node Classification","dataset":"Chameleon (48%/32%/20% fixed splits)","model":"GNNMoE(GAT-like P)","rank_in_archive_order":4,"of":4,"metrics":{"Accuracy":"45.56±3.94"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-coauthor-cs","task":"Node Classification","dataset":"Coauthor CS","model":"GNNMoE(GCN-like P)","rank_in_archive_order":5,"of":24,"metrics":{"Accuracy":"95.81±0.26"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-coauthor-cs","task":"Node Classification","dataset":"Coauthor CS","model":"GNNMoE(GAT-like P)","rank_in_archive_order":8,"of":24,"metrics":{"Accuracy":"95.72±0.23"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-coauthor-cs","task":"Node Classification","dataset":"Coauthor CS","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":10,"of":24,"metrics":{"Accuracy":"95.68±0.24"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-coauthor-physics","task":"Node Classification","dataset":"Coauthor Physics","model":"GNNMoE(GAT-like P)","rank_in_archive_order":6,"of":14,"metrics":{"Accuracy":"97.05±0.19"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-coauthor-physics","task":"Node Classification","dataset":"Coauthor Physics","model":"GNNMoE(GCN-like P)","rank_in_archive_order":7,"of":14,"metrics":{"Accuracy":"97.03±0.13"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-coauthor-physics","task":"Node Classification","dataset":"Coauthor Physics","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":12,"of":14,"metrics":{"Accuracy":"96.81±0.22"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-facebook","task":"Node Classification","dataset":"Facebook","model":"GNNMoE(GCN-like P)","rank_in_archive_order":1,"of":8,"metrics":{"Accuracy":"95.53±0.35"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-facebook","task":"Node Classification","dataset":"Facebook","model":"GNNMoE(GAT-like P)","rank_in_archive_order":2,"of":8,"metrics":{"Accuracy":"95.21±0.25"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-facebook","task":"Node Classification","dataset":"Facebook","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":3,"of":8,"metrics":{"Accuracy":"94.63±0.36"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-penn94","task":"Node Classification","dataset":"Penn94","model":"GNNMoE(GCN-like P)","rank_in_archive_order":6,"of":32,"metrics":{"Accuracy":"85.11±0.39"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-penn94","task":"Node Classification","dataset":"Penn94","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":12,"of":32,"metrics":{"Accuracy":"84.05±0.37"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-penn94","task":"Node Classification","dataset":"Penn94","model":"GNNMoE(GAT-like P)","rank_in_archive_order":16,"of":32,"metrics":{"Accuracy":"81.98±0.47"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-squirrel-48-32-20","task":"Node Classification","dataset":"Squirrel (48%/32%/20% fixed splits)","model":"GNNMoE(GCN-like P)","rank_in_archive_order":2,"of":4,"metrics":{"Accuracy":"44.02±2.59"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-squirrel-48-32-20","task":"Node Classification","dataset":"Squirrel (48%/32%/20% fixed splits)","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":3,"of":4,"metrics":{"Accuracy":"39.19±2.84"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-squirrel-48-32-20","task":"Node Classification","dataset":"Squirrel (48%/32%/20% fixed splits)","model":"GNNMoE(GAT-like P)","rank_in_archive_order":4,"of":4,"metrics":{"Accuracy":"39.19±3.94"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-roman-empire","task":"Node Classification","dataset":"roman-empire","model":"GNNMoE(GAT-like P)","rank_in_archive_order":5,"of":7,"metrics":{"Accuracy (% )":"87.29±0.60"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-roman-empire","task":"Node Classification","dataset":"roman-empire","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":6,"of":7,"metrics":{"Accuracy (% )":"86.00±0.45"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-roman-empire","task":"Node Classification","dataset":"roman-empire","model":"GNNMoE(GCN-like P)","rank_in_archive_order":7,"of":7,"metrics":{"Accuracy (% )":"85.05±0.55"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-tolokers","task":"Node Classification","dataset":"tolokers","model":"GNNMoE(GAT-like P)","rank_in_archive_order":2,"of":4,"metrics":{"AUCROC":"85.45±0.94"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-tolokers","task":"Node Classification","dataset":"tolokers","model":"GNNMoE(GCN-like P)","rank_in_archive_order":3,"of":4,"metrics":{"AUCROC":"84.77±0.93"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-tolokers","task":"Node Classification","dataset":"tolokers","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":4,"of":4,"metrics":{"AUCROC":"83.96±0.75"},"uses_additional_data":false},{"leaderboard":"/sota/node-property-prediction-on-ogbn-arxiv","task":"Node Property Prediction","dataset":"ogbn-arxiv","model":"GNNMoE(GAT-like P)","rank_in_archive_order":60,"of":86,"metrics":{"Test Accuracy":"0.7245±0.0032"},"uses_additional_data":false},{"leaderboard":"/sota/node-property-prediction-on-ogbn-arxiv","task":"Node Property Prediction","dataset":"ogbn-arxiv","model":"GNNMoE(GCN-like P)","rank_in_archive_order":61,"of":86,"metrics":{"Test Accuracy":"0.7229±0.0016"},"uses_additional_data":false},{"leaderboard":"/sota/node-property-prediction-on-ogbn-arxiv","task":"Node Property Prediction","dataset":"ogbn-arxiv","model":"GNNMoE(SAGE-like P)","rank_in_archive_order":73,"of":86,"metrics":{"Test Accuracy":"0.7194±0.0025"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}