{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mitigating-degree-biases-in-message-passing","title":"Mitigating Degree Biases in Message Passing Mechanism by Utilizing Community Structures","arxiv_id":"2312.16788","date":"2023-12-28","proceeding":null,"authors":["Van Thuy Hoang","O-Joun Lee"],"abstract":"This study utilizes community structures to address node degree biases in message-passing (MP) via learnable graph augmentations and novel graph transformers. Recent augmentation-based methods showed that MP neural networks often perform poorly on low-degree nodes, leading to degree biases due to a lack of messages reaching low-degree nodes. Despite their success, most methods use heuristic or uniform random augmentations, which are non-differentiable and may not always generate valuable edges for learning representations. In this paper, we propose Community-aware Graph Transformers, namely CGT, to learn degree-unbiased representations based on learnable augmentations and graph transformers by extracting within community structures. We first design a learnable graph augmentation to generate more within-community edges connecting low-degree nodes through edge perturbation. Second, we propose an improved self-attention to learn underlying proximity and the roles of nodes within the community. Third, we propose a self-supervised learning task that could learn the representations to preserve the global graph structure and regularize the graph augmentations. Extensive experiments on various benchmark datasets showed CGT outperforms state-of-the-art baselines and significantly improves the node degree biases. The source code is available at https://github.com/NSLab-CUK/Community-aware-Graph-Transformer.","url_abs":"https://arxiv.org/abs/2312.16788v1","url_pdf":"https://arxiv.org/pdf/2312.16788v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mitigating-degree-biases-in-message-passing","repo_url":"https://github.com/nslab-cuk/community-aware-graph-transformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"node-classification","task_name":"Node Classification"},{"task_slug":"node-clustering","task_name":"Node Clustering"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/node-classification-on-amz-photo","task":"Node Classification","dataset":"AMZ Photo","model":"CGT","rank_in_archive_order":2,"of":14,"metrics":{"Accuracy":"95.73±0.84"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-amazon-computers-1","task":"Node Classification","dataset":"Amazon Computers","model":"CGT","rank_in_archive_order":7,"of":12,"metrics":{"Accuracy":"91.45±0.58"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-citeseer","task":"Node Classification","dataset":"Citeseer","model":"CGT","rank_in_archive_order":11,"of":71,"metrics":{"Accuracy":"76.59±0.98"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-cora","task":"Node Classification","dataset":"Cora","model":"CGT","rank_in_archive_order":17,"of":73,"metrics":{"Accuracy":"87.10±1.53"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-pubmed","task":"Node Classification","dataset":"Pubmed","model":"CGT","rank_in_archive_order":18,"of":70,"metrics":{"Accuracy":"86.86±0.12"},"uses_additional_data":false},{"leaderboard":"/sota/node-classification-on-wiki-cs","task":"Node Classification","dataset":"Wiki-CS","model":"CGT","rank_in_archive_order":1,"of":6,"metrics":{"Accuracy":"84.61±0.53"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}