{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diffusing-graph-attention","title":"Diffusing Graph Attention","arxiv_id":"2303.00613","date":"2023-03-01","proceeding":null,"authors":["Daniel Glickman","Eran Yahav"],"abstract":"The dominant paradigm for machine learning on graphs uses Message Passing Graph Neural Networks (MP-GNNs), in which node representations are updated by aggregating information in their local neighborhood. Recently, there have been increasingly more attempts to adapt the Transformer architecture to graphs in an effort to solve some known limitations of MP-GNN. A challenging aspect of designing Graph Transformers is integrating the arbitrary graph structure into the architecture. We propose Graph Diffuser (GD) to address this challenge. GD learns to extract structural and positional relationships between distant nodes in the graph, which it then uses to direct the Transformer's attention and node representation. We demonstrate that existing GNNs and Graph Transformers struggle to capture long-range interactions and how Graph Diffuser does so while admitting intuitive visualizations. Experiments on eight benchmarks show Graph Diffuser to be a highly competitive model, outperforming the state-of-the-art in a diverse set of domains.","url_abs":"https://arxiv.org/abs/2303.00613v1","url_pdf":"https://arxiv.org/pdf/2303.00613v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"graph-attention","task_name":"Graph Attention"},{"task_slug":"graph-classification","task_name":"Graph Classification"},{"task_slug":"graph-regression","task_name":"Graph Regression"},{"task_slug":"link-prediction","task_name":"Link Prediction"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/graph-classification-on-peptides-func","task":"Graph Classification","dataset":"Peptides-func","model":"Graph Diffuser","rank_in_archive_order":26,"of":44,"metrics":{"AP":"0.6651±0.0010"},"uses_additional_data":false},{"leaderboard":"/sota/graph-regression-on-peptides-struct","task":"Graph Regression","dataset":"Peptides-struct","model":"Graph Diffuser","rank_in_archive_order":12,"of":39,"metrics":{"MAE":"0.2461±0.0010"},"uses_additional_data":false},{"leaderboard":"/sota/link-prediction-on-pcqm-contact","task":"Link Prediction","dataset":"PCQM-Contact","model":"Graph Diffuser","rank_in_archive_order":2,"of":18,"metrics":{"Hits@1":"0.1369±0.0012","Hits@10":"0.8592±0.0007","Hits@3":"0.4053±0.0011","MRR":"0.3388±0.0011"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.00613","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}