{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-generalization-of-vit-mlp-mixer-to-graphs","title":"A Generalization of ViT/MLP-Mixer to Graphs","arxiv_id":"2212.13350","date":"2022-12-27","proceeding":null,"authors":["Xiaoxin He","Bryan Hooi","Thomas Laurent","Adam Perold","Yann Lecun","Xavier Bresson"],"abstract":"Graph Neural Networks (GNNs) have shown great potential in the field of graph representation learning. Standard GNNs define a local message-passing mechanism which propagates information over the whole graph domain by stacking multiple layers. This paradigm suffers from two major limitations, over-squashing and poor long-range dependencies, that can be solved using global attention but significantly increases the computational cost to quadratic complexity. In this work, we propose an alternative approach to overcome these structural limitations by leveraging the ViT/MLP-Mixer architectures introduced in computer vision. We introduce a new class of GNNs, called Graph ViT/MLP-Mixer, that holds three key properties. First, they capture long-range dependency and mitigate the issue of over-squashing as demonstrated on Long Range Graph Benchmark and TreeNeighbourMatch datasets. Second, they offer better speed and memory efficiency with a complexity linear to the number of nodes and edges, surpassing the related Graph Transformer and expressive GNN models. Third, they show high expressivity in terms of graph isomorphism as they can distinguish at least 3-WL non-isomorphic graphs. We test our architecture on 4 simulated datasets and 7 real-world benchmarks, and show highly competitive results on all of them. The source code is available for reproducibility at: \\url{https://github.com/XiaoxinHe/Graph-ViT-MLPMixer}.","url_abs":"https://arxiv.org/abs/2212.13350v2","url_pdf":"https://arxiv.org/pdf/2212.13350v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-generalization-of-vit-mlp-mixer-to-graphs","repo_url":"https://github.com/XiaoxinHe/Graph-ViT-MLPMixer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"a-generalization-of-vit-mlp-mixer-to-graphs","repo_url":"https://github.com/2023-MindSpore-1/ms-code-20","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":{"status":"ok"}},{"paper_slug":"a-generalization-of-vit-mlp-mixer-to-graphs","repo_url":"https://github.com/asarigun/GraphMixerNetworks","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"graph-classification","task_name":"Graph Classification"},{"task_slug":"graph-regression","task_name":"Graph Regression"},{"task_slug":"graph-representation-learning","task_name":"Graph Representation Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"graph-transformer","method_name":"Graph Transformer"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"lapeigen","method_name":"LapEigen"},{"method_slug":"laplacian-pe","method_name":"Laplacian PE"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"mlp-mixer","method_name":"MLP-Mixer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"speed","method_name":"SPEED"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"test","method_name":"Test"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/graph-classification-on-peptides-func","task":"Graph Classification","dataset":"Peptides-func","model":"Graph ViT","rank_in_archive_order":16,"of":44,"metrics":{"AP":"0.6942±0.0075"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-peptides-func","task":"Graph Classification","dataset":"Peptides-func","model":"GraphMLPMixer","rank_in_archive_order":17,"of":44,"metrics":{"AP":"0.6921±0.0054"},"uses_additional_data":false},{"leaderboard":"/sota/graph-regression-on-peptides-struct","task":"Graph Regression","dataset":"Peptides-struct","model":"Graph ViT","rank_in_archive_order":6,"of":39,"metrics":{"MAE":"0.2449±0.0016"},"uses_additional_data":false},{"leaderboard":"/sota/graph-regression-on-peptides-struct","task":"Graph Regression","dataset":"Peptides-struct","model":"GraphMLPMixer","rank_in_archive_order":15,"of":39,"metrics":{"MAE":"0.2475±0.0015"},"uses_additional_data":false},{"leaderboard":"/sota/graph-regression-on-zinc","task":"Graph Regression","dataset":"ZINC","model":"GraphMLPMixer","rank_in_archive_order":13,"of":27,"metrics":{"MAE":"0.075 ± 0.001"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2212.13350","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.13350"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/XiaoxinHe/Graph-ViT-MLPMixer","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/asarigun/GraphMixerNetworks","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/2023-MindSpore-1/ms-code-20","reach":{"status":"ok"}}],"summary":{"ran":2,"unverified":1},"by_repo_kind":{"listed":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"acd928e182953160","entry":"ResidualModule","repo":"asarigun/GraphMixerNetworks","repo_kind":"listed","path":"gmn_layer.py","file_url":"https://github.com/asarigun/GraphMixerNetworks/blob/HEAD/gmn_layer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"acd928e182953160"}},{"code_sha256_prefix":"a01ab8d795d99737","entry":"TransposeModule","repo":"asarigun/GraphMixerNetworks","repo_kind":"listed","path":"gmn_layer.py","file_url":"https://github.com/asarigun/GraphMixerNetworks/blob/HEAD/gmn_layer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a01ab8d795d99737"}},{"code_sha256_prefix":"cef63c1ed64f69e2","entry":"GMNConv","repo":"asarigun/GraphMixerNetworks","repo_kind":"listed","path":"gmn_layer.py","file_url":"https://github.com/asarigun/GraphMixerNetworks/blob/HEAD/gmn_layer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cef63c1ed64f69e2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}