{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/graph-external-attention-enhanced-transformer","title":"Graph External Attention Enhanced Transformer","arxiv_id":"2405.21061","date":"2024-05-31","proceeding":null,"authors":["Jianqing Liang","Min Chen","Jiye Liang"],"abstract":"The Transformer architecture has recently gained considerable attention in the field of graph representation learning, as it naturally overcomes several limitations of Graph Neural Networks (GNNs) with customized attention mechanisms or positional and structural encodings. Despite making some progress, existing works tend to overlook external information of graphs, specifically the correlation between graphs. Intuitively, graphs with similar structures should have similar representations. Therefore, we propose Graph External Attention (GEA) -- a novel attention mechanism that leverages multiple external node/edge key-value units to capture inter-graph correlations implicitly. On this basis, we design an effective architecture called Graph External Attention Enhanced Transformer (GEAET), which integrates local structure and global interaction information for more comprehensive graph representations. Extensive experiments on benchmark datasets demonstrate that GEAET achieves state-of-the-art empirical performance. The source code is available for reproducibility at: https://github.com/icm1018/GEAET.","url_abs":"https://arxiv.org/abs/2405.21061v2","url_pdf":"https://arxiv.org/pdf/2405.21061v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"graph-external-attention-enhanced-transformer","repo_url":"https://github.com/icm1018/geaet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"graph-representation-learning","task_name":"Graph Representation Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.21061","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.21061"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/icm1018/GEAET","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/icm1018/geaet","reach":{"status":"ok"}}],"summary":{"ran_violates":2,"ran":3,"unverified":6},"by_repo_kind":{"official":{"samples":11,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":"icm1018/GEAET","repo_kind":"official","path":"GEAET/layer/performer_layer.py","file_url":"https://github.com/icm1018/GEAET/blob/HEAD/GEAET/layer/performer_layer.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"19925529d217754e","entry":"empty","repo":"icm1018/GEAET","repo_kind":"official","path":"GEAET/layer/performer_layer.py","file_url":"https://github.com/icm1018/GEAET/blob/HEAD/GEAET/layer/performer_layer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19925529d217754e"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"icm1018/GEAET","repo_kind":"official","path":"GEAET/layer/performer_layer.py","file_url":"https://github.com/icm1018/GEAET/blob/HEAD/GEAET/layer/performer_layer.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"7220304f00385e02","entry":"external_norm","repo":"icm1018/GEAET","repo_kind":"official","path":"GEAET/layer/external_layer.py","file_url":"https://github.com/icm1018/GEAET/blob/HEAD/GEAET/layer/external_layer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7220304f00385e02"}},{"code_sha256_prefix":"1bb331bf70a78136","entry":"get_final_pretrained_ckpt","repo":"icm1018/GEAET","repo_kind":"official","path":"GEAET/finetuning.py","file_url":"https://github.com/icm1018/GEAET/blob/HEAD/GEAET/finetuning.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1bb331bf70a78136"}},{"code_sha256_prefix":"ae56bcfaae96da3c","entry":"apply_chunking_to_forward","repo":"icm1018/GEAET","repo_kind":"official","path":"GEAET/layer/bigbird_layer.py","file_url":"https://github.com/icm1018/GEAET/blob/HEAD/GEAET/layer/bigbird_layer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ae56bcfaae96da3c"}},{"code_sha256_prefix":"3db4474abe13fdee","entry":"init_model_from_pretrained","repo":"icm1018/GEAET","repo_kind":"official","path":"GEAET/finetuning.py","file_url":"https://github.com/icm1018/GEAET/blob/HEAD/GEAET/finetuning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3db4474abe13fdee"}},{"code_sha256_prefix":"c98a702be0660ac9","entry":"is_seed","repo":"icm1018/GEAET","repo_kind":"official","path":"GEAET/agg_runs.py","file_url":"https://github.com/icm1018/GEAET/blob/HEAD/GEAET/agg_runs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c98a702be0660ac9"}},{"code_sha256_prefix":"438635c08adfe2aa","entry":"is_split","repo":"icm1018/GEAET","repo_kind":"official","path":"GEAET/agg_runs.py","file_url":"https://github.com/icm1018/GEAET/blob/HEAD/GEAET/agg_runs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"438635c08adfe2aa"}},{"code_sha256_prefix":"40e98e1ecbf39f34","entry":"join_list","repo":"icm1018/GEAET","repo_kind":"official","path":"GEAET/agg_runs.py","file_url":"https://github.com/icm1018/GEAET/blob/HEAD/GEAET/agg_runs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"40e98e1ecbf39f34"}},{"code_sha256_prefix":"d796cdea5889c847","entry":"load_pretrained_model_cfg","repo":"icm1018/GEAET","repo_kind":"official","path":"GEAET/finetuning.py","file_url":"https://github.com/icm1018/GEAET/blob/HEAD/GEAET/finetuning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d796cdea5889c847"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}