{"url":"/method/graph-self-attention","slug":"graph-self-attention","name":"Graph Self-Attention","full_name":"Graph Self-Attention","full_name_withheld":false,"description_markdown":"**Graph Self-Attention (GSA)** is a self-attention module used in the [BP-Transformer](https://paperswithcode.com/method/bp-transformer) architecture, and is based on the [graph attentional layer](https://paperswithcode.com/method/graph-attentional-layer).\r\n\r\nFor a given node $u$, we update its representation according to its neighbour nodes, formulated as $\\mathbf{h}\\_{u} \\leftarrow \\text{GSA}\\left(\\mathcal{G}, \\mathbf{h}^{u}\\right)$.\r\n\r\nLet $\\mathbf{A}\\left(u\\right)$ denote the set of the neighbour nodes of $u$ in $\\mathcal{G}$, $\\text{GSA}\\left(\\mathcal{G}, \\mathbf{h}^{u}\\right)$ is detailed as follows:\r\n\r\n$$ \\mathbf{A}^{u} = \\text{concat}\\left(\\{\\mathbf{h}\\_{v} | v \\in \\mathcal{A}\\left(u\\right)\\}\\right) $$\r\n\r\n$$ \\mathbf{Q}^{u}\\_{i} = \\mathbf{H}\\_{k}\\mathbf{W}^{Q}\\_{i},\\mathbf{K}\\_{i}^{u} = \\mathbf{A}^{u}\\mathbf{W}^{K}\\_{i},\\mathbf{V}^{u}\\_{i} = \\mathbf{A}^{u}\\mathbf{W}\\_{i}^{V} $$\r\n\r\n$$ \\text{head}^{u}\\_{i} = \\text{softmax}\\left(\\frac{\\mathbf{Q}^{u}\\_{i}\\mathbf{K}\\_{i}^{uT}}{\\sqrt{d}}\\right)\\mathbf{V}\\_{i}^{u} $$\r\n\r\n$$ \\text{GSA}\\left(\\mathcal{G}, \\mathbf{h}^{u}\\right) = \\left[\\text{head}^{u}\\_{1}, \\dots, \\text{head}^{u}\\_{h}\\right]\\mathbf{W}^{O}$$\r\n\r\nwhere d is the dimension of h, and $\\mathbf{W}^{Q}\\_{i}$, $\\mathbf{W}^{K}\\_{i}$ and $\\mathbf{W}^{V}\\_{i}$ are trainable parameters of the $i$-th attention head.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://arxiv.org/abs/1911.04070v1","title":"BP-Transformer: Modelling Long-Range Context via Binary Partitioning","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/yzh119/BPT/blob/fa87c1c993eac4e4ad9997a60fc0052dcfc075da/modules/attention.py#L22","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Attention Modules","url":"/methods/category/attention-modules","pwc_aliases":[]}],"n_papers_tagged":64,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"The Hype Index: an NLP-driven Measure of Market News Attention","date":"2025-05-30","arxiv_id":"2506.06329","n_code_links":0,"syntology":null},{"paper":null,"title":"Model Editing with Graph-Based External Memory","date":"2025-05-23","arxiv_id":"2505.18343","n_code_links":0,"syntology":null},{"paper":null,"title":"Exploring the Innovation Opportunities for Pre-trained Models","date":"2025-05-21","arxiv_id":"2505.15790","n_code_links":0,"syntology":null},{"paper":null,"title":"RL in Name Only? Analyzing the Structural Assumptions in RL post-training for LLMs","date":"2025-05-19","arxiv_id":"2505.13697","n_code_links":0,"syntology":null},{"paper":null,"title":"Comparative Analysis of Evolutionary Algorithms for Energy-Aware Production Scheduling","date":"2025-04-22","arxiv_id":"2504.15672","n_code_links":0,"syntology":null},{"paper":null,"title":"Beyond the Hype: Embeddings vs. Prompting for Multiclass Classification Tasks","date":"2025-04-05","arxiv_id":"2504.04277","n_code_links":0,"syntology":null},{"paper":null,"title":"What the F*ck Is Artificial General Intelligence?","date":"2025-03-31","arxiv_id":"2503.23923","n_code_links":0,"syntology":null},{"paper":"/paper/bridging-evolutionary-multiobjective","title":"Bridging Evolutionary Multiobjective Optimization and GPU Acceleration via Tensorization","date":"2025-03-26","arxiv_id":"2503.20286","n_code_links":2,"syntology":null},{"paper":"/paper/guidelines-for-the-choice-of-the-baseline-in","title":"Guidelines For The Choice Of The Baseline in XAI Attribution Methods","date":"2025-03-25","arxiv_id":"2503.19813","n_code_links":1,"syntology":null},{"paper":null,"title":"The Imitation Game According To Turing","date":"2025-01-29","arxiv_id":"2501.17629","n_code_links":0,"syntology":null},{"paper":"/paper/why-these-documents-explainable-generative","title":"Why These Documents? Explainable Generative Retrieval with Hierarchical Category Paths","date":"2024-11-08","arxiv_id":"2411.05572","n_code_links":1,"syntology":null},{"paper":null,"title":"Danoliteracy of Generative, Large Language Models","date":"2024-10-30","arxiv_id":"2410.22839","n_code_links":0,"syntology":null},{"paper":null,"title":"Misrepresented Technological Solutions in Imagined Futures: The Origins and Dangers of AI Hype in the Research Community","date":"2024-08-08","arxiv_id":"2408.15244","n_code_links":0,"syntology":null},{"paper":null,"title":"Decoding Knowledge Claims: The Evaluation of Scientific Publication Contributions through Semantic Analysis","date":"2024-07-26","arxiv_id":"2407.18646","n_code_links":0,"syntology":null},{"paper":null,"title":"Continuous fake media detection: adapting deepfake detectors to new generative techniques","date":"2024-06-12","arxiv_id":"2406.08171","n_code_links":0,"syntology":null},{"paper":"/paper/hype-hyperbolic-entailment-filtering-for","title":"HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Texts","date":"2024-04-26","arxiv_id":"2404.17507","n_code_links":1,"syntology":null},{"paper":null,"title":"AI Safety: Necessary, but insufficient and possibly problematic","date":"2024-03-26","arxiv_id":"2403.17419","n_code_links":0,"syntology":null},{"paper":"/paper/sghormer-an-energy-saving-graph-transformer","title":"SGHormer: An Energy-Saving Graph Transformer Driven by Spikes","date":"2024-03-26","arxiv_id":"2403.17656","n_code_links":1,"syntology":null},{"paper":null,"title":"The Interplay of Learning, Analytics, and Artificial Intelligence in Education: A Vision for Hybrid Intelligence","date":"2024-03-24","arxiv_id":"2403.16081","n_code_links":0,"syntology":null},{"paper":null,"title":"Are you a robot? Detecting Autonomous Vehicles from Behavior Analysis","date":"2024-03-14","arxiv_id":"2403.09571","n_code_links":0,"syntology":null},{"paper":null,"title":"Assessing the Reasoning Abilities of ChatGPT in the Context of Claim Verification","date":"2024-02-16","arxiv_id":"2402.10735","n_code_links":0,"syntology":null},{"paper":"/paper/masked-attention-is-all-you-need-for-graphs","title":"An end-to-end attention-based approach for learning on graphs","date":"2024-02-16","arxiv_id":"2402.10793","n_code_links":1,"syntology":null},{"paper":null,"title":"Investigating Out-of-Distribution Generalization of GNNs: An Architecture Perspective","date":"2024-02-13","arxiv_id":"2402.08228","n_code_links":0,"syntology":null},{"paper":"/paper/lightgcn-evaluated-and-enhanced","title":"LightGCN: Evaluated and Enhanced","date":"2023-12-17","arxiv_id":"2312.16183","n_code_links":1,"syntology":null},{"paper":null,"title":"Neither hype nor gloom do DNNs justice","date":"2023-12-08","arxiv_id":"2312.05355","n_code_links":0,"syntology":null},{"paper":null,"title":"Hypothesis Network Planned Exploration for Rapid Meta-Reinforcement Learning Adaptation","date":"2023-11-07","arxiv_id":"2311.03701","n_code_links":0,"syntology":null},{"paper":null,"title":"HyPE: Attention with Hyperbolic Biases for Relative Positional Encoding","date":"2023-10-30","arxiv_id":"2310.19676","n_code_links":0,"syntology":null},{"paper":null,"title":"Web3 Meets AI Marketplace: Exploring Opportunities, Analyzing Challenges, and Suggesting Solutions","date":"2023-10-29","arxiv_id":"2310.19099","n_code_links":0,"syntology":null},{"paper":null,"title":"Does Artificial Intelligence benefit UK businesses? An empirical study of the impact of AI on productivity","date":"2023-10-06","arxiv_id":"2310.05985","n_code_links":0,"syntology":null},{"paper":"/paper/circle-feature-graphormer-can-circle-features","title":"Circle Feature Graphormer: Can Circle Features Stimulate Graph Transformer?","date":"2023-09-11","arxiv_id":"2309.06574","n_code_links":1,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/graph-attention","name":"Graph Attention","papers":3},{"task":"/task/graph-classification","name":"Graph Classification","papers":3},{"task":"/task/representation-learning","name":"Representation Learning","papers":3},{"task":"/task/sentence","name":"Sentence","papers":3},{"task":"/task/translation","name":"Translation","papers":3},{"task":"/task/abstractive-dialogue-summarization","name":"Abstractive Dialogue Summarization","papers":2},{"task":"/task/anomaly-detection","name":"Anomaly Detection","papers":2},{"task":"/task/articles","name":"Articles","papers":2},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":2},{"task":"/task/machine-learning","name":"BIG-bench Machine Learning","papers":2},{"task":"/task/graph-regression","name":"Graph Regression","papers":2},{"task":"/task/graph-representation-learning","name":"Graph Representation Learning","papers":2},{"task":"/task/language-modeling","name":"Language Modeling","papers":2},{"task":"/task/language-modelling","name":"Language Modelling","papers":2},{"task":"/task/machine-translation","name":"Machine Translation","papers":2},{"task":"/task/molecular-property-prediction","name":"Molecular Property Prediction","papers":2},{"task":"/task/sentiment-analysis","name":"Sentiment Analysis","papers":2},{"task":"/task/aspect-based-sentiment-analysis","name":"Aspect-Based Sentiment Analysis (ABSA)","papers":1},{"task":"/task/attribute","name":"Attribute","papers":1},{"task":"/task/autonomous-vehicles","name":"Autonomous Vehicles","papers":1}],"tasks_shown":20,"n_tasks":86,"usage_by_year":[{"year":"2019","papers":1},{"year":"2020","papers":4},{"year":"2021","papers":9},{"year":"2022","papers":10},{"year":"2023","papers":17},{"year":"2024","papers":13},{"year":"2025","papers":10}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/graph-self-attention"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}