{"url":"/method/gala","slug":"gala","name":"GALA","full_name":"Global-and-Local attention","full_name_withheld":false,"description_markdown":"Most attention mechanisms learn where to focus using only weak supervisory signals from class labels, which inspired Linsley et al. to investigate how explicit human supervision can affect the performance and interpretability of attention models. As a proof of concept, Linsley et al. proposed the global-and-local attention (GALA) module, which extends an SE block with a spatial attention mechanism.\r\n\r\nGiven the input feature map $X$, GALA uses an attention mask that combines global and local attention to tell the network where and on what to focus. As in SE blocks, global attention aggregates global information by global average pooling and then produces a channel-wise attention weight vector using a multilayer perceptron. In local attention, two consecutive $1\\times 1$ convolutions are conducted on the input to produce a positional weight map. The outputs of the local and global pathways are combined by addition and multiplication. Formally, GALA can be represented as:\r\n\\begin{align}\r\n    s_g &= W_{2} \\delta (W_{1}\\text{GAP}(x))\r\n\\end{align}\r\n\r\n\\begin{align}\r\n    s_l &= Conv_2^{1\\times 1} (\\delta(Conv_1^{1\\times1}(X)))\r\n\\end{align}\r\n\r\n\\begin{align}\r\n    s_g^* &= \\text{Expand}(s_g)\r\n\\end{align}\r\n\r\n\\begin{align}\r\n    s_l^* &= \\text{Expand}(s_l) \r\n\\end{align}\r\n\r\n\\begin{align}\r\n    s &= \\tanh(a(s_g^\\* + s_l^\\*) +m \\cdot (s_g^\\* s_l^\\*) )\r\n\\end{align}\r\n\r\n\\begin{align}\r\n    Y &= sX\r\n\\end{align}\r\n\r\nwhere $a,m \\in \\mathbb{R}^{C}$ are learnable parameters representing channel-wise weight vectors. \r\n\r\nSupervised by human-provided feature importance maps, GALA has significantly improved representational power and can be combined with any CNN backbone.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Learning what and where to attend","paper":"/paper/global-and-local-attention-networks-for","first_author":"Drew Linsley","n_authors":4,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/global-and-local-attention-networks-for"},"source":{"url":"https://arxiv.org/abs/1805.08819v4","title":"Learning what and where to attend","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Attention Mechanisms","url":"/methods/category/attention-mechanisms","pwc_aliases":["attention-mechanisms-1"]}],"n_papers_tagged":9,"archive_num_papers":9,"papers_newest_first":[{"paper":null,"title":"Online Learning-guided Learning Rate Adaptation via Gradient Alignment","date":"2025-06-10","arxiv_id":"2506.08419","n_code_links":0,"syntology":null},{"paper":"/paper/gala-graph-diffusion-based-alignment-with","title":"GALA: Graph Diffusion-based Alignment with Jigsaw for Source-free Domain Adaptation","date":"2024-10-22","arxiv_id":"2410.16606","n_code_links":1,"syntology":null},{"paper":null,"title":"GALA: Geometry-Aware Local Adaptive Grids for Detailed 3D Generation","date":"2024-10-13","arxiv_id":"2410.10037","n_code_links":0,"syntology":null},{"paper":null,"title":"GALA: Generating Animatable Layered Assets from a Single Scan","date":"2024-01-23","arxiv_id":"2401.12979","n_code_links":0,"syntology":null},{"paper":"/paper/does-invariant-graph-learning-via-environment-1","title":"Does Invariant Graph Learning via Environment Augmentation Learn Invariance?","date":"2023-10-29","arxiv_id":"2310.19035","n_code_links":1,"syntology":{"ran":3,"of":3,"unverified":0,"pointer_only":0}},{"paper":null,"title":"Temporal and Contextual Transformer for Multi-Camera Editing of TV Shows","date":"2022-10-17","arxiv_id":"2210.08737","n_code_links":0,"syntology":null},{"paper":null,"title":"GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositing","date":"2022-03-31","arxiv_id":"2204.00125","n_code_links":0,"syntology":null},{"paper":"/paper/gossip-based-actor-learner-architectures-for","title":"Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning","date":"2019-06-09","arxiv_id":"1906.04585","n_code_links":1,"syntology":null},{"paper":"/paper/global-and-local-attention-networks-for","title":"Learning what and where to attend","date":"2018-05-22","arxiv_id":"1805.08819","n_code_links":1,"syntology":null}],"papers_shown":9,"tasks":[{"task":"/task/3d-generation","name":"3D Generation","papers":1},{"task":"/task/3d-geometry","name":"3D geometry","papers":1},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":1},{"task":"/task/diagnostic","name":"Diagnostic","papers":1},{"task":"/task/domain-adaptation","name":"Domain Adaptation","papers":1},{"task":null,"name":"GPU","papers":1},{"task":"/task/graph-domain-adaptation","name":"GRAPH DOMAIN ADAPTATION","papers":1},{"task":"/task/general-knowledge","name":"General Knowledge","papers":1},{"task":"/task/graph-learning","name":"Graph Learning","papers":1},{"task":"/task/graph-neural-network","name":"Graph Neural Network","papers":1},{"task":"/task/graph-representation-learning","name":"Graph Representation Learning","papers":1},{"task":"/task/image-categorization","name":"Image Categorization","papers":1},{"task":"/task/object","name":"Object","papers":1},{"task":"/task/object-recognition","name":"Object Recognition","papers":1},{"task":"/task/out-of-distribution-generalization","name":"Out-of-Distribution Generalization","papers":1},{"task":"/task/quantization","name":"Quantization","papers":1},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":1},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":1},{"task":"/task/representation-learning","name":"Representation Learning","papers":1},{"task":"/task/source-free-domain-adaptation","name":"Source-Free Domain Adaptation","papers":1}],"tasks_shown":20,"n_tasks":21,"usage_by_year":[{"year":"2018","papers":1},{"year":"2019","papers":1},{"year":"2022","papers":2},{"year":"2023","papers":1},{"year":"2024","papers":3},{"year":"2025","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/gala"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}