{"url":"/method/gln","slug":"gln","name":"GLN","full_name":"Gated Linear Network","full_name_withheld":false,"description_markdown":"A **Gated Linear Network**, or **GLN**, is a type of backpropagation-free neural architecture. What distinguishes GLNs from contemporary neural networks is the distributed and local nature of their credit assignment mechanism; each neuron directly predicts the target, forgoing the ability to learn feature representations in favor of rapid online learning. Individual neurons can model nonlinear functions via the use of data-dependent gating in conjunction with online convex optimization. \r\n\r\nGLNs are feedforward networks composed of many layers of gated geometric mixing neurons as shown in the Figure . Each neuron in a given layer outputs a gated geometric mixture of the predictions from the previous layer, with the final layer consisting of just a single neuron. In a supervised learning setting, a $\\mathrm{GLN}$ is trained on (side information, base predictions, label) triplets $\\left(z\\_{t}, p\\_{t}, x\\_{t}\\right)_{t=1,2,3, \\ldots}$ derived from input-label pairs $\\left(z\\_{t}, x\\_{t}\\right)$. There are two types of input to neurons in the network: the first is the side information $z\\_{t}$, which can be thought of as the input features; the second is the input to the neuron, which will be the predictions output by the previous layer, or in the case of layer 0 , some (optionally) provided base predictions $p\\_{t}$ that typically will be a function of $z\\_{t} .$ Each neuron will also take in a constant bias prediction, which helps empirically and is essential for universality guarantees.\r\n\r\nWeights are learnt in a Gated Linear Network using Online Gradient Descent (OGD) locally at each neuron. They key observation is that as each neuron $(i, k)$ in layers $i>0$ is itself a gated geometric mixture, all of these neurons can be thought of as individually predicting the target. Given side information $z$ , each neuron $(i, k)$ suffers a loss convex in its active weights $u:=w\\_{i k c\\_{i k}(z)}$ of\r\n$$\r\n\\ell\\_{t}(u):=-\\log \\left(\\operatorname{GEO}\\_{u}\\left(x_{t} ; p\\_{i-1}\\right)\\right)\r\n$$","description_state":"present","introduced_year":null,"introduced_by":{"title":"Gated Linear Networks","paper":"/paper/gated-linear-networks","first_author":"Joel Veness","n_authors":11,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/gated-linear-networks"},"source":{"url":"https://arxiv.org/abs/1910.01526v2","title":"Gated Linear Networks","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Gated Linear Networks","url":"/methods/category/gated-linear-networks","pwc_aliases":[]}],"n_papers_tagged":8,"archive_num_papers":8,"papers_newest_first":[{"paper":"/paper/noisygl-a-comprehensive-benchmark-for-graph","title":"NoisyGL: A Comprehensive Benchmark for Graph Neural Networks under Label Noise","date":"2024-06-06","arxiv_id":"2406.04299","n_code_links":2,"syntology":null},{"paper":"/paper/semiretro-semi-template-framework-boosts-deep-1","title":"SemiRetro: Semi-template framework boosts deep retrosynthesis prediction","date":"2022-02-12","arxiv_id":"2202.08205","n_code_links":0,"syntology":null},{"paper":"/paper/zero-shot-multi-view-indoor-localization-via","title":"Zero-Shot Multi-View Indoor Localization via Graph Location Networks","date":"2020-08-06","arxiv_id":"2008.02492","n_code_links":1,"syntology":null},{"paper":"/paper/gait-lateral-network-learning-discriminative","title":"Gait Lateral Network: Learning Discriminative and Compact Representations for Gait Recognition","date":"2020-08-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"A Generative Graph Method to Solve the Travelling Salesman Problem","date":"2020-07-09","arxiv_id":"2007.04949","n_code_links":0,"syntology":null},{"paper":"/paper/gaussian-gated-linear-networks","title":"Gaussian Gated Linear Networks","date":"2020-06-10","arxiv_id":"2006.05964","n_code_links":3,"syntology":null},{"paper":null,"title":"Online Learning in Contextual Bandits using Gated Linear Networks","date":"2020-02-21","arxiv_id":"2002.11611","n_code_links":0,"syntology":null},{"paper":"/paper/gated-linear-networks","title":"Gated Linear Networks","date":"2019-09-30","arxiv_id":"1910.01526","n_code_links":2,"syntology":{"ran":3,"of":4,"unverified":1,"pointer_only":4}}],"papers_shown":8,"tasks":[{"task":"/task/graph-learning","name":"Graph Learning","papers":2},{"task":"/task/multi-armed-bandits","name":"Multi-Armed Bandits","papers":2},{"task":"/task/combinatorial-optimization","name":"Combinatorial Optimization","papers":1},{"task":"/task/denoising","name":"Denoising","papers":1},{"task":"/task/density-estimation","name":"Density Estimation","papers":1},{"task":"/task/gait-recognition","name":"Gait Recognition","papers":1},{"task":"/task/image-classification","name":"Image Classification","papers":1},{"task":"/task/indoor-localization","name":"Indoor Localization","papers":1},{"task":"/task/node-classification","name":"Node Classification","papers":1},{"task":"/task/prediction","name":"Prediction","papers":1},{"task":"/task/retrosynthesis","name":"Retrosynthesis","papers":1},{"task":"/task/visual-place-recognition","name":"Visual Place Recognition","papers":1},{"task":"/task/regression-1","name":"regression","papers":1}],"tasks_shown":13,"n_tasks":13,"usage_by_year":[{"year":"2019","papers":1},{"year":"2020","papers":5},{"year":"2022","papers":1},{"year":"2024","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/gln"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}