{"url":"/method/lightconv","slug":"lightconv","name":"LightConv","full_name":"Lightweight Convolution","full_name_withheld":false,"description_markdown":"**LightConv** is a type of [depthwise convolution](https://paperswithcode.com/method/depthwise-convolution) for sequential modelling which shares certain output channels and whose weights are normalized across the temporal dimension using a [softmax](https://paperswithcode.com/method/softmax). Compared to self-attention, LightConv has a fixed context window and it determines the importance of context elements with a set of weights that do not change over time steps. LightConv computes the following for the $i$-th element in the sequence and output channel $c$:\r\n\r\n$$ \\text{LightConv}\\left(X, W\\_{\\text{ceil}\\left(\\frac{cH}{d}\\right),:}, i, c\\right) = \\text{DepthwiseConv}\\left(X,\\text{softmax}\\left(W\\_{\\text{ceil}\\left(\\frac{cH}{d}\\right),:}\\right), i, c\\right) $$","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1901.10430v2","title":"Pay Less Attention with Lightweight and Dynamic Convolutions","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/pytorch/fairseq/blob/28876638114948711fd4bd4e350fdd6809013f1e/fairseq/modules/lightweight_convolution.py#L33","code_snippet_url_on_a_code_host":true,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Convolutions","url":"/methods/category/convolutions","pwc_aliases":[]}],"n_papers_tagged":1,"archive_num_papers":null,"papers_newest_first":[{"paper":"/paper/pay-less-attention-with-lightweight-and","title":"Pay Less Attention with Lightweight and Dynamic Convolutions","date":"2019-01-29","arxiv_id":"1901.10430","n_code_links":3,"syntology":null}],"papers_shown":1,"tasks":[{"task":"/task/abstractive-text-summarization","name":"Abstractive Text Summarization","papers":1},{"task":"/task/language-modeling","name":"Language Modeling","papers":1},{"task":"/task/language-modelling","name":"Language Modelling","papers":1},{"task":"/task/machine-translation","name":"Machine Translation","papers":1},{"task":"/task/translation","name":"Translation","papers":1}],"tasks_shown":5,"n_tasks":5,"usage_by_year":[{"year":"2019","papers":1}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/lightconv"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}