{"url":"/method/span-based-dynamic-convolution","slug":"span-based-dynamic-convolution","name":"Span-Based Dynamic Convolution","full_name":"Span-Based Dynamic Convolution","full_name_withheld":false,"description_markdown":"**Span-Based Dynamic Convolution** is a type of convolution used in the [ConvBERT](https://paperswithcode.com/method/convbert) architecture to capture local dependencies between tokens.  Kernels are generated by taking in a local span of current token, which better utilizes local dependency and discriminates different meanings of the same token (e.g., if “a” is in front of “can” in the input sentence, “can” is apparently a noun not a verb).\r\n\r\nSpecifically, with [classic convolution](https://paperswithcode.com/method/convolution), we would have fixed parameters shared for all input tokens. [Dynamic convolution](https://paperswithcode.com/method/dynamicconv) is therefore preferable because it has  higher flexibility in capturing local dependencies of different tokens. Dynamic convolution uses a kernel generator to produce different kernels for different input tokens. However, such dynamic convolution cannot differentiate the same tokens within different context and\r\ngenerate the same kernels (e.g., the three “can” in Figure (b)).\r\n\r\nTherefore the span-based dynamic convolution is developed to produce more adaptive convolution kernels by receiving an input span instead of only a single token, which enables discrimination of generated kernels for the same tokens within different context. For example, as shown in Figure (c), span-based dynamic convolution produces different kernels for different “can” tokens.","description_state":"present","introduced_year":null,"introduced_by":{"title":"ConvBERT: Improving BERT with Span-based Dynamic Convolution","paper":"/paper/convbert-improving-bert-with-span-based","first_author":"Zi-Hang Jiang","n_authors":6,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/convbert-improving-bert-with-span-based"},"source":{"url":"https://arxiv.org/abs/2008.02496v3","title":"ConvBERT: Improving BERT with Span-based Dynamic Convolution","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Sequential","area_id":"sequential","collection":"Temporal Convolutions","url":"/methods/category/temporal-convolutions","pwc_aliases":[]},{"area":"Computer Vision","area_id":"computer-vision","collection":"Convolutions","url":"/methods/category/convolutions","pwc_aliases":[]}],"n_papers_tagged":6,"archive_num_papers":6,"papers_newest_first":[{"paper":null,"title":"Beyond Simple Concatenation: Fairly Assessing PLM Architectures for Multi-Chain Protein-Protein Interactions Prediction","date":"2025-05-26","arxiv_id":"2505.20036","n_code_links":0,"syntology":null},{"paper":"/paper/navigating-nuance-in-quest-for-political","title":"Navigating Nuance: In Quest for Political Truth","date":"2025-01-01","arxiv_id":"2501.00782","n_code_links":1,"syntology":null},{"paper":null,"title":"ChatGPT v.s. Media Bias: A Comparative Study of GPT-3.5 and Fine-tuned Language Models","date":"2024-03-29","arxiv_id":"2403.20158","n_code_links":0,"syntology":null},{"paper":"/paper/m-3-net-multilevel-mixed-and-multistage","title":"M$^3$Net: Multilevel, Mixed and Multistage Attention Network for Salient Object Detection","date":"2023-09-15","arxiv_id":"2309.08365","n_code_links":1,"syntology":null},{"paper":"/paper/transformer-based-punctuation-restoration-for","title":"Transformer Based Punctuation Restoration for Turkish","date":"2023-09-15","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/convbert-improving-bert-with-span-based","title":"ConvBERT: Improving BERT with Span-based Dynamic Convolution","date":"2020-08-06","arxiv_id":"2008.02496","n_code_links":8,"syntology":null}],"papers_shown":6,"tasks":[{"task":"/task/automatic-speech-recognition-2","name":"Automatic Speech Recognition","papers":1},{"task":"/task/automatic-speech-recognition","name":"Automatic Speech Recognition (ASR)","papers":1},{"task":"/task/bias-detection","name":"Bias Detection","papers":1},{"task":"/task/drug-discovery","name":"Drug Discovery","papers":1},{"task":"/task/language-modeling","name":"Language Modeling","papers":1},{"task":"/task/language-modelling","name":"Language Modelling","papers":1},{"task":"/task/misinformation","name":"Misinformation","papers":1},{"task":"/task/natural-language-understanding","name":"Natural Language Understanding","papers":1},{"task":"/task/object-detection","name":"Object Detection","papers":1},{"task":"/task/punctuation-restoration","name":"Punctuation Restoration","papers":1},{"task":"/task/salient-object-detection","name":"RGB Salient Object Detection","papers":1},{"task":"/task/salient-object-detection-1","name":"Salient Object Detection","papers":1},{"task":"/task/speech-recognition","name":"Speech Recognition","papers":1},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":1},{"task":"/task/object-detection-1","name":"object-detection","papers":1},{"task":"/task/speech-recognition-1","name":"speech-recognition","papers":1}],"tasks_shown":16,"n_tasks":16,"usage_by_year":[{"year":"2020","papers":1},{"year":"2023","papers":2},{"year":"2024","papers":1},{"year":"2025","papers":2}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/span-based-dynamic-convolution"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}