{"url":"/method/location-sensitive-attention","slug":"location-sensitive-attention","name":"Location Sensitive Attention","full_name":"Location Sensitive Attention","full_name_withheld":false,"description_markdown":"**Location Sensitive Attention** is an attention mechanism that extends the [additive attention mechanism](https://paperswithcode.com/method/additive-attention) to use cumulative attention weights from previous decoder time steps as an additional feature. This encourages the model to move forward consistently through the input, mitigating potential failure modes where some subsequences are repeated or ignored by the decoder.\r\n\r\nStarting with additive attention where $h$ is a sequential representation from a BiRNN encoder and ${s}\\_{i-1}$ is the $(i − 1)$-th state of a recurrent neural network (e.g. a [LSTM](https://paperswithcode.com/method/lstm) or [GRU](https://paperswithcode.com/method/gru)):\r\n\r\n$$ e\\_{i, j} = w^{T}\\tanh\\left(W{s}\\_{i-1} + Vh\\_{j} + b\\right) $$\r\n\r\nwhere $w$ and $b$ are vectors, $W$ and $V$ are matrices. We extend this to be location-aware by making it take into account the alignment produced at the previous step. First, we extract $k$ vectors\r\n$f\\_{i,j} \\in \\mathbb{R}^{k}$ for every position $j$ of the previous alignment $\\alpha\\_{i−1}$ by convolving it with a matrix $F \\in R^{k\\times{r}}$:\r\n\r\n$$ f\\_{i} = F ∗ \\alpha\\_{i−1} $$\r\n\r\nThese additional vectors $f\\_{i,j}$ are then used by the scoring mechanism $e\\_{i,j}$:\r\n\r\n$$ e\\_{i,j} = w^{T}\\tanh\\left(Ws\\_{i−1} + Vh\\_{j} + Uf\\_{i,j} + b\\right) $$","description_state":"present","introduced_year":null,"introduced_by":{"title":"Attention-Based Models for Speech Recognition","paper":"/paper/attention-based-models-for-speech-recognition","first_author":"Jan Chorowski","n_authors":5,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/attention-based-models-for-speech-recognition"},"source":{"url":"http://arxiv.org/abs/1506.07503v1","title":"Attention-Based Models for Speech Recognition","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/begeekmyfriend/tacotron2/blob/cbafb1dd8c79e9dcc0ce2fd6f756dbfb1c1153da/tacotron2/model.py#L58","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Attention Mechanisms","url":"/methods/category/attention-mechanisms","pwc_aliases":["attention-mechanisms-1"]}],"n_papers_tagged":25,"archive_num_papers":25,"papers_newest_first":[{"paper":null,"title":"Training Universal Vocoders with Feature Smoothing-Based Augmentation Methods for High-Quality TTS Systems","date":"2024-09-04","arxiv_id":"2409.02517","n_code_links":0,"syntology":null},{"paper":null,"title":"An overview of text-to-speech systems and media applications","date":"2023-10-22","arxiv_id":"2310.14301","n_code_links":0,"syntology":null},{"paper":null,"title":"Energy-Based Models For Speech Synthesis","date":"2023-10-19","arxiv_id":"2310.12765","n_code_links":0,"syntology":null},{"paper":"/paper/multilingual-text-to-speech-synthesis-for","title":"Multilingual Text-to-Speech Synthesis for Turkic Languages Using Transliteration","date":"2023-05-25","arxiv_id":"2305.15749","n_code_links":1,"syntology":null},{"paper":null,"title":"ArmanTTS single-speaker Persian dataset","date":"2023-04-07","arxiv_id":"2304.03585","n_code_links":0,"syntology":null},{"paper":"/paper/facial-landmark-predictions-with-applications","title":"Facial Landmark Predictions with Applications to Metaverse","date":"2022-09-29","arxiv_id":"2209.14698","n_code_links":1,"syntology":null},{"paper":null,"title":"Zero-Shot Long-Form Voice Cloning with Dynamic Convolution Attention","date":"2022-01-25","arxiv_id":"2201.10375","n_code_links":0,"syntology":null},{"paper":"/paper/itacotron-2-transfering-english-speech","title":"ITAcotron 2: Transfering English Speech Synthesis Architectures and Speech Features to Italian","date":"2021-11-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Neural Sequence-to-Sequence Speech Synthesis Using a Hidden Semi-Markov Model Based Structured Attention Mechanism","date":"2021-08-31","arxiv_id":"2108.13985","n_code_links":0,"syntology":null},{"paper":"/paper/neural-hmms-are-all-you-need-for-high-quality","title":"Neural HMMs are all you need (for high-quality attention-free TTS)","date":"2021-08-30","arxiv_id":"2108.13320","n_code_links":2,"syntology":null},{"paper":null,"title":"Ctrl-P: Temporal Control of Prosodic Variation for Speech Synthesis","date":"2021-06-15","arxiv_id":"2106.08352","n_code_links":0,"syntology":null},{"paper":null,"title":"VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention","date":"2021-02-12","arxiv_id":"2102.06431","n_code_links":0,"syntology":null},{"paper":"/paper/bidirectional-variational-inference-for-non","title":"Bidirectional Variational Inference for Non-Autoregressive Text-to-Speech","date":"2021-01-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Using previous acoustic context to improve Text-to-Speech synthesis","date":"2020-12-07","arxiv_id":"2012.03763","n_code_links":0,"syntology":null},{"paper":"/paper/learning-speaker-embedding-from-text-to","title":"Learning Speaker Embedding from Text-to-Speech","date":"2020-10-21","arxiv_id":"2010.11221","n_code_links":1,"syntology":null},{"paper":"/paper/non-attentive-tacotron-robust-and-1","title":"Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling","date":"2020-10-08","arxiv_id":"2010.04301","n_code_links":6,"syntology":{"ran":0,"of":7,"unverified":7,"pointer_only":0}},{"paper":"/paper/speedyspeech-efficient-neural-speech","title":"SpeedySpeech: Efficient Neural Speech Synthesis","date":"2020-08-09","arxiv_id":"2008.03802","n_code_links":3,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":0}},{"paper":"/paper/one-model-many-languages-meta-learning-for","title":"One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech","date":"2020-08-03","arxiv_id":"2008.00768","n_code_links":1,"syntology":null},{"paper":"/paper/flowtron-an-autoregressive-flow-based","title":"Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis","date":"2020-05-12","arxiv_id":"2005.05957","n_code_links":3,"syntology":{"ran":4,"of":19,"unverified":15,"pointer_only":0}},{"paper":null,"title":"Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis","date":"2020-02-06","arxiv_id":"2002.03785","n_code_links":0,"syntology":null},{"paper":"/paper/wavetts-tacotron-based-tts-with-joint-time","title":"WaveTTS: Tacotron-based TTS with Joint Time-Frequency Domain Loss","date":"2020-02-02","arxiv_id":"2002.00417","n_code_links":0,"syntology":null},{"paper":"/paper/probing-the-phonetic-and-phonological","title":"Probing the phonetic and phonological knowledge of tones in Mandarin TTS models","date":"2019-12-23","arxiv_id":"1912.10915","n_code_links":1,"syntology":null},{"paper":"/paper/mellotron-multispeaker-expressive-voice","title":"Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens","date":"2019-10-26","arxiv_id":"1910.11997","n_code_links":5,"syntology":null},{"paper":"/paper/natural-tts-synthesis-by-conditioning-wavenet","title":"Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions","date":"2017-12-16","arxiv_id":"1712.05884","n_code_links":33,"syntology":{"ran":7,"of":7,"unverified":0,"pointer_only":2}},{"paper":"/paper/attention-based-models-for-speech-recognition","title":"Attention-Based Models for Speech Recognition","date":"2015-06-24","arxiv_id":"1506.07503","n_code_links":14,"syntology":null}],"papers_shown":25,"tasks":[{"task":"/task/text-to-speech","name":"Text to Speech","papers":16},{"task":"/task/text-to-speech-1","name":"text-to-speech","papers":16},{"task":"/task/speech-synthesis","name":"Speech Synthesis","papers":14},{"task":"/task/text-to-speech-synthesis","name":"Text-To-Speech Synthesis","papers":4},{"task":"/task/decoder","name":"Decoder","papers":3},{"task":"/task/speech-recognition","name":"Speech Recognition","papers":2},{"task":"/task/style-transfer","name":"Style Transfer","papers":2},{"task":"/task/voice-cloning","name":"Voice Cloning","papers":2},{"task":"/task/acoustic-modelling","name":"Acoustic Modelling","papers":1},{"task":"/task/all","name":"All","papers":1},{"task":"/task/audio-synthesis","name":"Audio Synthesis","papers":1},{"task":null,"name":"CPU","papers":1},{"task":"/task/classification-1","name":"Classification","papers":1},{"task":"/task/disentanglement","name":"Disentanglement","papers":1},{"task":"/task/form","name":"Form","papers":1},{"task":null,"name":"GPU","papers":1},{"task":"/task/classification","name":"General Classification","papers":1},{"task":"/task/machine-translation","name":"Machine Translation","papers":1},{"task":"/task/meta-learning","name":"Meta-Learning","papers":1},{"task":"/task/phoneme-recognition","name":"Phoneme Recognition","papers":1}],"tasks_shown":20,"n_tasks":29,"usage_by_year":[{"year":"2015","papers":1},{"year":"2017","papers":1},{"year":"2019","papers":2},{"year":"2020","papers":8},{"year":"2021","papers":6},{"year":"2022","papers":2},{"year":"2023","papers":4},{"year":"2024","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/location-sensitive-attention"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}