{"url":"/method/deep-lstm-reader","slug":"deep-lstm-reader","name":"Deep LSTM Reader","full_name":"Deep LSTM Reader","full_name_withheld":false,"description_markdown":"The **Deep LSTM Reader** is a neural network for reading comprehension. We feed documents one word at a time into a Deep [LSTM](https://paperswithcode.com/method/lstm) encoder, after a delimiter we then also feed the query into the encoder. The model therefore processes each document query pair as a single long sequence. Given the embedded document and query the network predicts which token in the document answers the query.\r\n\r\nThe model consists of a Deep LSTM cell with skip connections from each input $x\\left(t\\right)$ to every hidden layer, and from every hidden layer to the output $y\\left(t\\right)$:\r\n\r\n$$x'\\left(t, k\\right) = x\\left(t\\right)||y'\\left(t, k - 1\\right) \\text{,  } y\\left(t\\right) = y'\\left(t, 1\\right)|| \\dots ||y'\\left(t, K\\right) $$\r\n\r\n$$ i\\left(t, k\\right) =  \\left(W\\_{kxi}x'\\left(t, k\\right) + W\\_{khi}h(t - 1, k) + W\\_{kci}c\\left(t - 1, k\\right) + b\\_{ki}\\right) $$\r\n\r\n$$ f\\left(t, k\\right) =  \\left(W\\_{kxf}x\\left(t\\right) + W\\_{khf}h\\left(t - 1, k\\right) + W\\_{kcf}c\\left(t - 1, k\\right) + b\\_{kf}\\right) $$\r\n\r\n$$ c\\left(t, k\\right) = f\\left(t, k\\right)c\\left(t - 1, k\\right) + i\\left(t, k\\right)\\text{tanh}\\left(W\\_{kxc}x'\\left(t, k\\right) + W\\_{khc}h\\left(t -  1, k\\right) + b\\_{kc}\\right) $$\r\n\r\n$$ o\\left(t, k\\right) =  \\left(W\\_{kxo}x'\\left(t, k\\right) + W\\_{kho}h\\left(t - 1, k\\right) + W\\_{kco}c\\left(t, k\\right) + b\\_{ko}\\right) $$\r\n\r\n$$ h\\left(t, k\\right) = o\\left(t, k\\right)\\text{tanh}\\left(c\\left(t, k\\right)\\right) $$\r\n\r\n$$ y'\\left(t, k\\right) = W\\_{kyh}\\left(t, k\\right) + b\\_{ky} $$\r\n\r\nwhere || indicates vector concatenation, $h\\left(t, k\\right)$ is the hidden state for layer $k$ at time $t$, and $i$, $f$, $o$ are the input, forget, and output gates respectively. Thus our Deep LSTM Reader is defined by $g^{\\text{LSTM}}\\left(d, q\\right) = y\\left(|d|+|q|\\right)$ with input $x\\left(t\\right)$ the concatenation of $d$ and $q$ separated by the delimiter |||.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Teaching Machines to Read and Comprehend","paper":"/paper/teaching-machines-to-read-and-comprehend","first_author":"Karl Moritz Hermann","n_authors":7,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/teaching-machines-to-read-and-comprehend"},"source":{"url":"http://arxiv.org/abs/1506.03340v3","title":"Teaching Machines to Read and Comprehend","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Reading Comprehension Models","url":"/methods/category/reading-comprehension-models","pwc_aliases":[]},{"area":"Sequential","area_id":"sequential","collection":"Recurrent Neural Networks","url":"/methods/category/recurrent-neural-networks","pwc_aliases":[]}],"n_papers_tagged":1,"archive_num_papers":1,"papers_newest_first":[{"paper":"/paper/teaching-machines-to-read-and-comprehend","title":"Teaching Machines to Read and Comprehend","date":"2015-06-10","arxiv_id":"1506.03340","n_code_links":12,"syntology":{"ran":0,"of":2,"unverified":2,"pointer_only":0}}],"papers_shown":1,"tasks":[{"task":"/task/reading-comprehension","name":"Reading Comprehension","papers":1}],"tasks_shown":1,"n_tasks":1,"usage_by_year":[{"year":"2015","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/deep-lstm-reader"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}