{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-critical-review-of-recurrent-neural","title":"A Critical Review of Recurrent Neural Networks for Sequence Learning","arxiv_id":"1506.00019","date":"2015-05-29","proceeding":null,"authors":["Zachary C. Lipton","John Berkowitz","Charles Elkan"],"abstract":"Countless learning tasks require dealing with sequential data. Image\ncaptioning, speech synthesis, and music generation all require that a model\nproduce outputs that are sequences. In other domains, such as time series\nprediction, video analysis, and musical information retrieval, a model must\nlearn from inputs that are sequences. Interactive tasks, such as translating\nnatural language, engaging in dialogue, and controlling a robot, often demand\nboth capabilities. Recurrent neural networks (RNNs) are connectionist models\nthat capture the dynamics of sequences via cycles in the network of nodes.\nUnlike standard feedforward neural networks, recurrent networks retain a state\nthat can represent information from an arbitrarily long context window.\nAlthough recurrent neural networks have traditionally been difficult to train,\nand often contain millions of parameters, recent advances in network\narchitectures, optimization techniques, and parallel computation have enabled\nsuccessful large-scale learning with them. In recent years, systems based on\nlong short-term memory (LSTM) and bidirectional (BRNN) architectures have\ndemonstrated ground-breaking performance on tasks as varied as image\ncaptioning, language translation, and handwriting recognition. In this survey,\nwe review and synthesize the research that over the past three decades first\nyielded and then made practical these powerful learning models. When\nappropriate, we reconcile conflicting notation and nomenclature. Our goal is to\nprovide a self-contained explication of the state of the art together with a\nhistorical perspective and references to primary research.","url_abs":"http://arxiv.org/abs/1506.00019v4","url_pdf":"http://arxiv.org/pdf/1506.00019v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-critical-review-of-recurrent-neural","repo_url":"https://github.com/junwang23/deepdirtycodes","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"a-critical-review-of-recurrent-neural","repo_url":"https://github.com/mamhoud/CapstoneProject","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"a-critical-review-of-recurrent-neural","repo_url":"https://github.com/mindspore-courses/heads-on-mindspore/tree/main/4-neural_poet_RNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"handwriting-recognition","task_name":"Handwriting Recognition"},{"task_slug":"image-captioning","task_name":"Image Captioning"},{"task_slug":"information-retrieval","task_name":"Information Retrieval"},{"task_slug":"music-generation","task_name":"Music Generation"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"speech-synthesis","task_name":"Speech Synthesis"},{"task_slug":"time-series","task_name":"Time Series Analysis"},{"task_slug":"time-series-prediction","task_name":"Time Series Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1506.00019","atlas_url":"https://app.syntology.ai/?focus=1506.00019","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}