{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/long-term-recurrent-convolutional-networks","title":"Long-term Recurrent Convolutional Networks for Visual Recognition and Description","arxiv_id":"1411.4389","date":"2014-11-17","proceeding":"CVPR 2015 6","authors":["Jeff Donahue","Lisa Anne Hendricks","Marcus Rohrbach","Subhashini Venugopalan","Sergio Guadarrama","Kate Saenko","Trevor Darrell"],"abstract":"Models based on deep convolutional networks have dominated recent image\ninterpretation tasks; we investigate whether models which are also recurrent,\nor \"temporally deep\", are effective for tasks involving sequences, visual and\notherwise. We develop a novel recurrent convolutional architecture suitable for\nlarge-scale visual learning which is end-to-end trainable, and demonstrate the\nvalue of these models on benchmark video recognition tasks, image description\nand retrieval problems, and video narration challenges. In contrast to current\nmodels which assume a fixed spatio-temporal receptive field or simple temporal\naveraging for sequential processing, recurrent convolutional models are \"doubly\ndeep\"' in that they can be compositional in spatial and temporal \"layers\". Such\nmodels may have advantages when target concepts are complex and/or training\ndata are limited. Learning long-term dependencies is possible when\nnonlinearities are incorporated into the network state updates. Long-term RNN\nmodels are appealing in that they directly can map variable-length inputs\n(e.g., video frames) to variable length outputs (e.g., natural language text)\nand can model complex temporal dynamics; yet they can be optimized with\nbackpropagation. Our recurrent long-term models are directly connected to\nmodern visual convnet models and can be jointly trained to simultaneously learn\ntemporal dynamics and convolutional perceptual representations. Our results\nshow such models have distinct advantages over state-of-the-art models for\nrecognition or generation which are separately defined and/or optimized.","url_abs":"http://arxiv.org/abs/1411.4389v4","url_pdf":"http://arxiv.org/pdf/1411.4389v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"long-term-recurrent-convolutional-networks","repo_url":"https://github.com/DJAlexJ/LRCN-for-Video-Regression","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"long-term-recurrent-convolutional-networks","repo_url":"https://github.com/Deepu1992/VideoClassification","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"long-term-recurrent-convolutional-networks","repo_url":"https://github.com/doronharitan/human_activity_recognition_LRCN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"long-term-recurrent-convolutional-networks","repo_url":"https://github.com/garythung/torch-lrcn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"long-term-recurrent-convolutional-networks","repo_url":"https://github.com/kahnchana/RNN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok"}},{"paper_slug":"long-term-recurrent-convolutional-networks","repo_url":"https://github.com/rlaengud123/CMC_LRCN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"long-term-recurrent-convolutional-networks","repo_url":"https://github.com/sujaygarlanka/tennis_stroke_recognition","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":null,"task_name":"Image Description"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"video-recognition","task_name":"Video Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/human-interaction-recognition-on-bit","task":"Human Interaction Recognition","dataset":"BIT","model":"Donahue et al.","rank_in_archive_order":3,"of":3,"metrics":{"Accuracy":"80.13"},"uses_additional_data":false},{"leaderboard":"/sota/human-interaction-recognition-on-ut","task":"Human Interaction Recognition","dataset":"UT","model":"Donahue et al.","rank_in_archive_order":4,"of":4,"metrics":{"Accuracy":"85.00"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1411.4389","atlas_url":"https://app.syntology.ai/?focus=1411.4389","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}