{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/excitation-backprop-for-rnns","title":"Excitation Backprop for RNNs","arxiv_id":"1711.06778","date":"2017-11-18","proceeding":"CVPR 2018 6","authors":["Sarah Adel Bargal","Andrea Zunino","Donghyun Kim","Jianming Zhang","Vittorio Murino","Stan Sclaroff"],"abstract":"Deep models are state-of-the-art for many vision tasks including video action\nrecognition and video captioning. Models are trained to caption or classify\nactivity in videos, but little is known about the evidence used to make such\ndecisions. Grounding decisions made by deep networks has been studied in\nspatial visual content, giving more insight into model predictions for images.\nHowever, such studies are relatively lacking for models of spatiotemporal\nvisual content - videos. In this work, we devise a formulation that\nsimultaneously grounds evidence in space and time, in a single pass, using\ntop-down saliency. We visualize the spatiotemporal cues that contribute to a\ndeep model's classification/captioning output using the model's internal\nrepresentation. Based on these spatiotemporal cues, we are able to localize\nsegments within a video that correspond with a specific action, or phrase from\na caption, without explicitly optimizing/training for these tasks.","url_abs":"http://arxiv.org/abs/1711.06778v3","url_pdf":"http://arxiv.org/pdf/1711.06778v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"excitation-backprop-for-rnns","repo_url":"https://github.com/sbargal/Caffe-ExcitationBP-RNNs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"},{"task_slug":"video-captioning","task_name":"Video Captioning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}