{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/weakly-supervised-action-localization-by-2","title":"Weakly-Supervised Action Localization by Generative Attention Modeling","arxiv_id":"2003.12424","date":"2020-03-27","proceeding":"CVPR 2020 6","authors":["Baifeng Shi","Qi Dai","Yadong Mu","Jingdong Wang"],"abstract":"Weakly-supervised temporal action localization is a problem of learning an action localization model with only video-level action labeling available. The general framework largely relies on the classification activation, which employs an attention model to identify the action-related frames and then categorizes them into different classes. Such method results in the action-context confusion issue: context frames near action clips tend to be recognized as action frames themselves, since they are closely related to the specific classes. To solve the problem, in this paper we propose to model the class-agnostic frame-wise probability conditioned on the frame attention using conditional Variational Auto-Encoder (VAE). With the observation that the context exhibits notable difference from the action at representation level, a probabilistic model, i.e., conditional VAE, is learned to model the likelihood of each frame given the attention. By maximizing the conditional probability with respect to the attention, the action and non-action frames are well separated. Experiments on THUMOS14 and ActivityNet1.2 demonstrate advantage of our method and effectiveness in handling action-context confusion problem. Code is now available on GitHub.","url_abs":"https://arxiv.org/abs/2003.12424v2","url_pdf":"https://arxiv.org/pdf/2003.12424v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"weakly-supervised-action-localization-by-2","repo_url":"https://github.com/bfshi/DGAM-Weakly-Supervised-Action-Localization","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"action-localization","task_name":"Action Localization"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"},{"task_slug":"weakly-supervised-action-localization","task_name":"Weakly Supervised Action Localization"},{"task_slug":"weakly-supervised-temporal-action","task_name":"Weakly-supervised Temporal Action Localization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/weakly-supervised-action-localization-on-2","task":"Weakly Supervised Action Localization","dataset":"ActivityNet-1.2","model":"DGAM","rank_in_archive_order":11,"of":19,"metrics":{"Mean mAP":"24.4","mAP@0.5":"41.0"},"uses_additional_data":false},{"leaderboard":"/sota/weakly-supervised-action-localization-on","task":"Weakly Supervised Action Localization","dataset":"THUMOS 2014","model":"DGAM","rank_in_archive_order":18,"of":30,"metrics":{"mAP@0.1:0.5":"45.6","mAP@0.1:0.7":"37.0","mAP@0.5":"28.8"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2003.12424","atlas_url":"https://app.syntology.ai/?focus=2003.12424","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}