{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/revisiting-foreground-and-background-1","title":"Revisiting Foreground and Background Separation in Weakly-supervised Temporal Action Localization: A Clustering-based Approach","arxiv_id":"2312.14138","date":"2023-12-21","proceeding":"ICCV 2023 1","authors":["Qinying Liu","Zilei Wang","Shenghai Rong","Junjie Li","Yixin Zhang"],"abstract":"Weakly-supervised temporal action localization aims to localize action instances in videos with only video-level action labels. Existing methods mainly embrace a localization-by-classification pipeline that optimizes the snippet-level prediction with a video classification loss. However, this formulation suffers from the discrepancy between classification and detection, resulting in inaccurate separation of foreground and background (F\\&B) snippets. To alleviate this problem, we propose to explore the underlying structure among the snippets by resorting to unsupervised snippet clustering, rather than heavily relying on the video classification loss. Specifically, we propose a novel clustering-based F\\&B separation algorithm. It comprises two core components: a snippet clustering component that groups the snippets into multiple latent clusters and a cluster classification component that further classifies the cluster as foreground or background. As there are no ground-truth labels to train these two components, we introduce a unified self-labeling mechanism based on optimal transport to produce high-quality pseudo-labels that match several plausible prior distributions. This ensures that the cluster assignments of the snippets can be accurately associated with their F\\&B labels, thereby boosting the F\\&B separation. We evaluate our method on three benchmarks: THUMOS14, ActivityNet v1.2 and v1.3. Our method achieves promising performance on all three benchmarks while being significantly more lightweight than previous methods. Code is available at https://github.com/Qinying-Liu/CASE","url_abs":"https://arxiv.org/abs/2312.14138v1","url_pdf":"https://arxiv.org/pdf/2312.14138v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"revisiting-foreground-and-background-1","repo_url":"https://github.com/qinying-liu/case","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"action-localization","task_name":"Action Localization"},{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"},{"task_slug":"video-classification","task_name":"Video Classification"},{"task_slug":"weakly-supervised-action-localization","task_name":"Weakly Supervised Action Localization"},{"task_slug":"weakly-supervised-temporal-action","task_name":"Weakly-supervised Temporal Action Localization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/weakly-supervised-action-localization-on-2","task":"Weakly Supervised Action Localization","dataset":"ActivityNet-1.2","model":"CASE","rank_in_archive_order":4,"of":19,"metrics":{"Mean mAP":"27.9","mAP@0.5":"43.8"},"uses_additional_data":false},{"leaderboard":"/sota/weakly-supervised-action-localization-on-1","task":"Weakly Supervised Action Localization","dataset":"ActivityNet-1.3","model":"CASE","rank_in_archive_order":5,"of":17,"metrics":{"mAP@0.5":"43.2","mAP@0.5:0.95":"26.8"},"uses_additional_data":false},{"leaderboard":"/sota/weakly-supervised-action-localization-on","task":"Weakly Supervised Action Localization","dataset":"THUMOS 2014","model":"CASE + Zhou et al.","rank_in_archive_order":5,"of":30,"metrics":{"mAP@0.1:0.7":"49.2"},"uses_additional_data":false},{"leaderboard":"/sota/weakly-supervised-action-localization-on","task":"Weakly Supervised Action Localization","dataset":"THUMOS 2014","model":"CASE","rank_in_archive_order":10,"of":30,"metrics":{"mAP@0.1:0.5":"57.1","mAP@0.1:0.7":"46.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.14138","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}