{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/attention-clusters-purely-attention-based","title":"Attention Clusters: Purely Attention Based Local Feature Integration for Video Classification","arxiv_id":"1711.09550","date":"2017-11-27","proceeding":"CVPR 2018 6","authors":["Xiang Long","Chuang Gan","Gerard de Melo","Jiajun Wu","Xiao Liu","Shilei Wen"],"abstract":"Recently, substantial research effort has focused on how to apply CNNs or\nRNNs to better extract temporal patterns from videos, so as to improve the\naccuracy of video classification. In this paper, however, we show that temporal\ninformation, especially longer-term patterns, may not be necessary to achieve\ncompetitive results on common video classification datasets. We investigate the\npotential of a purely attention based local feature integration. Accounting for\nthe characteristics of such features in video classification, we propose a\nlocal feature integration framework based on attention clusters, and introduce\na shifting operation to capture more diverse signals. We carefully analyze and\ncompare the effect of different attention mechanisms, cluster sizes, and the\nuse of the shifting operation, and also investigate the combination of\nattention clusters for multimodal integration. We demonstrate the effectiveness\nof our framework on three real-world video classification datasets. Our model\nachieves competitive results across all of these. In particular, on the\nlarge-scale Kinetics dataset, our framework obtains an excellent single model\naccuracy of 79.4% in terms of the top-1 and 94.0% in terms of the top-5\naccuracy on the validation set. The attention clusters are the backbone of our\nwinner solution at ActivityNet Kinetics Challenge 2017. Code and models will be\nreleased soon.","url_abs":"http://arxiv.org/abs/1711.09550v1","url_pdf":"http://arxiv.org/pdf/1711.09550v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"attention-clusters-purely-attention-based","repo_url":"https://github.com/longxiang92/Flash-MNIST","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"attention-clusters-purely-attention-based","repo_url":"https://github.com/pomonam/AttentionCluster","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"attention-clusters-purely-attention-based","repo_url":"https://github.com/2024-MindSpore-1/Code6/tree/main/AttentionCluster","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"attention-clusters-purely-attention-based","repo_url":"https://github.com/MindSpore-paper-code-3/code6/tree/main/AttentionCluster","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"attention-clusters-purely-attention-based","repo_url":"https://github.com/mindspore-ai/models/tree/master/research/cv/AttentionCluster","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"video-classification","task_name":"Video Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1711.09550","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}