{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-video-classification-using-fewer","title":"Efficient Video Classification Using Fewer Frames","arxiv_id":"1902.10640","date":"2019-02-27","proceeding":"CVPR 2019 6","authors":["Shweta Bhardwaj","Mukundhan Srinivasan","Mitesh M. Khapra"],"abstract":"Recently,there has been a lot of interest in building compact models for\nvideo classification which have a small memory footprint (<1 GB). While these\nmodels are compact, they typically operate by repeated application of a small\nweight matrix to all the frames in a video. E.g. recurrent neural network based\nmethods compute a hidden state for every frame of the video using a recurrent\nweight matrix. Similarly, cluster-and-aggregate based methods such as NetVLAD,\nhave a learnable clustering matrix which is used to assign soft-clusters to\nevery frame in the video. Since these models look at every frame in the video,\nthe number of floating point operations (FLOPs) is still large even though the\nmemory footprint is small. We focus on building compute-efficient video\nclassification models which process fewer frames and hence have less number of\nFLOPs. Similar to memory efficient models, we use the idea of distillation\nalbeit in a different setting. Specifically, in our case, a compute-heavy\nteacher which looks at all the frames in the video is used to train a\ncompute-efficient student which looks at only a small fraction of frames in the\nvideo. This is in contrast to a typical memory efficient Teacher-Student\nsetting, wherein both the teacher and the student look at all the frames in the\nvideo but the student has fewer parameters. Our work thus complements the\nresearch on memory efficient video classification. We do an extensive\nevaluation with three types of models for video classification,viz.(i)\nrecurrent models (ii) cluster-and-aggregate models and (iii) memory-efficient\ncluster-and-aggregate models and show that in each of these cases, a see-it-all\nteacher can be used to train a compute efficient see-very-little student. We\nshow that the proposed student network can reduce the inference time by 30% and\nthe number of FLOPs by approximately 90% with a negligible drop in the\nperformance.","url_abs":"http://arxiv.org/abs/1902.10640v1","url_pdf":"http://arxiv.org/pdf/1902.10640v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-video-classification-using-fewer","repo_url":"https://github.com/shwetabhardwaj44/EfficentVideoClassification_Youtube8M","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"video-classification","task_name":"Video Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-classification-on-youtube-8m","task":"Video Classification","dataset":"YouTube-8M","model":"Hierarchical LSTM with MoE","rank_in_archive_order":2,"of":3,"metrics":{"Global Average Precision":"81.1","Hit@1":"86.8","mAP":"41.4"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1902.10640","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}