{"url":"/task/video-classification","name":"Video Classification","slug":"video-classification","description_markdown":"**Video Classification** is the task of producing a label that is relevant to the video given its frames. A good video level classifier is one that not only provides accurate frame labels, but also best describes the entire video given the features and the annotations of the various frames in the video. For example, a video might contain a tree in some frame, but the label that is central to the video might be something else (e.g., “hiking”). The granularity of the labels that are needed to describe the frames and the video depends on the task. Typical tasks include assigning one or more global labels to the video, and assigning one or more labels for each frame inside the video.\r\n\r\n\r\n<span class=\"description-source\">Source: [Efficient Large Scale Video Classification ](https://arxiv.org/abs/1505.06250)</span>","categories":[{"name":"Computer Vision","url":"/area/computer-vision"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"archive_url"},"counts":{"papers_tagged":455,"papers_with_code":206,"benchmarks":12,"benchmark_tables_in_archive":12,"benchmark_tables_shown":12,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":22,"subtasks":2,"parent_tasks":1},"benchmarks":[{"leaderboard":"/sota/video-classification-on-breakfast","slug":"video-classification-on-breakfast","dataset":"Breakfast","dataset_url":"/dataset/breakfast","rows_in_archive":9,"metrics":["Accuracy (%)"],"first_row_in_archive_order":{"model":"HERMES","paper_title":"HERMES: temporal-coHERent long-forM understanding with Episodes and Semantics","paper_url":"/paper/bridging-episodes-and-semantics-a-novel","paper_date":"2024-08-30","arxiv_id":"2408.17443","code_links":[{"title":"joslefaure/HERMES","url":"https://github.com/joslefaure/HERMES"}],"syntology":{"n":9,"n_ran":7,"n_unverified":2,"n_pointer_only":0}}},{"leaderboard":"/sota/video-classification-on-coin-1","slug":"video-classification-on-coin-1","dataset":"COIN","dataset_url":"/dataset/coin","rows_in_archive":7,"metrics":["Accuracy (%)"],"first_row_in_archive_order":{"model":"HERMES","paper_title":"HERMES: temporal-coHERent long-forM understanding with Episodes and Semantics","paper_url":"/paper/bridging-episodes-and-semantics-a-novel","paper_date":"2024-08-30","arxiv_id":"2408.17443","code_links":[{"title":"joslefaure/HERMES","url":"https://github.com/joslefaure/HERMES"}],"syntology":{"n":9,"n_ran":7,"n_unverified":2,"n_pointer_only":0}}},{"leaderboard":"/sota/video-classification-on-mob","slug":"video-classification-on-mob","dataset":"MoB","dataset_url":"/dataset/mob","rows_in_archive":3,"metrics":["Accuracy"],"first_row_in_archive_order":{"model":"VTN","paper_title":"Malicious or Benign? Towards Effective Content Moderation for Children's Videos","paper_url":"/paper/malicious-or-benign-towards-effective-content","paper_date":"2023-05-24","arxiv_id":"2305.15551","code_links":[{"title":"syedhammadahmed/mob","url":"https://github.com/syedhammadahmed/mob"},{"title":"andreascas/oran_gai","url":"https://github.com/andreascas/oran_gai"}],"syntology":null}},{"leaderboard":"/sota/video-classification-on-youtube-8m","slug":"video-classification-on-youtube-8m","dataset":"YouTube-8M","dataset_url":"/dataset/youtube-8m","rows_in_archive":3,"metrics":["Hit@1","PERR","Hit@5","Global Average Precision","mAP"],"first_row_in_archive_order":{"model":"DCGN (self-attention graph pooling)","paper_title":"Hierarchical Video Frame Sequence Representation with Deep Convolutional Graph Network","paper_url":"/paper/190600377","paper_date":"2019-06-02","arxiv_id":"1906.00377","code_links":[],"syntology":null}},{"leaderboard":"/sota/video-classification-on-hockey-fight","slug":"video-classification-on-hockey-fight","dataset":"Hockey Fight Detection Dataset","dataset_url":"/dataset/hockey-fight-detection-dataset","rows_in_archive":2,"metrics":["1:1 Accuracy","Accuracy"],"first_row_in_archive_order":{"model":"CNN+LSTM","paper_title":"Robust Real-Time Violence Detection in Video Using CNN And LSTM","paper_url":"/paper/robust-real-time-violence-detection-in-video","paper_date":"2019-03-27","arxiv_id":null,"code_links":[{"title":"mamonraab/Real-Time-Violence-Detection-in-Video-","url":"https://github.com/mamonraab/Real-Time-Violence-Detection-in-Video-"}],"syntology":null}},{"leaderboard":"/sota/video-classification-on-charades","slug":"video-classification-on-charades","dataset":"Charades","dataset_url":"/dataset/charades","rows_in_archive":1,"metrics":["mAP"],"first_row_in_archive_order":{"model":"Multigrid","paper_title":"A Multigrid Method for Efficiently Training Video Models","paper_url":"/paper/a-multigrid-method-for-efficiently-training","paper_date":"2019-12-02","arxiv_id":"1912.00998","code_links":[{"title":"facebookresearch/SlowFast","url":"https://github.com/facebookresearch/SlowFast"},{"title":"kkahatapitiya/X3D-Multigrid","url":"https://github.com/kkahatapitiya/X3D-Multigrid"},{"title":"alexandrosstergiou/Squeeze-and-Recursion-Temporal-Gates","url":"https://github.com/alexandrosstergiou/Squeeze-and-Recursion-Temporal-Gates"}],"syntology":{"n":10,"n_ran":2,"n_unverified":8,"n_pointer_only":3}}},{"leaderboard":"/sota/video-classification-on-home-action-genome","slug":"video-classification-on-home-action-genome","dataset":"Home Action Genome","dataset_url":"/dataset/home-action-genome","rows_in_archive":1,"metrics":["Accuracy (%)"],"first_row_in_archive_order":{"model":"Cooperative Ours (3rd-person)","paper_title":"Home Action Genome: Cooperative Compositional Action Understanding","paper_url":"/paper/home-action-genome-cooperative-compositional","paper_date":"2021-05-11","arxiv_id":"2105.05226","code_links":[{"title":"nishantrai18/homage","url":"https://github.com/nishantrai18/homage"}],"syntology":{"n":16,"n_ran":2,"n_unverified":14,"n_pointer_only":0}}},{"leaderboard":"/sota/video-classification-on-kinetics","slug":"video-classification-on-kinetics","dataset":"Kinetics","dataset_url":"/dataset/kinetics","rows_in_archive":1,"metrics":["Top-1"],"first_row_in_archive_order":{"model":"Multigrid","paper_title":"A Multigrid Method for Efficiently Training Video Models","paper_url":"/paper/a-multigrid-method-for-efficiently-training","paper_date":"2019-12-02","arxiv_id":"1912.00998","code_links":[{"title":"facebookresearch/SlowFast","url":"https://github.com/facebookresearch/SlowFast"},{"title":"kkahatapitiya/X3D-Multigrid","url":"https://github.com/kkahatapitiya/X3D-Multigrid"},{"title":"alexandrosstergiou/Squeeze-and-Recursion-Temporal-Gates","url":"https://github.com/alexandrosstergiou/Squeeze-and-Recursion-Temporal-Gates"}],"syntology":{"n":10,"n_ran":2,"n_unverified":8,"n_pointer_only":3}}},{"leaderboard":"/sota/video-classification-on-multimodal-pisa","slug":"video-classification-on-multimodal-pisa","dataset":"Multimodal PISA","dataset_url":"/dataset/multimodal-pisa","rows_in_archive":1,"metrics":["Accuracy (%)"],"first_row_in_archive_order":{"model":"Video","paper_title":"Piano Skills Assessment","paper_url":"/paper/piano-skills-assessment","paper_date":"2021-01-13","arxiv_id":"2101.04884","code_links":[{"title":"ParitoshParmar/Piano-Skills-Assessment","url":"https://github.com/ParitoshParmar/Piano-Skills-Assessment"}],"syntology":null}},{"leaderboard":"/sota/video-classification-on-something-something","slug":"video-classification-on-something-something","dataset":"Something-Something V1","dataset_url":"/dataset/something-something-v1","rows_in_archive":1,"metrics":["Top-5 Accuracy"],"first_row_in_archive_order":{"model":"MSNet-R50En (ours)","paper_title":"MotionSqueeze: Neural Motion Feature Learning for Video Understanding","paper_url":"/paper/motionsqueeze-neural-motion-feature-learning","paper_date":"2020-07-20","arxiv_id":"2007.09933","code_links":[{"title":"arunos728/MotionSqueeze","url":"https://github.com/arunos728/MotionSqueeze"},{"title":"arunos728/arunos728.github.io","url":"https://github.com/arunos728/arunos728.github.io"}],"syntology":{"n":8,"n_ran":3,"n_unverified":5,"n_pointer_only":0}}},{"leaderboard":"/sota/video-classification-on-something-something-1","slug":"video-classification-on-something-something-1","dataset":"Something-Something V2","dataset_url":"/dataset/something-something-v2","rows_in_archive":1,"metrics":["Top-5 Accuracy"],"first_row_in_archive_order":{"model":"MSNet-R50En (ours)","paper_title":"MotionSqueeze: Neural Motion Feature Learning for Video Understanding","paper_url":"/paper/motionsqueeze-neural-motion-feature-learning","paper_date":"2020-07-20","arxiv_id":"2007.09933","code_links":[{"title":"arunos728/MotionSqueeze","url":"https://github.com/arunos728/MotionSqueeze"},{"title":"arunos728/arunos728.github.io","url":"https://github.com/arunos728/arunos728.github.io"}],"syntology":{"n":8,"n_ran":3,"n_unverified":5,"n_pointer_only":0}}},{"leaderboard":"/sota/video-classification-on-sri-approve-fine","slug":"video-classification-on-sri-approve-fine","dataset":"SRI-APPROVE Fine-Grained Video Classification","dataset_url":"/dataset/sri-approve-fine-grained-video-classification","rows_in_archive":1,"metrics":["AUPR"],"first_row_in_archive_order":{"model":"Multi-Label Prototypes Contrastive Learning","paper_title":"Class Prototypes Based Contrastive Learning for Classifying Multi-Label and Fine-Grained Educational Videos","paper_url":"/paper/class-prototypes-based-contrastive-learning","paper_date":"2023-01-01","arxiv_id":null,"code_links":[],"syntology":null}}],"datasets":[{"url":"/dataset/kinetics","name":"Kinetics","full_name":"Kinetics Human Action Video Dataset","num_papers_in_archive":1341},{"url":"/dataset/charades","name":"Charades","full_name":"","num_papers_in_archive":428},{"url":"/dataset/something-something-v2","name":"Something-Something V2","full_name":"","num_papers_in_archive":290},{"url":"/dataset/breakfast","name":"Breakfast","full_name":"The Breakfast Actions Dataset","num_papers_in_archive":179},{"url":"/dataset/youtube-8m","name":"YouTube-8M","full_name":"","num_papers_in_archive":147},{"url":"/dataset/something-something-v1","name":"Something-Something V1","full_name":"","num_papers_in_archive":117},{"url":"/dataset/coin","name":"COIN","full_name":"","num_papers_in_archive":105},{"url":"/dataset/charades-ego","name":"Charades-Ego","full_name":"","num_papers_in_archive":33},{"url":"/dataset/home-action-genome","name":"Home Action Genome","full_name":"","num_papers_in_archive":9},{"url":"/dataset/moma-lrg","name":"MOMA-LRG","full_name":"Multi-Object Multi-Actor activity parsing with Language-Refined Graphs","num_papers_in_archive":3},{"url":"/dataset/hockey-fight-detection-dataset","name":"Hockey Fight Detection Dataset","full_name":"","num_papers_in_archive":2},{"url":"/dataset/metavd","name":"MetaVD","full_name":"Meta Video Dataset","num_papers_in_archive":2},{"url":"/dataset/mob","name":"MoB","full_name":"Malicious or Benign Cartoon Videos","num_papers_in_archive":2},{"url":"/dataset/airletters","name":"AirLetters","full_name":"","num_papers_in_archive":1},{"url":"/dataset/bukva","name":"Bukva","full_name":"Bukva: Russian Sign Language Alphabet","num_papers_in_archive":1},{"url":"/dataset/multimodal-pisa","name":"Multimodal PISA","full_name":"Multimodal Piano Skills Assessment","num_papers_in_archive":1},{"url":"/dataset/sri-approve-fine-grained-video-classification","name":"SRI-APPROVE Fine-Grained Video Classification","full_name":"","num_papers_in_archive":1},{"url":"/dataset/trailers12k","name":"Trailers12k","full_name":"","num_papers_in_archive":1},{"url":"/dataset/win-fail-action-understanding","name":"Win-Fail Action Understanding","full_name":"Win-Fail Action Understanding","num_papers_in_archive":1},{"url":"/dataset/crowd11","name":"Crowd 11","full_name":"A Dataset for Fine Grained Crowd Behaviour Analysis","num_papers_in_archive":0},{"url":"/dataset/raise-lpbf","name":"RAISE-LPBF","full_name":"","num_papers_in_archive":0},{"url":"/dataset/storytelling-video-dataset","name":"Video Dataset","full_name":"Storytelling Video Dataset (Russian, Emotion, Gesture, Speech)","num_papers_in_archive":0}],"subtasks":[{"url":"/task/multi-class-classification-four-level-video","name":"Multi Class Classification (Four-level Video Classification)"},{"url":"/task/student-engagement-level-detection-four-class","name":"Student Engagement Level Detection (Four Class Video Classification)"}],"parent_tasks":[{"url":"/task/video","name":"Video"}],"papers":{"order":"repositories listed in the archive (desc), then date (desc); the archive holds no stars","population":"papers tagged with this task that list at least one repository in the archive","shown":30,"of":206,"tagged_in_all":455,"items":[{"url":"/paper/non-local-neural-networks","title":"Non-local Neural Networks","date":"2017-11-21","arxiv_id":"1711.07971","repositories_listed":32,"syntology":{"n":4,"n_ran":3,"n_unverified":1,"n_pointer_only":4}},{"url":"/paper/group-normalization","title":"Group Normalization","date":"2018-03-22","arxiv_id":"1803.08494","repositories_listed":22,"syntology":{"n":15,"n_ran":5,"n_unverified":10,"n_pointer_only":4}},{"url":"/paper/is-space-time-attention-all-you-need-for","title":"Is Space-Time Attention All You Need for Video Understanding?","date":"2021-02-09","arxiv_id":"2102.05095","repositories_listed":16,"syntology":{"n":43,"n_ran":35,"n_unverified":8,"n_pointer_only":14}},{"url":"/paper/video-swin-transformer","title":"Video Swin Transformer","date":"2021-06-24","arxiv_id":"2106.13230","repositories_listed":15,"syntology":{"n":32,"n_ran":7,"n_unverified":25,"n_pointer_only":0}},{"url":"/paper/temporal-segment-networks-for-action","title":"Temporal Segment Networks for Action Recognition in Videos","date":"2017-05-08","arxiv_id":"1705.02953","repositories_listed":11,"syntology":null},{"url":"/paper/learning-representations-from-eeg-with-deep","title":"Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks","date":"2015-11-19","arxiv_id":"1511.06448","repositories_listed":11,"syntology":null},{"url":"/paper/2103-15691","title":"ViViT: A Video Vision Transformer","date":"2021-03-29","arxiv_id":"2103.15691","repositories_listed":10,"syntology":{"n":21,"n_ran":14,"n_unverified":7,"n_pointer_only":1}},{"url":"/paper/would-mega-scale-datasets-further-enhance","title":"Would Mega-scale Datasets Further Enhance Spatiotemporal 3D CNNs?","date":"2020-04-10","arxiv_id":"2004.04968","repositories_listed":10,"syntology":{"n":22,"n_ran":2,"n_unverified":20,"n_pointer_only":0}},{"url":"/paper/improved-multiscale-vision-transformers-for","title":"MViTv2: Improved Multiscale Vision Transformers for Classification and Detection","date":"2021-12-02","arxiv_id":"2112.01526","repositories_listed":9,"syntology":null},{"url":"/paper/uniformer-unifying-convolution-and-self","title":"UniFormer: Unifying Convolution and Self-attention for Visual Recognition","date":"2022-01-24","arxiv_id":"2201.09450","repositories_listed":8,"syntology":null},{"url":"/paper/x3d-expanding-architectures-for-efficient","title":"X3D: Expanding Architectures for Efficient Video Recognition","date":"2020-04-09","arxiv_id":"2004.04730","repositories_listed":8,"syntology":{"n":15,"n_ran":2,"n_unverified":13,"n_pointer_only":0}},{"url":"/paper/video-classification-with-channel-separated","title":"Video Classification with Channel-Separated Convolutional Networks","date":"2019-04-04","arxiv_id":"1904.02811","repositories_listed":7,"syntology":{"n":4,"n_ran":1,"n_unverified":3,"n_pointer_only":0}},{"url":"/paper/youtube-8m-a-large-scale-video-classification","title":"YouTube-8M: A Large-Scale Video Classification Benchmark","date":"2016-09-27","arxiv_id":"1609.08675","repositories_listed":7,"syntology":{"n":9,"n_ran":1,"n_unverified":8,"n_pointer_only":0}},{"url":"/paper/two-stream-convolutional-networks-for-action","title":"Two-Stream Convolutional Networks for Action Recognition in Videos","date":"2014-06-09","arxiv_id":"1406.2199","repositories_listed":7,"syntology":{"n":7,"n_ran":1,"n_unverified":6,"n_pointer_only":2}},{"url":"/paper/eco-efficient-convolutional-network-for","title":"ECO: Efficient Convolutional Network for Online Video Understanding","date":"2018-04-24","arxiv_id":"1804.09066","repositories_listed":6,"syntology":{"n":2,"n_ran":0,"n_unverified":2,"n_pointer_only":0}},{"url":"/paper/transferring-textual-knowledge-for-visual","title":"Revisiting Classifier: Transferring Vision-Language Models for Video Recognition","date":"2022-07-04","arxiv_id":"2207.01297","repositories_listed":5,"syntology":null},{"url":"/paper/representation-flow-for-action-recognition","title":"Representation Flow for Action Recognition","date":"2018-10-02","arxiv_id":"1810.01455","repositories_listed":5,"syntology":null},{"url":"/paper/attention-clusters-purely-attention-based","title":"Attention Clusters: Purely Attention Based Local Feature Integration for Video Classification","date":"2017-11-27","arxiv_id":"1711.09550","repositories_listed":5,"syntology":null},{"url":"/paper/learnable-pooling-with-context-gating-for","title":"Learnable pooling with Context Gating for video classification","date":"2017-06-21","arxiv_id":"1706.06905","repositories_listed":5,"syntology":null},{"url":"/paper/reversible-vision-transformers-1","title":"Reversible Vision Transformers","date":"2023-02-09","arxiv_id":"2302.04869","repositories_listed":4,"syntology":{"n":29,"n_ran":12,"n_unverified":17,"n_pointer_only":26}},{"url":"/paper/billion-scale-semi-supervised-learning-for","title":"Billion-scale semi-supervised learning for image classification","date":"2019-05-02","arxiv_id":"1905.00546","repositories_listed":4,"syntology":null},{"url":"/paper/ts-lstm-and-temporal-inception-exploiting","title":"TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognition","date":"2017-03-30","arxiv_id":"1703.10667","repositories_listed":4,"syntology":null},{"url":"/paper/patch-n-pack-navit-a-vision-transformer-for","title":"Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution","date":"2023-07-12","arxiv_id":"2307.06304","repositories_listed":3,"syntology":{"n":3,"n_ran":3,"n_unverified":0,"n_pointer_only":2}},{"url":"/paper/token-shift-transformer-for-video","title":"Token Shift Transformer for Video Classification","date":"2021-08-05","arxiv_id":"2108.02432","repositories_listed":3,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":1}},{"url":"/paper/revisiting-resnets-improved-training-and","title":"Revisiting ResNets: Improved Training and Scaling Strategies","date":"2021-03-13","arxiv_id":"2103.07579","repositories_listed":3,"syntology":null},{"url":"/paper/videomix-rethinking-data-augmentation-for","title":"VideoMix: Rethinking Data Augmentation for Video Classification","date":"2020-12-07","arxiv_id":"2012.03457","repositories_listed":3,"syntology":{"n":3,"n_ran":2,"n_unverified":1,"n_pointer_only":0}},{"url":"/paper/pyramidal-convolution-rethinking","title":"Pyramidal Convolution: Rethinking Convolutional Neural Networks for Visual Recognition","date":"2020-06-20","arxiv_id":"2006.11538","repositories_listed":3,"syntology":null},{"url":"/paper/non-local-neural-networks-with-grouped","title":"Non-Local Neural Networks With Grouped Bilinear Attentional Transforms","date":"2020-06-01","arxiv_id":null,"repositories_listed":3,"syntology":null},{"url":"/paper/on-translation-invariance-in-cnns","title":"On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location","date":"2020-03-16","arxiv_id":"2003.07064","repositories_listed":3,"syntology":null},{"url":"/paper/a-multigrid-method-for-efficiently-training","title":"A Multigrid Method for Efficiently Training Video Models","date":"2019-12-02","arxiv_id":"1912.00998","repositories_listed":3,"syntology":{"n":10,"n_ran":2,"n_unverified":8,"n_pointer_only":3}}],"syntology_records":16,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-09-24T18:15:14+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; our fixture could not drive it","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"}},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"}}