{"url":"/task/video-summarization","name":"Video Summarization","slug":"video-summarization","description_markdown":"**Video Summarization** aims to generate a short synopsis that summarizes the video content by selecting its most informative and important parts. The produced summary is usually composed of a set of representative video frames (a.k.a. *video key-frames*), or video fragments (a.k.a. *video key-fragments*) that have been stitched in chronological order to form a shorter video. The former type of a video summary is known as **video storyboard**, and the latter type is known as **video skim**.\r\n\r\n<span class=\"description-source\">Source: [Video Summarization Using Deep Neural Networks: A Survey](https://arxiv.org/abs/2101.06072)</span> <br>\r\n<span style=\"color:grey; opacity: 0.6\">Image credit: [iJRASET](https://www.ijraset.com/fileserve.php?FID=12932)</span>","categories":[{"name":"Computer Vision","url":"/area/computer-vision"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"archive_url"},"counts":{"papers_tagged":280,"papers_with_code":89,"benchmarks":6,"benchmark_tables_in_archive":6,"benchmark_tables_shown":6,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":16,"subtasks":2,"parent_tasks":1},"benchmarks":[{"leaderboard":"/sota/video-summarization-on-summe","slug":"video-summarization-on-summe","dataset":"SumMe","dataset_url":"/dataset/summe","rows_in_archive":6,"metrics":["F1-score (Canonical)","F1-score (Augmented)","Kendall's Tau","Spearman's Rho"],"first_row_in_archive_order":{"model":"PGL-SUM","paper_title":"Combining Global and Local Attention with Positional Encoding for Video Summarization","paper_url":"/paper/combining-global-and-local-attention-with","paper_date":"2021-12-01","arxiv_id":null,"code_links":[{"title":"e-apostolidis/PGL-SUM","url":"https://github.com/e-apostolidis/PGL-SUM"}],"syntology":null}},{"leaderboard":"/sota/video-summarization-on-tvsum","slug":"video-summarization-on-tvsum","dataset":"TvSum","dataset_url":"/dataset/tvsum","rows_in_archive":6,"metrics":["F1-score (Canonical)","F1-score (Augmented)","Kendall's Tau","Spearman's Rho"],"first_row_in_archive_order":{"model":"RR-STG","paper_title":"Relational Reasoning Over Spatial-Temporal Graphs for Video Summarization","paper_url":"/paper/relational-reasoning-over-spatial-temporal","paper_date":"2022-04-06","arxiv_id":null,"code_links":[],"syntology":null}},{"leaderboard":"/sota/video-summarization-on-query-focused-video","slug":"video-summarization-on-query-focused-video","dataset":"Query-Focused Video Summarization Dataset","dataset_url":"/dataset/query-focused-video-summarization-dataset","rows_in_archive":2,"metrics":["F1 (avg)"],"first_row_in_archive_order":{"model":"EgoVLPv2","paper_title":"EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone","paper_url":"/paper/egovlpv2-egocentric-video-language-pre","paper_date":"2023-07-11","arxiv_id":"2307.05463","code_links":[{"title":"facebookresearch/EgoVLPv2","url":"https://github.com/facebookresearch/EgoVLPv2"}],"syntology":null}},{"leaderboard":"/sota/video-summarization-on-shot2story20k","slug":"video-summarization-on-shot2story20k","dataset":"Shot2Story20K","dataset_url":"/dataset/shot2story20k","rows_in_archive":2,"metrics":["CIDEr","BLEU-4","METEOR","ROUGE"],"first_row_in_archive_order":{"model":"Shotluck-Holmes (3.1B)","paper_title":"Shotluck Holmes: A Family of Efficient Small-Scale Large Language Vision Models For Video Captioning and Summarization","paper_url":"/paper/shotluck-holmes-a-family-of-efficient-small","paper_date":"2024-05-31","arxiv_id":"2405.20648","code_links":[{"title":"Skyline-9/Shotluck-Holmes","url":"https://github.com/Skyline-9/Shotluck-Holmes"}],"syntology":null}},{"leaderboard":"/sota/video-summarization-on-mr-hisum","slug":"video-summarization-on-mr-hisum","dataset":"Mr. HiSum","dataset_url":"/dataset/mr-hisum","rows_in_archive":1,"metrics":["MAP (50%)"],"first_row_in_archive_order":{"model":"PGL-SUM","paper_title":"Mr. HiSum: A Large-scale Dataset for Video Highlight Detection and Summarization","paper_url":"/paper/mr-hisum-a-large-scale-dataset-for-video","paper_date":"2023-09-26","arxiv_id":null,"code_links":[],"syntology":null}},{"leaderboard":"/sota/video-summarization-on-videoxum","slug":"video-summarization-on-videoxum","dataset":"videoxum","dataset_url":"/dataset/videoxum-1","rows_in_archive":1,"metrics":["1 shot Micro-F1"],"first_row_in_archive_order":{"model":"VTSUM-BLIP","paper_title":"VideoXum: Cross-modal Visual and Textural Summarization of Videos","paper_url":"/paper/videoxum-cross-modal-visual-and-textural","paper_date":"2023-03-21","arxiv_id":"2303.12060","code_links":[{"title":"jylins/videoxum","url":"https://github.com/jylins/videoxum"}],"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":1}}}],"datasets":[{"url":"/dataset/summe","name":"SumMe","full_name":"SumMe","num_papers_in_archive":146},{"url":"/dataset/tvsum","name":"TvSum","full_name":"TVSum: Summarizing Web Videos Using Titles","num_papers_in_archive":28},{"url":"/dataset/hyper-kvasir-dataset","name":"Hyper-Kvasir Dataset","full_name":"","num_papers_in_archive":12},{"url":"/dataset/video2gif","name":"Video2GIF","full_name":"","num_papers_in_archive":11},{"url":"/dataset/videoxum-1","name":"VideoXum","full_name":"","num_papers_in_archive":8},{"url":"/dataset/query-focused-video-summarization-dataset","name":"Query-Focused Video Summarization Dataset","full_name":"","num_papers_in_archive":4},{"url":"/dataset/tour20","name":"Tour20","full_name":"","num_papers_in_archive":4},{"url":"/dataset/ego4d-hcap","name":"Ego4D-HCap","full_name":"","num_papers_in_archive":3},{"url":"/dataset/shot2story20k","name":"Shot2Story20K","full_name":"","num_papers_in_archive":3},{"url":"/dataset/rad","name":"RAD","full_name":"RELEVANCE AND DIVERSITY DATASET","num_papers_in_archive":2},{"url":"/dataset/mr-hisum","name":"Mr. HiSum","full_name":"","num_papers_in_archive":1},{"url":"/dataset/multi-ego","name":"Multi-Ego","full_name":"","num_papers_in_archive":1},{"url":"/dataset/multisum","name":"MultiSum","full_name":"","num_papers_in_archive":1},{"url":"/dataset/mvs1k","name":"MVS1K","full_name":"","num_papers_in_archive":1},{"url":"/dataset/ostd","name":"OSTD","full_name":"Open-Source-Total-Distance","num_papers_in_archive":1},{"url":"/dataset/soccernet-echoes-1","name":"SoccerNet-Echoes","full_name":"SoccerNet-Echoes: A Soccer Game Audio Commentary Dataset","num_papers_in_archive":1}],"subtasks":[{"url":"/task/supervised-video-summarization","name":"Supervised Video Summarization"},{"url":"/task/unsupervised-video-summarization","name":"Unsupervised Video Summarization"}],"parent_tasks":[{"url":"/task/video","name":"Video"}],"papers":{"order":"repositories listed in the archive (desc), then date (desc); the archive holds no stars","population":"papers tagged with this task that list at least one repository in the archive","shown":30,"of":89,"tagged_in_all":280,"items":[{"url":"/paper/deep-reinforcement-learning-for-unsupervised","title":"Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward","date":"2017-12-29","arxiv_id":"1801.00054","repositories_listed":6,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/summarizing-videos-with-attention","title":"Summarizing Videos with Attention","date":"2018-12-05","arxiv_id":"1812.01969","repositories_listed":5,"syntology":{"n":15,"n_ran":1,"n_unverified":14,"n_pointer_only":0}},{"url":"/paper/gpt2mvs-generative-pre-trained-transformer-2","title":"GPT2MVS: Generative Pre-trained Transformer-2 for Multi-modal Video Summarization","date":"2021-04-26","arxiv_id":"2104.12465","repositories_listed":4,"syntology":null},{"url":"/paper/supervised-video-summarization-via-multiple","title":"Supervised Video Summarization via Multiple Feature Sets with Parallel Attention","date":"2021-04-23","arxiv_id":"2104.11530","repositories_listed":3,"syntology":null},{"url":"/paper/video-summarization-using-keyframe-extraction","title":"Unsupervised video summarization framework using keyframe extraction and video skimming","date":"2019-10-10","arxiv_id":"1910.04792","repositories_listed":3,"syntology":null},{"url":"/paper/multisum-a-dataset-for-multimodal","title":"MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos","date":"2023-06-07","arxiv_id":"2306.04216","repositories_listed":2,"syntology":null},{"url":"/paper/align-and-attend-multimodal-summarization","title":"Align and Attend: Multimodal Summarization with Dual Contrastive Losses","date":"2023-03-13","arxiv_id":"2303.07284","repositories_listed":2,"syntology":null},{"url":"/paper/egocentric-video-language-pretraining","title":"Egocentric Video-Language Pretraining","date":"2022-06-03","arxiv_id":"2206.01670","repositories_listed":2,"syntology":{"n":3,"n_ran":3,"n_unverified":0,"n_pointer_only":2}},{"url":"/paper/video-summarization-based-on-video-text","title":"Progressive Video Summarization via Multimodal Self-supervised Learning","date":"2022-01-07","arxiv_id":"2201.02494","repositories_listed":2,"syntology":null},{"url":"/paper/video-joint-modelling-based-on-hierarchical","title":"Video Joint Modelling Based on Hierarchical Transformer for Co-summarization","date":"2021-12-27","arxiv_id":"2112.13478","repositories_listed":2,"syntology":null},{"url":"/paper/convolutional-hierarchical-attention-network","title":"Convolutional Hierarchical Attention Network for Query-Focused Video Summarization","date":"2020-01-31","arxiv_id":"2002.03740","repositories_listed":2,"syntology":null},{"url":"/paper/rethinking-the-evaluation-of-video-summaries","title":"Rethinking the Evaluation of Video Summaries","date":"2019-03-27","arxiv_id":"1903.11328","repositories_listed":2,"syntology":{"n":1,"n_ran":0,"n_unverified":1,"n_pointer_only":0}},{"url":"/paper/iterative-projection-and-matching-finding","title":"Iterative Projection and Matching: Finding Structure-preserving Representatives and Its Application to Computer Vision","date":"2018-11-29","arxiv_id":"1811.12326","repositories_listed":2,"syntology":null},{"url":"/paper/video-summarization-using-deep-semantic","title":"Video Summarization using Deep Semantic Features","date":"2016-09-28","arxiv_id":"1609.08758","repositories_listed":2,"syntology":null},{"url":"/paper/sd-vsum-a-method-and-dataset-for-script","title":"SD-VSum: A Method and Dataset for Script-Driven Video Summarization","date":"2025-05-06","arxiv_id":"2505.03319","repositories_listed":1,"syntology":null},{"url":"/paper/an-egocentric-vision-language-model-based","title":"An Egocentric Vision-Language Model based Portable Real-time Smart Assistant","date":"2025-03-06","arxiv_id":"2503.04250","repositories_listed":1,"syntology":null},{"url":"/paper/integrate-the-temporal-scheme-for","title":"Integrate the temporal scheme for unsupervised video summarization via attention mechanism","date":"2025-02-26","arxiv_id":null,"repositories_listed":1,"syntology":null},{"url":"/paper/what-is-that-talk-about-a-video-to-text","title":"What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentations","date":"2025-02-12","arxiv_id":"2502.08279","repositories_listed":1,"syntology":null},{"url":"/paper/do-language-models-understand-time","title":"Do Language Models Understand Time?","date":"2024-12-18","arxiv_id":"2412.13845","repositories_listed":1,"syntology":null},{"url":"/paper/video-repurposing-from-user-generated-content","title":"Video Repurposing from User Generated Content: A Large-scale Dataset and Benchmark","date":"2024-12-12","arxiv_id":"2412.08879","repositories_listed":1,"syntology":{"n":11,"n_ran":0,"n_unverified":11,"n_pointer_only":0}},{"url":"/paper/your-interest-your-summaries-query-focused","title":"Your Interest, Your Summaries: Query-Focused Long Video Summarization","date":"2024-10-17","arxiv_id":"2410.14087","repositories_listed":1,"syntology":null},{"url":"/paper/does-spatiotemporal-information-benefit-two","title":"Does SpatioTemporal information benefit Two video summarization benchmarks?","date":"2024-10-04","arxiv_id":"2410.03323","repositories_listed":1,"syntology":null},{"url":"/paper/video-xl-extra-long-vision-language-model-for","title":"Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding","date":"2024-09-22","arxiv_id":"2409.14485","repositories_listed":1,"syntology":{"n":3,"n_ran":3,"n_unverified":0,"n_pointer_only":2}},{"url":"/paper/ubiss-a-unified-framework-for-bimodal","title":"UBiSS: A Unified Framework for Bimodal Semantic Summarization of Videos","date":"2024-06-24","arxiv_id":"2406.16301","repositories_listed":1,"syntology":null},{"url":"/paper/a-human-annotated-video-dataset-for-training","title":"A Human-Annotated Video Dataset for Training and Evaluation of 360-Degree Video Summarization Methods","date":"2024-06-05","arxiv_id":"2406.02991","repositories_listed":1,"syntology":null},{"url":"/paper/shotluck-holmes-a-family-of-efficient-small","title":"Shotluck Holmes: A Family of Efficient Small-Scale Large Language Vision Models For Video Captioning and Summarization","date":"2024-05-31","arxiv_id":"2405.20648","repositories_listed":1,"syntology":null},{"url":"/paper/vtg-llm-integrating-timestamp-knowledge-into","title":"VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Grounding","date":"2024-05-22","arxiv_id":"2405.13382","repositories_listed":1,"syntology":{"n":17,"n_ran":13,"n_unverified":4,"n_pointer_only":0}},{"url":"/paper/csta-cnn-based-spatiotemporal-attention-for","title":"CSTA: CNN-based Spatiotemporal Attention for Video Summarization","date":"2024-05-20","arxiv_id":"2405.11905","repositories_listed":1,"syntology":{"n":8,"n_ran":7,"n_unverified":1,"n_pointer_only":0}},{"url":"/paper/an-integrated-framework-for-multi-granular","title":"An Integrated Framework for Multi-Granular Explanation of Video Summarization","date":"2024-05-16","arxiv_id":"2405.10082","repositories_listed":1,"syntology":null},{"url":"/paper/videosage-video-summarization-with-graph","title":"VideoSAGE: Video Summarization with Graph Representation Learning","date":"2024-04-14","arxiv_id":"2404.10539","repositories_listed":1,"syntology":null}],"syntology_records":8,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-09-24T18:15:14+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; our fixture could not drive it","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"}},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"}}