{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/relational-reasoning-over-spatial-temporal","title":"Relational Reasoning Over Spatial-Temporal Graphs for Video Summarization","arxiv_id":null,"date":"2022-04-06","proceeding":"IEEE Transactions on Image Processing 2022 4","authors":["Wencheng Zhu","Yucheng Han","Jiwen Lu","Jie zhou"],"abstract":"In this paper, we propose a dynamic graph modeling approach to learn spatial-temporal representations for video summarization. Most existing video summarization methods extract image-level features with ImageNet pre-trained deep models. Differently, our method exploits object-level and relation-level information to capture spatial-temporal dependencies. Specifically, our method builds spatial graphs on the detected object proposals. Then, we construct a temporal graph by using the aggregated representations of spatial graphs. Afterward, we perform relational reasoning over spatial and temporal graphs with graph convolutional networks and extract spatial-temporal representations for importance score prediction and key shot selection. To eliminate relation clutters caused by densely connected nodes, we further design a self-attention edge pooling module, which disregards meaningless relations of graphs. We conduct extensive experiments on two popular benchmarks, including the SumMe and TVSum datasets. Experimental results demonstrate that the proposed method achieves superior performance against state-of-the-art video summarization methods.","url_abs":"https://ieeexplore.ieee.org/abstract/document/9750933","url_pdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9750933","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"graph-classification","task_name":"Graph Classification"},{"task_slug":null,"task_name":"Relation"},{"task_slug":"relational-reasoning","task_name":"Relational Reasoning"},{"task_slug":"supervised-video-summarization","task_name":"Supervised Video Summarization"},{"task_slug":"video-summarization","task_name":"Video Summarization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/graph-classification-on-nci1","task":"Graph Classification","dataset":"NCI1","model":"SAEPool_g","rank_in_archive_order":53,"of":69,"metrics":{"Accuracy":"74.48%"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-nci109","task":"Graph Classification","dataset":"NCI109","model":"SAEPool_h","rank_in_archive_order":25,"of":38,"metrics":{"Accuracy":"75.85"},"uses_additional_data":false},{"leaderboard":"/sota/graph-classification-on-proteins","task":"Graph Classification","dataset":"PROTEINS","model":"SAEPool","rank_in_archive_order":9,"of":103,"metrics":{"Accuracy":"80.36%"},"uses_additional_data":false},{"leaderboard":"/sota/supervised-video-summarization-on-summe","task":"Supervised Video Summarization","dataset":"SumMe","model":"RR-STG","rank_in_archive_order":7,"of":21,"metrics":{"F1-score (Augmented)":"54.8","F1-score (Canonical)":"53.4","Kendall's Tau":"0.211","Spearman's Rho":"0.234"},"uses_additional_data":false},{"leaderboard":"/sota/supervised-video-summarization-on-tvsum","task":"Supervised Video Summarization","dataset":"TvSum","model":"RR-STG","rank_in_archive_order":7,"of":21,"metrics":{"F1-score (Augmented)":"63.6","F1-score (Canonical)":"63.0","Kendall's Tau":"0.162","Spearman's Rho":"0.212"},"uses_additional_data":false},{"leaderboard":"/sota/video-summarization-on-summe","task":"Video Summarization","dataset":"SumMe","model":"RR-STG","rank_in_archive_order":2,"of":6,"metrics":{"F1-score (Augmented)":"55.3","F1-score (Canonical)":"54.5"},"uses_additional_data":false},{"leaderboard":"/sota/video-summarization-on-tvsum","task":"Video Summarization","dataset":"TvSum","model":"RR-STG","rank_in_archive_order":1,"of":6,"metrics":{"F1-score (Augmented)":"63.6","F1-score (Canonical)":"63.0"},"uses_additional_data":true}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}