{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/susinet-see-understand-and-summarize-it","title":"SUSiNet: See, Understand and Summarize it","arxiv_id":"1812.00722","date":"2018-12-03","proceeding":null,"authors":["Petros Koutras","Petros Maragos"],"abstract":"In this work we propose a multi-task spatio-temporal network, called SUSiNet,\nthat can jointly tackle the spatio-temporal problems of saliency estimation,\naction recognition and video summarization. Our approach employs a single\nnetwork that is jointly end-to-end trained for all tasks with multiple and\ndiverse datasets related to the exploring tasks. The proposed network uses a\nunified architecture that includes global and task specific layer and produces\nmultiple output types, i.e., saliency maps or classification labels, by\nemploying the same video input. Moreover, one additional contribution is that\nthe proposed network can be deeply supervised through an attention module that\nis related to human attention as it is expressed by eye-tracking data. From the\nextensive evaluation, on seven different datasets, we have observed that the\nmulti-task network performs as well as the state-of-the-art single-task methods\n(or in some cases better), while it requires less computational budget than\nhaving one independent network per each task.","url_abs":"http://arxiv.org/abs/1812.00722v2","url_pdf":"http://arxiv.org/pdf/1812.00722v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"saliency-prediction","task_name":"Saliency Prediction"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"},{"task_slug":"video-summarization","task_name":"Video Summarization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-recognition-in-videos-on-hmdb-51","task":"Action Recognition","dataset":"HMDB-51","model":"SUSiNet (multi, Kinetics pretrained)","rank_in_archive_order":67,"of":77,"metrics":{"Average accuracy of 3 splits":"62.7"},"uses_additional_data":true}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}