{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/stfcn-spatio-temporal-fcn-for-semantic-video","title":"STFCN: Spatio-Temporal FCN for Semantic Video Segmentation","arxiv_id":"1608.05971","date":"2016-08-21","proceeding":null,"authors":["Mohsen Fayyaz","Mohammad Hajizadeh Saffar","Mohammad Sabokrou","Mahmood Fathy","Reinhard klette","Fay Huang"],"abstract":"This paper presents a novel method to involve both spatial and temporal\nfeatures for semantic video segmentation. Current work on convolutional neural\nnetworks(CNNs) has shown that CNNs provide advanced spatial features supporting\na very good performance of solutions for both image and video analysis,\nespecially for the semantic segmentation task. We investigate how involving\ntemporal features also has a good effect on segmenting video data. We propose a\nmodule based on a long short-term memory (LSTM) architecture of a recurrent\nneural network for interpreting the temporal characteristics of video frames\nover time. Our system takes as input frames of a video and produces a\ncorrespondingly-sized output; for segmenting the video our method combines the\nuse of three components: First, the regional spatial features of frames are\nextracted using a CNN; then, using LSTM the temporal features are added;\nfinally, by deconvolving the spatio-temporal features we produce pixel-wise\npredictions. Our key insight is to build spatio-temporal convolutional networks\n(spatio-temporal CNNs) that have an end-to-end architecture for semantic video\nsegmentation. We adapted fully some known convolutional network architectures\n(such as FCN-AlexNet and FCN-VGG16), and dilated convolution into our\nspatio-temporal CNNs. Our spatio-temporal CNNs achieve state-of-the-art\nsemantic segmentation, as demonstrated for the Camvid and NYUDv2 datasets.","url_abs":"http://arxiv.org/abs/1608.05971v2","url_pdf":"http://arxiv.org/pdf/1608.05971v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"stfcn-spatio-temporal-fcn-for-semantic-video","repo_url":"https://github.com/MohsenFayyaz89/STFCN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"torch","reach":null}],"tasks":[{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-segmentation","task_name":"Video Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dilated-convolution","method_name":"Dilated Convolution"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1608.05971","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}