{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/youtube-vos-sequence-to-sequence-video-object","title":"YouTube-VOS: Sequence-to-Sequence Video Object Segmentation","arxiv_id":"1809.00461","date":"2018-09-03","proceeding":"ECCV 2018 9","authors":["Ning Xu","Linjie Yang","Yuchen Fan","Jianchao Yang","Dingcheng Yue","Yuchen Liang","Brian Price","Scott Cohen","Thomas Huang"],"abstract":"Learning long-term spatial-temporal features are critical for many video\nanalysis tasks. However, existing video segmentation methods predominantly rely\non static image segmentation techniques, and methods capturing temporal\ndependency for segmentation have to depend on pretrained optical flow models,\nleading to suboptimal solutions for the problem. End-to-end sequential learning\nto explore spatial-temporal features for video segmentation is largely limited\nby the scale of available video segmentation datasets, i.e., even the largest\nvideo segmentation dataset only contains 90 short video clips. To solve this\nproblem, we build a new large-scale video object segmentation dataset called\nYouTube Video Object Segmentation dataset (YouTube-VOS). Our dataset contains\n3,252 YouTube video clips and 78 categories including common objects and human\nactivities. This is by far the largest video object segmentation dataset to our\nknowledge and we have released it at https://youtube-vos.org. Based on this\ndataset, we propose a novel sequence-to-sequence network to fully exploit\nlong-term spatial-temporal information in videos for segmentation. We\ndemonstrate that our method is able to achieve the best results on our\nYouTube-VOS test set and comparable results on DAVIS 2016 compared to the\ncurrent state-of-the-art methods. Experiments show that the large scale dataset\nis indeed a key factor to the success of our model.","url_abs":"http://arxiv.org/abs/1809.00461v1","url_pdf":"http://arxiv.org/pdf/1809.00461v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"youtube-vos-sequence-to-sequence-video-object","repo_url":"https://github.com/BehradToghi/ConvLSTM_VOS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"youtube-vos-sequence-to-sequence-video-object","repo_url":"https://github.com/BehradToghi/ECCV_Youtube_VOS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"youtube-vos-sequence-to-sequence-video-object","repo_url":"https://github.com/T-a-g-z/Yotube-VOS-OVS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"youtube-vos-sequence-to-sequence-video-object","repo_url":"https://github.com/shashankvkt/video_object_segmentation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"object","task_name":"Object"},{"task_slug":"one-shot-visual-object-segmentation","task_name":"One-shot visual object segmentation"},{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"semi-supervised-video-object-segmentation","task_name":"Semi-Supervised Video Object Segmentation"},{"task_slug":"video-object-segmentation","task_name":"Video Object Segmentation"},{"task_slug":"video-segmentation","task_name":"Video Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"},{"task_slug":"visual-object-tracking","task_name":"Visual Object Tracking"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-object-segmentation-on-youtube-vos","task":"Semi-Supervised Video Object Segmentation","dataset":"YouTube-VOS 2018","model":"S2S","rank_in_archive_order":48,"of":53,"metrics":{"F-Measure (Seen)":"70.0","F-Measure (Unseen)":"61.2","Jaccard (Seen)":"71.0","Overall":"64.4","Speed  (FPS)":"55.5"},"uses_additional_data":false},{"leaderboard":"/sota/video-object-segmentation-on-youtube-vos-1","task":"Video Object Segmentation","dataset":"YouTube-VOS 2018","model":"S2S (offline)","rank_in_archive_order":17,"of":17,"metrics":{"F-Measure (Unseen)":"50.3"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1809.00461","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}