{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/kernelized-memory-network-for-video-object","title":"Kernelized Memory Network for Video Object Segmentation","arxiv_id":"2007.08270","date":"2020-07-16","proceeding":"ECCV 2020 8","authors":["Hongje Seong","Junhyuk Hyun","Euntai Kim"],"abstract":"Semi-supervised video object segmentation (VOS) is a task that involves predicting a target object in a video when the ground truth segmentation mask of the target object is given in the first frame. Recently, space-time memory networks (STM) have received significant attention as a promising solution for semi-supervised VOS. However, an important point is overlooked when applying STM to VOS. The solution (STM) is non-local, but the problem (VOS) is predominantly local. To solve the mismatch between STM and VOS, we propose a kernelized memory network (KMN). Before being trained on real videos, our KMN is pre-trained on static images, as in previous works. Unlike in previous works, we use the Hide-and-Seek strategy in pre-training to obtain the best possible results in handling occlusions and segment boundary extraction. The proposed KMN surpasses the state-of-the-art on standard benchmarks by a significant margin (+5% on DAVIS 2017 test-dev set). In addition, the runtime of KMN is 0.12 seconds per frame on the DAVIS 2016 validation set, and the KMN rarely requires extra computation, when compared with STM.","url_abs":"https://arxiv.org/abs/2007.08270v1","url_pdf":"https://arxiv.org/pdf/2007.08270v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"kernelized-memory-network-for-video-object","repo_url":"https://github.com/hkchengrex/Mask-Propagation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"object","task_name":"Object"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"semi-supervised-video-object-segmentation","task_name":"Semi-Supervised Video Object Segmentation"},{"task_slug":"video-object-segmentation","task_name":"Video Object Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[{"method_slug":"memory-network","method_name":"Memory Network"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/semi-supervised-video-object-segmentation-on-20","task":"Semi-Supervised Video Object Segmentation","dataset":"DAVIS (no YouTube-VOS training)","model":"KMN","rank_in_archive_order":7,"of":26,"metrics":{"D16 val (F)":"88.1","D16 val (G)":"87.6","D16 val (J)":"87.1","D17 val (F)":"77.8","D17 val (G)":"76.0","D17 val (J)":"74.2","FPS":"8.33"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-davis-2016","task":"Semi-Supervised Video Object Segmentation","dataset":"DAVIS 2016","model":"KMN","rank_in_archive_order":28,"of":78,"metrics":{"F-measure (Mean)":"91.5","J&F":"90.5","Jaccard (Mean)":"89.5"},"uses_additional_data":false},{"leaderboard":"/sota/semi-supervised-video-object-segmentation-on-1","task":"Semi-Supervised Video Object Segmentation","dataset":"DAVIS 2017 (test-dev)","model":"KMN","rank_in_archive_order":29,"of":59,"metrics":{"F-measure (Mean)":"80.3","J&F":"77.2","Jaccard (Mean)":"74.1"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-davis-2017","task":"Semi-Supervised Video Object Segmentation","dataset":"DAVIS 2017 (val)","model":"KMN","rank_in_archive_order":36,"of":81,"metrics":{"F-measure (Mean)":"85.6","J&F":"82.8","Jaccard (Mean)":"80"},"uses_additional_data":false},{"leaderboard":"/sota/video-object-segmentation-on-youtube-vos","task":"Semi-Supervised Video Object Segmentation","dataset":"YouTube-VOS 2018","model":"KMN","rank_in_archive_order":37,"of":53,"metrics":{"F-Measure (Seen)":"85.6","F-Measure (Unseen)":"83.3","Jaccard (Seen)":"81.4","Jaccard (Unseen)":"75.3","Overall":"81.4"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2007.08270","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}