{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/in-defense-of-online-models-for-video","title":"In Defense of Online Models for Video Instance Segmentation","arxiv_id":"2207.10661","date":"2022-07-21","proceeding":null,"authors":["Junfeng Wu","Qihao Liu","Yi Jiang","Song Bai","Alan Yuille","Xiang Bai"],"abstract":"In recent years, video instance segmentation (VIS) has been largely advanced by offline models, while online models gradually attracted less attention possibly due to their inferior performance. However, online methods have their inherent advantage in handling long video sequences and ongoing videos while offline models fail due to the limit of computational resources. Therefore, it would be highly desirable if online models can achieve comparable or even better performance than offline models. By dissecting current online models and offline models, we demonstrate that the main cause of the performance gap is the error-prone association between frames caused by the similar appearance among different instances in the feature space. Observing this, we propose an online framework based on contrastive learning that is able to learn more discriminative instance embeddings for association and fully exploit history information for stability. Despite its simplicity, our method outperforms all online and offline methods on three benchmarks. Specifically, we achieve 49.5 AP on YouTube-VIS 2019, a significant improvement of 13.2 AP and 2.1 AP over the prior online and offline art, respectively. Moreover, we achieve 30.2 AP on OVIS, a more challenging dataset with significant crowding and occlusions, surpassing the prior art by 14.8 AP. The proposed method won first place in the video instance segmentation track of the 4th Large-scale Video Object Segmentation Challenge (CVPR2022). We hope the simplicity and effectiveness of our method, as well as our insight into current methods, could shed light on the exploration of VIS models.","url_abs":"https://arxiv.org/abs/2207.10661v1","url_pdf":"https://arxiv.org/pdf/2207.10661v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"in-defense-of-online-models-for-video","repo_url":"https://github.com/wjf5203/vnext","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"in-defense-of-online-models-for-video","repo_url":"https://github.com/mkimhi/RISE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-instance-segmentation","task_name":"Video Instance Segmentation"},{"task_slug":"video-object-segmentation","task_name":"Video Object Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-instance-segmentation-on-ovis-1","task":"Video Instance Segmentation","dataset":"OVIS validation","model":"IDOL (Swin-L)","rank_in_archive_order":16,"of":44,"metrics":{"AP50":"65.7","AP75":"45.2","AR1":"17.9","AR10":"49.6","mask AP":"42.6"},"uses_additional_data":false},{"leaderboard":"/sota/video-instance-segmentation-on-ovis-1","task":"Video Instance Segmentation","dataset":"OVIS validation","model":"IDOL (ResNet-50)","rank_in_archive_order":30,"of":44,"metrics":{"AP50":"51.3","AP75":"30","AR1":"15","AR10":"37.5","mask AP":"30.2"},"uses_additional_data":false},{"leaderboard":"/sota/video-instance-segmentation-on-youtube-vis-2","task":"Video Instance Segmentation","dataset":"YouTube-VIS 2021","model":"IDOL (Swin-L)","rank_in_archive_order":14,"of":26,"metrics":{"AP50":"80.8","AP75":"63.5","AR1":"45","AR10":"60.1","mask AP":"56.1"},"uses_additional_data":false},{"leaderboard":"/sota/video-instance-segmentation-on-youtube-vis-1","task":"Video Instance Segmentation","dataset":"YouTube-VIS validation","model":"IDOL (ResNet-50)","rank_in_archive_order":15,"of":44,"metrics":{"AP50":"74","AP75":"52.9","AR1":"47.7","AR10":"58.7","mask AP":"49.5"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2207.10661","atlas_url":"https://app.syntology.ai/?focus=2207.10661","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.10661"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mkimhi/RISE","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wjf5203/vnext","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/wjf5203/VNext","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"ran_fixture":1,"unverified":2},"by_repo_kind":{"listed":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"84ec806321d4d1db","entry":"dynamic_k_matching","repo":"mkimhi/RISE","repo_kind":"listed","path":"projects/RISE/rise/models/pos_neg_select.py","file_url":"https://github.com/mkimhi/RISE/blob/HEAD/projects/RISE/rise/models/pos_neg_select.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"84ec806321d4d1db"}},{"code_sha256_prefix":"d0ce32748ce099f4","entry":"generalized_box_iou","repo":"mkimhi/RISE","repo_kind":"listed","path":"projects/RISE/rise/models/pos_neg_select.py","file_url":"https://github.com/mkimhi/RISE/blob/HEAD/projects/RISE/rise/models/pos_neg_select.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d0ce32748ce099f4"}},{"code_sha256_prefix":"c10508ee5ab04682","entry":"get_in_boxes_info","repo":"mkimhi/RISE","repo_kind":"listed","path":"projects/RISE/rise/models/pos_neg_select.py","file_url":"https://github.com/mkimhi/RISE/blob/HEAD/projects/RISE/rise/models/pos_neg_select.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c10508ee5ab04682"}},{"code_sha256_prefix":"801c3bc9596af85a","entry":"get_pos_idx","repo":"mkimhi/RISE","repo_kind":"listed","path":"projects/RISE/rise/models/pos_neg_select.py","file_url":"https://github.com/mkimhi/RISE/blob/HEAD/projects/RISE/rise/models/pos_neg_select.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"801c3bc9596af85a"}},{"code_sha256_prefix":"48e66a2981766770","entry":"select_pos_neg","repo":"mkimhi/RISE","repo_kind":"listed","path":"projects/RISE/rise/models/pos_neg_select.py","file_url":"https://github.com/mkimhi/RISE/blob/HEAD/projects/RISE/rise/models/pos_neg_select.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"48e66a2981766770"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}