{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ctvis-consistent-training-for-online-video","title":"CTVIS: Consistent Training for Online Video Instance Segmentation","arxiv_id":"2307.12616","date":"2023-07-24","proceeding":"ICCV 2023 1","authors":["Kaining Ying","Qing Zhong","Weian Mao","Zhenhua Wang","Hao Chen","Lin Yuanbo Wu","Yifan Liu","Chengxiang Fan","Yunzhi Zhuge","Chunhua Shen"],"abstract":"The discrimination of instance embeddings plays a vital role in associating instances across time for online video instance segmentation (VIS). Instance embedding learning is directly supervised by the contrastive loss computed upon the contrastive items (CIs), which are sets of anchor/positive/negative embeddings. Recent online VIS methods leverage CIs sourced from one reference frame only, which we argue is insufficient for learning highly discriminative embeddings. Intuitively, a possible strategy to enhance CIs is replicating the inference phase during training. To this end, we propose a simple yet effective training strategy, called Consistent Training for Online VIS (CTVIS), which devotes to aligning the training and inference pipelines in terms of building CIs. Specifically, CTVIS constructs CIs by referring inference the momentum-averaged embedding and the memory bank storage mechanisms, and adding noise to the relevant embeddings. Such an extension allows a reliable comparison between embeddings of current instances and the stable representations of historical instances, thereby conferring an advantage in modeling VIS challenges such as occlusion, re-identification, and deformation. Empirically, CTVIS outstrips the SOTA VIS models by up to +5.0 points on three VIS benchmarks, including YTVIS19 (55.1% AP), YTVIS21 (50.1% AP) and OVIS (35.5% AP). Furthermore, we find that pseudo-videos transformed from images can train robust models surpassing fully-supervised ones.","url_abs":"https://arxiv.org/abs/2307.12616v1","url_pdf":"https://arxiv.org/pdf/2307.12616v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ctvis-consistent-training-for-online-video","repo_url":"https://github.com/kainingying/ctvis","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-instance-segmentation","task_name":"Video Instance Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-instance-segmentation-on-ovis-1","task":"Video Instance Segmentation","dataset":"OVIS validation","model":"CTVIS (Swin-L)","rank_in_archive_order":9,"of":44,"metrics":{"AP50":"71.5","AP75":"47.5","APho":"19.1","APmo":"52.1","mask AP":"46.9"},"uses_additional_data":true},{"leaderboard":"/sota/video-instance-segmentation-on-ovis-1","task":"Video Instance Segmentation","dataset":"OVIS validation","model":"CTVIS (ResNet-50)","rank_in_archive_order":24,"of":44,"metrics":{"AP50":"60.8","AP75":"34.9","APho":"16.1","APmo":"41.9","mask AP":"35.5"},"uses_additional_data":true},{"leaderboard":"/sota/video-instance-segmentation-on-youtube-vis-3","task":"Video Instance Segmentation","dataset":"Youtube-VIS 2022 Validation","model":"CTVIS (Swin-L)","rank_in_archive_order":3,"of":7,"metrics":{"mAP_L":"46.4"},"uses_additional_data":true},{"leaderboard":"/sota/video-instance-segmentation-on-youtube-vis-3","task":"Video Instance Segmentation","dataset":"Youtube-VIS 2022 Validation","model":"CTVIS (ResNet-50)","rank_in_archive_order":5,"of":7,"metrics":{"mAP_L":"39.4"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2307.12616","atlas_url":"https://app.syntology.ai/?focus=2307.12616","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.12616"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kainingying/ctvis","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":3,"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":7,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e0a06ded5d4f6c3c","entry":"box_cxcywh_to_xyxy","repo":"kainingying/ctvis","repo_kind":"official","path":"ctvis/utils/utils.py","file_url":"https://github.com/kainingying/ctvis/blob/HEAD/ctvis/utils/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e0a06ded5d4f6c3c"}},{"code_sha256_prefix":"ce749f837424dd8c","entry":"box_iou","repo":"kainingying/ctvis","repo_kind":"official","path":"ctvis/utils/utils.py","file_url":"https://github.com/kainingying/ctvis/blob/HEAD/ctvis/utils/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce749f837424dd8c"}},{"code_sha256_prefix":"031b02fb990284ba","entry":"box_xyxy_to_cxcywh","repo":"kainingying/ctvis","repo_kind":"official","path":"ctvis/utils/utils.py","file_url":"https://github.com/kainingying/ctvis/blob/HEAD/ctvis/utils/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"031b02fb990284ba"}},{"code_sha256_prefix":"0c1f7797c3d1f89d","entry":"retry_if_cuda_oom","repo":"kainingying/ctvis","repo_kind":"official","path":"ctvis/utils/memory.py","file_url":"https://github.com/kainingying/ctvis/blob/HEAD/ctvis/utils/memory.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c1f7797c3d1f89d"}},{"code_sha256_prefix":"8e5e8f7bf1306736","entry":"get_sine_pos_embed","repo":"kainingying/ctvis","repo_kind":"official","path":"ctvis/modeling/transformer_decoder/base_arch.py","file_url":"https://github.com/kainingying/ctvis/blob/HEAD/ctvis/modeling/transformer_decoder/base_arch.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e5e8f7bf1306736"}},{"code_sha256_prefix":"48a9de57443c5046","entry":"loss_reid","repo":"kainingying/ctvis","repo_kind":"official","path":"ctvis/modeling/cl_plugin/ct_cl_plugin.py","file_url":"https://github.com/kainingying/ctvis/blob/HEAD/ctvis/modeling/cl_plugin/ct_cl_plugin.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48a9de57443c5046"}},{"code_sha256_prefix":"4037603d511c526c","entry":"loss_reid","repo":"kainingying/ctvis","repo_kind":"official","path":"ctvis/modeling/cl_plugin/multi_ref_cl_plugin.py","file_url":"https://github.com/kainingying/ctvis/blob/HEAD/ctvis/modeling/cl_plugin/multi_ref_cl_plugin.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4037603d511c526c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}