{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mdqe-mining-discriminative-query-embeddings","title":"MDQE: Mining Discriminative Query Embeddings to Segment Occluded Instances on Challenging Videos","arxiv_id":"2303.14395","date":"2023-03-25","proceeding":"CVPR 2023 1","authors":["Minghan Li","Shuai Li","Wangmeng Xiang","Lei Zhang"],"abstract":"While impressive progress has been achieved, video instance segmentation (VIS) methods with per-clip input often fail on challenging videos with occluded objects and crowded scenes. This is mainly because instance queries in these methods cannot encode well the discriminative embeddings of instances, making the query-based segmenter difficult to distinguish those `hard' instances. To address these issues, we propose to mine discriminative query embeddings (MDQE) to segment occluded instances on challenging videos. First, we initialize the positional embeddings and content features of object queries by considering their spatial contextual information and the inter-frame object motion. Second, we propose an inter-instance mask repulsion loss to distance each instance from its nearby non-target instances. The proposed MDQE is the first VIS method with per-clip input that achieves state-of-the-art results on challenging videos and competitive performance on simple videos. In specific, MDQE with ResNet50 achieves 33.0\\% and 44.5\\% mask AP on OVIS and YouTube-VIS 2021, respectively. Code of MDQE can be found at \\url{https://github.com/MinghanLi/MDQE_CVPR2023}.","url_abs":"https://arxiv.org/abs/2303.14395v1","url_pdf":"https://arxiv.org/pdf/2303.14395v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mdqe-mining-discriminative-query-embeddings","repo_url":"https://github.com/minghanli/mdqe_cvpr2023","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-instance-segmentation","task_name":"Video Instance Segmentation"}],"methods":[{"method_slug":"fail","method_name":"fail"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-instance-segmentation-on-ovis-1","task":"Video Instance Segmentation","dataset":"OVIS validation","model":"MDQE(SwinL)","rank_in_archive_order":15,"of":44,"metrics":{"AP50":"67.8","AP75":"44.3","APho":"21.6","APmo":"49.3","APso":"65.1","AR1":"18.3","AR10":"46.5","mask AP":"42.6"},"uses_additional_data":false},{"leaderboard":"/sota/video-instance-segmentation-on-youtube-vis-2","task":"Video Instance Segmentation","dataset":"YouTube-VIS 2021","model":"MDQE(Swin-L)","rank_in_archive_order":15,"of":26,"metrics":{"AP50":"80.7","AP75":"61.7","AR1":"45.4","AR10":"60.6","mask AP":"55.5"},"uses_additional_data":false},{"leaderboard":"/sota/video-instance-segmentation-on-youtube-vis-1","task":"Video Instance Segmentation","dataset":"YouTube-VIS validation","model":"MDQE(Swin-L)","rank_in_archive_order":8,"of":44,"metrics":{"AP50":"84.9","AP75":"67.3","AR1":"53.5","AR10":"65.0","mask AP":"59.9"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.14395","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.14395"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/MinghanLi/MDQE_CVPR2023","reach":null}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"07f457fdc5fd31da","entry":"NestedTensor","repo":"MinghanLi/MDQE_CVPR2023","repo_kind":"official","path":"mdqe/models/mdqe.py","file_url":"https://github.com/MinghanLi/MDQE_CVPR2023/blob/HEAD/mdqe/models/mdqe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"07f457fdc5fd31da"}},{"code_sha256_prefix":"d7c6d68b4e76b6c2","entry":"mdqe","repo":"MinghanLi/MDQE_CVPR2023","repo_kind":"official","path":"mdqe/models/mdqe.py","file_url":"https://github.com/MinghanLi/MDQE_CVPR2023/blob/HEAD/mdqe/models/mdqe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d7c6d68b4e76b6c2"}},{"code_sha256_prefix":"2f3dddf6597f2f3c","entry":"nested_tensor_from_tensor_list","repo":"MinghanLi/MDQE_CVPR2023","repo_kind":"official","path":"mdqe/models/mdqe.py","file_url":"https://github.com/MinghanLi/MDQE_CVPR2023/blob/HEAD/mdqe/models/mdqe.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2f3dddf6597f2f3c"}},{"code_sha256_prefix":"998843adabc1ce0f","entry":"_onnx_nested_tensor_from_tensor_list","repo":"MinghanLi/MDQE_CVPR2023","repo_kind":"official","path":"mdqe/models/mdqe.py","file_url":"https://github.com/MinghanLi/MDQE_CVPR2023/blob/HEAD/mdqe/models/mdqe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"998843adabc1ce0f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}