{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/detrs-with-collaborative-hybrid-assignments","title":"DETRs with Collaborative Hybrid Assignments Training","arxiv_id":"2211.12860","date":"2022-11-22","proceeding":"ICCV 2023 1","authors":["Zhuofan Zong","Guanglu Song","Yu Liu"],"abstract":"In this paper, we provide the observation that too few queries assigned as positive samples in DETR with one-to-one set matching leads to sparse supervision on the encoder's output which considerably hurt the discriminative feature learning of the encoder and vice visa for attention learning in the decoder. To alleviate this, we present a novel collaborative hybrid assignments training scheme, namely $\\mathcal{C}$o-DETR, to learn more efficient and effective DETR-based detectors from versatile label assignment manners. This new training scheme can easily enhance the encoder's learning ability in end-to-end detectors by training the multiple parallel auxiliary heads supervised by one-to-many label assignments such as ATSS and Faster RCNN. In addition, we conduct extra customized positive queries by extracting the positive coordinates from these auxiliary heads to improve the training efficiency of positive samples in the decoder. In inference, these auxiliary heads are discarded and thus our method introduces no additional parameters and computational cost to the original detector while requiring no hand-crafted non-maximum suppression (NMS). We conduct extensive experiments to evaluate the effectiveness of the proposed approach on DETR variants, including DAB-DETR, Deformable-DETR, and DINO-Deformable-DETR. The state-of-the-art DINO-Deformable-DETR with Swin-L can be improved from 58.5% to 59.5% AP on COCO val. Surprisingly, incorporated with ViT-L backbone, we achieve 66.0% AP on COCO test-dev and 67.9% AP on LVIS val, outperforming previous methods by clear margins with much fewer model sizes. Codes are available at \\url{https://github.com/Sense-X/Co-DETR}.","url_abs":"https://arxiv.org/abs/2211.12860v5","url_pdf":"https://arxiv.org/pdf/2211.12860v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"detrs-with-collaborative-hybrid-assignments","repo_url":"https://github.com/sense-x/co-detr","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"detrs-with-collaborative-hybrid-assignments","repo_url":"https://github.com/anenbergb/Co-DETR-TensorRT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"detrs-with-collaborative-hybrid-assignments","repo_url":"https://github.com/siyuanliii/masa","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"detrs-with-collaborative-hybrid-assignments","repo_url":"https://github.com/MindCode-4/code-3/tree/main/detr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"detrs-with-collaborative-hybrid-assignments","repo_url":"https://github.com/code-implementation1/Code1/tree/main/detr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"detrs-with-collaborative-hybrid-assignments","repo_url":"https://github.com/open-mmlab/mmdetection","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"set-matching","task_name":"set matching"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"atss","method_name":"ATSS"},{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"detr","method_name":"Detr"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"fcos","method_name":"FCOS"},{"method_slug":"fpn","method_name":"FPN"},{"method_slug":"feedforward-network","method_name":"Feedforward Network"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"non-maximum-suppression","method_name":"Non Maximum Suppression"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/instance-segmentation-on-coco-minival","task":"Instance Segmentation","dataset":"COCO minival","model":"Co-DETR","rank_in_archive_order":1,"of":93,"metrics":{"AP50":"79.7","AP75":"62.8","APL":"74.6","APM":"59.7","APS":"38.9","mask AP":"56.6"},"uses_additional_data":true},{"leaderboard":"/sota/instance-segmentation-on-coco","task":"Instance Segmentation","dataset":"COCO test-dev","model":"Co-DETR","rank_in_archive_order":1,"of":112,"metrics":{"AP50":"80.2","AP75":"63.4","APL":"72.0","APM":"60.1","APS":"41.6","mask AP":"57.1"},"uses_additional_data":true},{"leaderboard":"/sota/instance-segmentation-on-lvis-v1-0-val","task":"Instance Segmentation","dataset":"LVIS v1.0 val","model":"Co-DETR (single-scale)","rank_in_archive_order":1,"of":25,"metrics":{"mask AP":"60.7"},"uses_additional_data":true},{"leaderboard":"/sota/object-detection-on-coco-minival","task":"Object Detection","dataset":"COCO minival","model":"Co-DETR","rank_in_archive_order":2,"of":220,"metrics":{"Params (M)":"314","box AP":"65.9"},"uses_additional_data":true},{"leaderboard":"/sota/object-detection-on-coco-minival","task":"Object Detection","dataset":"COCO minival","model":"Co-DETR (Swin-L)","rank_in_archive_order":5,"of":220,"metrics":{"Params (M)":"218","box AP":"64.7"},"uses_additional_data":true},{"leaderboard":"/sota/object-detection-on-coco","task":"Object Detection","dataset":"COCO test-dev","model":"Co-DETR","rank_in_archive_order":1,"of":225,"metrics":{"Params (M)":"304","box mAP":"66.0"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco","task":"Object Detection","dataset":"COCO test-dev","model":"Co-DETR (Swin-L)","rank_in_archive_order":6,"of":225,"metrics":{"Params (M)":"218","box mAP":"64.8"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-lvis-v1-0-minival","task":"Object Detection","dataset":"LVIS v1.0 minival","model":"Co-DETR (single-scale)","rank_in_archive_order":1,"of":6,"metrics":{"box AP":"72.0"},"uses_additional_data":true},{"leaderboard":"/sota/object-detection-on-lvis-v1-0-val","task":"Object Detection","dataset":"LVIS v1.0 val","model":"Co-DETR (single-scale)","rank_in_archive_order":1,"of":15,"metrics":{"box AP":"68.0"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.12860","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.12860"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/siyuanliii/masa","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/open-mmlab/mmdetection","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sense-x/co-detr","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/code-implementation1/Code1/tree/main/detr","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/anenbergb/Co-DETR-TensorRT","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindCode-4/code-3/tree/main/detr","reach":null}],"summary":{"unverified":5},"by_repo_kind":{"listed":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fb28f2ba259e8f13","entry":"build_MLP","repo":"anenbergb/Co-DETR-TensorRT","repo_kind":"listed","path":"codetr/transformer.py","file_url":"https://github.com/anenbergb/Co-DETR-TensorRT/blob/HEAD/codetr/transformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fb28f2ba259e8f13"}},{"code_sha256_prefix":"012e330e87684edc","entry":"get_reference_points","repo":"anenbergb/Co-DETR-TensorRT","repo_kind":"listed","path":"codetr/transformer.py","file_url":"https://github.com/anenbergb/Co-DETR-TensorRT/blob/HEAD/codetr/transformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"012e330e87684edc"}},{"code_sha256_prefix":"2f3e30980c10ac1c","entry":"make_encoder_output_proposals","repo":"anenbergb/Co-DETR-TensorRT","repo_kind":"listed","path":"codetr/transformer.py","file_url":"https://github.com/anenbergb/Co-DETR-TensorRT/blob/HEAD/codetr/transformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2f3e30980c10ac1c"}},{"code_sha256_prefix":"5502568acd4c6cdf","entry":"multi_scale_deformable_attention_pytorch","repo":"anenbergb/Co-DETR-TensorRT","repo_kind":"listed","path":"codetr/ops.py","file_url":"https://github.com/anenbergb/Co-DETR-TensorRT/blob/HEAD/codetr/ops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5502568acd4c6cdf"}},{"code_sha256_prefix":"85e307a3efdf4aa0","entry":"swin_converter","repo":"anenbergb/Co-DETR-TensorRT","repo_kind":"listed","path":"codetr/swin.py","file_url":"https://github.com/anenbergb/Co-DETR-TensorRT/blob/HEAD/codetr/swin.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"85e307a3efdf4aa0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}