{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/conditional-detr-for-fast-training","title":"Conditional DETR for Fast Training Convergence","arxiv_id":"2108.06152","date":"2021-08-13","proceeding":"ICCV 2021 10","authors":["Depu Meng","Xiaokang Chen","Zejia Fan","Gang Zeng","Houqiang Li","Yuhui Yuan","Lei Sun","Jingdong Wang"],"abstract":"The recently-developed DETR approach applies the transformer encoder and decoder architecture to object detection and achieves promising performance. In this paper, we handle the critical issue, slow training convergence, and present a conditional cross-attention mechanism for fast DETR training. Our approach is motivated by that the cross-attention in DETR relies highly on the content embeddings for localizing the four extremities and predicting the box, which increases the need for high-quality content embeddings and thus the training difficulty. Our approach, named conditional DETR, learns a conditional spatial query from the decoder embedding for decoder multi-head cross-attention. The benefit is that through the conditional spatial query, each cross-attention head is able to attend to a band containing a distinct region, e.g., one object extremity or a region inside the object box. This narrows down the spatial range for localizing the distinct regions for object classification and box regression, thus relaxing the dependence on the content embeddings and easing the training. Empirical results show that conditional DETR converges 6.7x faster for the backbones R50 and R101 and 10x faster for stronger backbones DC5-R50 and DC5-R101. Code is available at https://github.com/Atten4Vis/ConditionalDETR.","url_abs":"https://arxiv.org/abs/2108.06152v3","url_pdf":"https://arxiv.org/pdf/2108.06152v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"conditional-detr-for-fast-training","repo_url":"https://github.com/atten4vis/conditionaldetr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"conditional-detr-for-fast-training","repo_url":"https://github.com/IDEA-Research/detrex","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"conditional-detr-for-fast-training","repo_url":"https://github.com/huggingface/transformers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"conditional-detr-for-fast-training","repo_url":"https://github.com/yangyucheng000/University/tree/main/model-2/conditional_detr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"detr","method_name":"Detr"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"feedforward-network","method_name":"Feedforward Network"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/object-detection-on-coco-minival","task":"Object Detection","dataset":"COCO minival","model":"Conditional DETR-DC5-R101","rank_in_archive_order":110,"of":220,"metrics":{"AP50":"66.8","AP75":"49.5","APL":"63.3","APM":"50.3","APS":"27.2","Params (M)":"63","box AP":"45.9"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-minival","task":"Object Detection","dataset":"COCO minival","model":"Conditional DETR-DC5-R50","rank_in_archive_order":116,"of":220,"metrics":{"AP50":"65.4","AP75":"48.5","APL":"62.2","APM":"49","APS":"25.3","Params (M)":"44","box AP":"45.1"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-minival","task":"Object Detection","dataset":"COCO minival","model":"Conditional DETR-R101","rank_in_archive_order":127,"of":220,"metrics":{"AP50":"65.6","AP75":"47.5","APL":"63.6","APM":"48.4","APS":"23.6","Params (M)":"63","box AP":"44.5"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-minival","task":"Object Detection","dataset":"COCO minival","model":"Conditional DETR-R50","rank_in_archive_order":147,"of":220,"metrics":{"AP50":"64","AP75":"45.7","APL":"61.5","APM":"46.7","APS":"22.7","Params (M)":"44","box AP":"43"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2108.06152","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.06152"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huggingface/transformers","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IDEA-Research/detrex","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yangyucheng000/University/tree/main/model-2/conditional_detr","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/atten4vis/conditionaldetr","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/Atten4Vis/ConditionalDETR","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1,"ran_honours":1,"ran_violates":1,"ran_draft_wrong":2,"unverified":4},"by_repo_kind":{"official":{"samples":8,"ran":4,"repositories":1},"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"47b02db49bdfd76e","entry":"ConditionalCrossAttention","repo":"IDEA-Research/detrex","repo_kind":"listed","path":"detrex/layers/attention.py","file_url":"https://github.com/IDEA-Research/detrex/blob/HEAD/detrex/layers/attention.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"47b02db49bdfd76e"}},{"code_sha256_prefix":"f31a58bf6457ced5","entry":"convert_to_xywh","repo":"atten4vis/conditionaldetr","repo_kind":"official","path":"datasets/coco_eval.py","file_url":"https://github.com/atten4vis/conditionaldetr/blob/HEAD/datasets/coco_eval.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f31a58bf6457ced5"}},{"code_sha256_prefix":"ac8fe530cdad4d8c","entry":"dice_loss","repo":"atten4vis/conditionaldetr","repo_kind":"official","path":"models/segmentation.py","file_url":"https://github.com/atten4vis/conditionaldetr/blob/HEAD/models/segmentation.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ac8fe530cdad4d8c"}},{"code_sha256_prefix":"fe0ddcc2d420c9a0","entry":"evaluate","repo":"atten4vis/conditionaldetr","repo_kind":"official","path":"datasets/coco_eval.py","file_url":"https://github.com/atten4vis/conditionaldetr/blob/HEAD/datasets/coco_eval.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe0ddcc2d420c9a0"}},{"code_sha256_prefix":"5c0711aada67957e","entry":"sigmoid_focal_loss","repo":"atten4vis/conditionaldetr","repo_kind":"official","path":"models/segmentation.py","file_url":"https://github.com/atten4vis/conditionaldetr/blob/HEAD/models/segmentation.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5c0711aada67957e"}},{"code_sha256_prefix":"630f5aa14fbee6c6","entry":"ConditionalDETR","repo":"Atten4Vis/ConditionalDETR","repo_kind":"official","path":"models/conditional_detr.py","file_url":"https://github.com/Atten4Vis/ConditionalDETR/blob/HEAD/models/conditional_detr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"630f5aa14fbee6c6"}},{"code_sha256_prefix":"bf149877606cbfba","entry":"build_transformer","repo":"atten4vis/conditionaldetr","repo_kind":"official","path":"models/transformer.py","file_url":"https://github.com/atten4vis/conditionaldetr/blob/HEAD/models/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bf149877606cbfba"}},{"code_sha256_prefix":"d573d92393e20123","entry":"gen_sineembed_for_position","repo":"atten4vis/conditionaldetr","repo_kind":"official","path":"models/transformer.py","file_url":"https://github.com/atten4vis/conditionaldetr/blob/HEAD/models/transformer.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d573d92393e20123"}},{"code_sha256_prefix":"f09175b0f39f0ad0","entry":"multi_head_attention_forward","repo":"atten4vis/conditionaldetr","repo_kind":"official","path":"models/attention.py","file_url":"https://github.com/atten4vis/conditionaldetr/blob/HEAD/models/attention.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f09175b0f39f0ad0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}