{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sparse-detr-efficient-end-to-end-object-1","title":"Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsity","arxiv_id":"2111.14330","date":"2021-11-29","proceeding":"ICLR 2022 4","authors":["Byungseok Roh","Jaewoong Shin","Wuhyun Shin","Saehoon Kim"],"abstract":"DETR is the first end-to-end object detector using a transformer encoder-decoder architecture and demonstrates competitive performance but low computational efficiency on high resolution feature maps. The subsequent work, Deformable DETR, enhances the efficiency of DETR by replacing dense attention with deformable attention, which achieves 10x faster convergence and improved performance. Deformable DETR uses the multiscale feature to ameliorate performance, however, the number of encoder tokens increases by 20x compared to DETR, and the computation cost of the encoder attention remains a bottleneck. In our preliminary experiment, we observe that the detection performance hardly deteriorates even if only a part of the encoder token is updated. Inspired by this observation, we propose Sparse DETR that selectively updates only the tokens expected to be referenced by the decoder, thus help the model effectively detect objects. In addition, we show that applying an auxiliary detection loss on the selected tokens in the encoder improves the performance while minimizing computational overhead. We validate that Sparse DETR achieves better performance than Deformable DETR even with only 10% encoder tokens on the COCO dataset. Albeit only the encoder tokens are sparsified, the total computation cost decreases by 38% and the frames per second (FPS) increases by 42% compared to Deformable DETR. Code is available at https://github.com/kakaobrain/sparse-detr","url_abs":"https://arxiv.org/abs/2111.14330v2","url_pdf":"https://arxiv.org/pdf/2111.14330v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sparse-detr-efficient-end-to-end-object-1","repo_url":"https://github.com/kakaobrain/sparse-detr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"deformable-attention-module","method_name":"Deformable Attention Module"},{"method_slug":"deformable-detr","method_name":"Deformable DETR"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"detr","method_name":"Detr"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"feedforward-network","method_name":"Feedforward Network"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2111.14330","atlas_url":"https://app.syntology.ai/?focus=2111.14330","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.14330"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/kakaobrain/sparse-detr","reach":null}],"summary":{"ran":3,"unverified":4},"by_repo_kind":{"official":{"samples":7,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"13b669de90269931","entry":"DeformableTransformerDecoder","repo":"kakaobrain/sparse-detr","repo_kind":"official","path":"models/deformable_transformer.py","file_url":"https://github.com/kakaobrain/sparse-detr/blob/HEAD/models/deformable_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"13b669de90269931"}},{"code_sha256_prefix":"648c97fae3ca9e11","entry":"DeformableTransformerEncoder","repo":"kakaobrain/sparse-detr","repo_kind":"official","path":"models/deformable_transformer.py","file_url":"https://github.com/kakaobrain/sparse-detr/blob/HEAD/models/deformable_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"648c97fae3ca9e11"}},{"code_sha256_prefix":"c181c1b23d599313","entry":"MaskPredictor","repo":"kakaobrain/sparse-detr","repo_kind":"official","path":"models/deformable_transformer.py","file_url":"https://github.com/kakaobrain/sparse-detr/blob/HEAD/models/deformable_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c181c1b23d599313"}},{"code_sha256_prefix":"87ecfe0a8fbc1ea2","entry":"DeformableTransformer","repo":"kakaobrain/sparse-detr","repo_kind":"official","path":"models/deformable_transformer.py","file_url":"https://github.com/kakaobrain/sparse-detr/blob/HEAD/models/deformable_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"87ecfe0a8fbc1ea2"}},{"code_sha256_prefix":"575115b86e118f85","entry":"DeformableTransformerDecoderLayer","repo":"kakaobrain/sparse-detr","repo_kind":"official","path":"models/deformable_transformer.py","file_url":"https://github.com/kakaobrain/sparse-detr/blob/HEAD/models/deformable_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"575115b86e118f85"}},{"code_sha256_prefix":"f8bfa71e9ead60c0","entry":"DeformableTransformerEncoderLayer","repo":"kakaobrain/sparse-detr","repo_kind":"official","path":"models/deformable_transformer.py","file_url":"https://github.com/kakaobrain/sparse-detr/blob/HEAD/models/deformable_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f8bfa71e9ead60c0"}},{"code_sha256_prefix":"196198adff3ea1cd","entry":"MSDeformAttn","repo":"kakaobrain/sparse-detr","repo_kind":"official","path":"models/deformable_transformer.py","file_url":"https://github.com/kakaobrain/sparse-detr/blob/HEAD/models/deformable_transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"196198adff3ea1cd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}