{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/grounding-dino-marrying-dino-with-grounded","title":"Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection","arxiv_id":"2303.05499","date":"2023-03-09","proceeding":null,"authors":["Shilong Liu","Zhaoyang Zeng","Tianhe Ren","Feng Li","Hao Zhang","Jie Yang","Qing Jiang","Chunyuan Li","Jianwei Yang","Hang Su","Jun Zhu","Lei Zhang"],"abstract":"In this paper, we present an open-set object detector, called Grounding DINO, by marrying Transformer-based detector DINO with grounded pre-training, which can detect arbitrary objects with human inputs such as category names or referring expressions. The key solution of open-set object detection is introducing language to a closed-set detector for open-set concept generalization. To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion. While previous works mainly evaluate open-set object detection on novel categories, we propose to also perform evaluations on referring expression comprehension for objects specified with attributes. Grounding DINO performs remarkably well on all three settings, including benchmarks on COCO, LVIS, ODinW, and RefCOCO/+/g. Grounding DINO achieves a $52.5$ AP on the COCO detection zero-shot transfer benchmark, i.e., without any training data from COCO. It sets a new record on the ODinW zero-shot benchmark with a mean $26.1$ AP. Code will be available at \\url{https://github.com/IDEA-Research/GroundingDINO}.","url_abs":"https://arxiv.org/abs/2303.05499v5","url_pdf":"https://arxiv.org/pdf/2303.05499v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"grounding-dino-marrying-dino-with-grounded","repo_url":"https://github.com/idea-research/groundingdino","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"grounding-dino-marrying-dino-with-grounded","repo_url":"https://github.com/IDEA-Research/Grounded-Segment-Anything","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"grounding-dino-marrying-dino-with-grounded","repo_url":"https://github.com/anthonyhuo/sam-dam-for-compositional-reasoning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"grounding-dino-marrying-dino-with-grounded","repo_url":"https://github.com/camenduru/grounded-segment-anything-colab","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Unlicense"}},{"paper_slug":"grounding-dino-marrying-dino-with-grounded","repo_url":"https://github.com/huggingface/transformers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"grounding-dino-marrying-dino-with-grounded","repo_url":"https://github.com/hzlbbfrog/generative-bim","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"grounding-dino-marrying-dino-with-grounded","repo_url":"https://github.com/idea-research/dino-x-api","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"grounding-dino-marrying-dino-with-grounded","repo_url":"https://github.com/idea-research/grounded-sam-2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"grounding-dino-marrying-dino-with-grounded","repo_url":"https://github.com/longzw1997/Open-GroundingDino","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"grounding-dino-marrying-dino-with-grounded","repo_url":"https://github.com/PaddlePaddle/PaddleMIX","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"paddle","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"referring-expression","task_name":"Referring Expression"},{"task_slug":"referring-expression-comprehension","task_name":"Referring Expression Comprehension"},{"task_slug":"zero-shot-segmentation","task_name":"Zero Shot Segmentation"},{"task_slug":"zero-shot-object-detection","task_name":"Zero-Shot Object Detection"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/object-detection-on-coco-minival","task":"Object Detection","dataset":"COCO minival","model":"Grounding DINO","rank_in_archive_order":14,"of":220,"metrics":{"box AP":"63.0"},"uses_additional_data":true},{"leaderboard":"/sota/object-detection-on-coco","task":"Object Detection","dataset":"COCO test-dev","model":"Grounding DINO","rank_in_archive_order":19,"of":225,"metrics":{"box mAP":"63.0"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-odinw-full-shot-13-tasks","task":"Object Detection","dataset":"ODinW Full-Shot 13 Tasks","model":"Grounding DINO","rank_in_archive_order":5,"of":8,"metrics":{"AP":"70.9"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-segmentation-on-segmentation-in-the","task":"Zero Shot Segmentation","dataset":"Segmentation in the Wild","model":"Grounded-SAM","rank_in_archive_order":2,"of":12,"metrics":{"Mean AP":"46.0"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-object-detection-on-lvis-v1-0","task":"Zero-Shot Object Detection","dataset":"LVIS v1.0 minival","model":"GroundingDINO-L","rank_in_archive_order":9,"of":11,"metrics":{"AP":"33.9"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-object-detection-on-mscoco","task":"Zero-Shot Object Detection","dataset":"MSCOCO","model":"Grounding DINO-L (without COCO data)","rank_in_archive_order":4,"of":7,"metrics":{"AP":"52.5"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-object-detection-on-odinw","task":"Zero-Shot Object Detection","dataset":"ODinW","model":"Grounding DINO","rank_in_archive_order":3,"of":5,"metrics":{"Average Score":"26.1"},"uses_additional_data":true}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2303.05499","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.05499"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huggingface/transformers","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/camenduru/grounded-segment-anything-colab","reach":{"status":"ok","spdx":"Unlicense"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/idea-research/groundingdino","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/longzw1997/Open-GroundingDino","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IDEA-Research/Grounded-Segment-Anything","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/idea-research/grounded-sam-2","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/anthonyhuo/sam-dam-for-compositional-reasoning","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/PaddlePaddle/PaddleMIX","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/idea-research/dino-x-api","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hzlbbfrog/generative-bim","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/IDEA-Research/GroundingDINO","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/Stability-AI/stablediffusion","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}}],"summary":{"ran":2,"unverified":3},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1},"listed":{"samples":3,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d5c391b32de453d0","entry":"ContrastiveEmbed","repo":"IDEA-Research/GroundingDINO","repo_kind":"official","path":"groundingdino/models/GroundingDINO/groundingdino.py","file_url":"https://github.com/IDEA-Research/GroundingDINO/blob/HEAD/groundingdino/models/GroundingDINO/groundingdino.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d5c391b32de453d0"}},{"code_sha256_prefix":"d76d1655fad01913","entry":"ContrastiveEmbed","repo":"IDEA-Research/Grounded-Segment-Anything","repo_kind":"listed","path":"GroundingDINO/groundingdino/models/GroundingDINO/groundingdino.py","file_url":"https://github.com/IDEA-Research/Grounded-Segment-Anything/blob/HEAD/GroundingDINO/groundingdino/models/GroundingDINO/groundingdino.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d76d1655fad01913"}},{"code_sha256_prefix":"f4c8ecd8274788c7","entry":"GroundingDINO","repo":"IDEA-Research/GroundingDINO","repo_kind":"official","path":"groundingdino/models/GroundingDINO/groundingdino.py","file_url":"https://github.com/IDEA-Research/GroundingDINO/blob/HEAD/groundingdino/models/GroundingDINO/groundingdino.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f4c8ecd8274788c7"}},{"code_sha256_prefix":"c37637f1c959fa14","entry":"GroundingDINO","repo":"IDEA-Research/Grounded-Segment-Anything","repo_kind":"listed","path":"GroundingDINO/groundingdino/models/GroundingDINO/groundingdino.py","file_url":"https://github.com/IDEA-Research/Grounded-Segment-Anything/blob/HEAD/GroundingDINO/groundingdino/models/GroundingDINO/groundingdino.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c37637f1c959fa14"}},{"code_sha256_prefix":"5ef1be1466f5a09a","entry":"GroundingDINO","repo":"longzw1997/Open-GroundingDino","repo_kind":"listed","path":"models/GroundingDINO/groundingdino.py","file_url":"https://github.com/longzw1997/Open-GroundingDino/blob/HEAD/models/GroundingDINO/groundingdino.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ef1be1466f5a09a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}