{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/enhancing-novel-object-detection-via","title":"Enhancing Novel Object Detection via Cooperative Foundational Models","arxiv_id":"2311.12068","date":"2023-11-19","proceeding":null,"authors":["Rohit Bharadwaj","Muzammal Naseer","Salman Khan","Fahad Shahbaz Khan"],"abstract":"In this work, we address the challenging and emergent problem of novel object detection (NOD), focusing on the accurate detection of both known and novel object categories during inference. Traditional object detection algorithms are inherently closed-set, limiting their capability to handle NOD. We present a novel approach to transform existing closed-set detectors into open-set detectors. This transformation is achieved by leveraging the complementary strengths of pre-trained foundational models, specifically CLIP and SAM, through our cooperative mechanism. Furthermore, by integrating this mechanism with state-of-the-art open-set detectors such as GDINO, we establish new benchmarks in object detection performance. Our method achieves 17.42 mAP in novel object detection and 42.08 mAP for known objects on the challenging LVIS dataset. Adapting our approach to the COCO OVD split, we surpass the current state-of-the-art by a margin of 7.2 $ \\text{AP}_{50} $ for novel classes. Our code is available at https://rohit901.github.io/coop-foundation-models/ .","url_abs":"https://arxiv.org/abs/2311.12068v4","url_pdf":"https://arxiv.org/pdf/2311.12068v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"enhancing-novel-object-detection-via","repo_url":"https://github.com/rohit901/cooperative-foundational-models","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"novel-class-discovery","task_name":"Novel Class Discovery"},{"task_slug":"novel-object-detection","task_name":"Novel Object Detection"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"open-vocabulary-object-detection","task_name":"Open Vocabulary Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"sam","method_name":"SAM"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/novel-object-detection-on-lvis-v1-0-val","task":"Novel Object Detection","dataset":"LVIS v1.0 val","model":"Cooperative Foundational Models","rank_in_archive_order":1,"of":5,"metrics":{"All mAP":"19.33","Known mAP":"42.08","Novel mAP":"17.42"},"uses_additional_data":false},{"leaderboard":"/sota/open-vocabulary-object-detection-on-mscoco","task":"Open Vocabulary Object Detection","dataset":"MSCOCO","model":"Cooperative Foundational Models","rank_in_archive_order":1,"of":32,"metrics":{"AP 0.5":"50.3"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.12068","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}