{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dipex-dispersing-prompt-expansion-for-class","title":"DiPEx: Dispersing Prompt Expansion for Class-Agnostic Object Detection","arxiv_id":"2406.14924","date":"2024-06-21","proceeding":null,"authors":["Jia Syuen Lim","Zhuoxiao Chen","Mahsa Baktashmotlagh","Zhi Chen","Xin Yu","Zi Huang","Yadan Luo"],"abstract":"Class-agnostic object detection (OD) can be a cornerstone or a bottleneck for many downstream vision tasks. Despite considerable advancements in bottom-up and multi-object discovery methods that leverage basic visual cues to identify salient objects, consistently achieving a high recall rate remains difficult due to the diversity of object types and their contextual complexity. In this work, we investigate using vision-language models (VLMs) to enhance object detection via a self-supervised prompt learning strategy. Our initial findings indicate that manually crafted text queries often result in undetected objects, primarily because detection confidence diminishes when the query words exhibit semantic overlap. To address this, we propose a Dispersing Prompt Expansion (DiPEx) approach. DiPEx progressively learns to expand a set of distinct, non-overlapping hyperspherical prompts to enhance recall rates, thereby improving performance in downstream tasks such as out-of-distribution OD. Specifically, DiPEx initiates the process by self-training generic parent prompts and selecting the one with the highest semantic uncertainty for further expansion. The resulting child prompts are expected to inherit semantics from their parent prompts while capturing more fine-grained semantics. We apply dispersion losses to ensure high inter-class discrepancy among child prompts while preserving semantic consistency between parent-child prompt pairs. To prevent excessive growth of the prompt sets, we utilize the maximum angular coverage (MAC) of the semantic space as a criterion for early termination. We demonstrate the effectiveness of DiPEx through extensive class-agnostic OD and OOD-OD experiments on MS-COCO and LVIS, surpassing other prompting methods by up to 20.1% in AR and achieving a 21.3% AP improvement over SAM. The code is available at https://github.com/jason-lim26/DiPEx.","url_abs":"https://arxiv.org/abs/2406.14924v1","url_pdf":"https://arxiv.org/pdf/2406.14924v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dipex-dispersing-prompt-expansion-for-class","repo_url":"https://github.com/jason-lim26/dipex","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"class-agnostic-object-detection","task_name":"Class-agnostic Object Detection"},{"task_slug":"multi-object-discovery","task_name":"Multi-object discovery"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-discovery","task_name":"Object Discovery"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"sam","method_name":"SAM"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.14924","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.14924"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/jason-lim26/DiPEx","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jason-lim26/dipex","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":3,"ran_honours":1,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fa56de74aac836b3","entry":"PromptTree","repo":"jason-lim26/dipex","repo_kind":"official","path":"Open-GroundingDino/models/GroundingDINO/dipex.py","file_url":"https://github.com/jason-lim26/dipex/blob/HEAD/Open-GroundingDino/models/GroundingDINO/dipex.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fa56de74aac836b3"}},{"code_sha256_prefix":"f31a58bf6457ced5","entry":"convert_to_xywh","repo":"jason-lim26/DiPEx","repo_kind":"official","path":"Open-GroundingDino/datasets/coco_eval.py","file_url":"https://github.com/jason-lim26/DiPEx/blob/HEAD/Open-GroundingDino/datasets/coco_eval.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f31a58bf6457ced5"}},{"code_sha256_prefix":"934833a6fa34d3a0","entry":"create_positive_map","repo":"jason-lim26/DiPEx","repo_kind":"official","path":"Open-GroundingDino/models/GroundingDINO/dipex.py","file_url":"https://github.com/jason-lim26/DiPEx/blob/HEAD/Open-GroundingDino/models/GroundingDINO/dipex.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"934833a6fa34d3a0"}},{"code_sha256_prefix":"fe0ddcc2d420c9a0","entry":"evaluate","repo":"jason-lim26/DiPEx","repo_kind":"official","path":"Open-GroundingDino/datasets/coco_eval.py","file_url":"https://github.com/jason-lim26/DiPEx/blob/HEAD/Open-GroundingDino/datasets/coco_eval.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe0ddcc2d420c9a0"}},{"code_sha256_prefix":"fb05961e12a99588","entry":"label2onehot","repo":"jason-lim26/DiPEx","repo_kind":"official","path":"Open-GroundingDino/datasets/coco.py","file_url":"https://github.com/jason-lim26/DiPEx/blob/HEAD/Open-GroundingDino/datasets/coco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fb05961e12a99588"}},{"code_sha256_prefix":"4373b99bf6b5fd7d","entry":"ciou","repo":"jason-lim26/DiPEx","repo_kind":"official","path":"Open-GroundingDino/util/box_loss.py","file_url":"https://github.com/jason-lim26/DiPEx/blob/HEAD/Open-GroundingDino/util/box_loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4373b99bf6b5fd7d"}},{"code_sha256_prefix":"7818a5040676702a","entry":"diou","repo":"jason-lim26/DiPEx","repo_kind":"official","path":"Open-GroundingDino/util/box_loss.py","file_url":"https://github.com/jason-lim26/DiPEx/blob/HEAD/Open-GroundingDino/util/box_loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7818a5040676702a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}