{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/find-n-propagate-open-vocabulary-3d-object","title":"Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environments","arxiv_id":"2403.13556","date":"2024-03-20","proceeding":null,"authors":["Djamahl Etchegaray","Zi Huang","Tatsuya Harada","Yadan Luo"],"abstract":"In this work, we tackle the limitations of current LiDAR-based 3D object detection systems, which are hindered by a restricted class vocabulary and the high costs associated with annotating new object classes. Our exploration of open-vocabulary (OV) learning in urban environments aims to capture novel instances using pre-trained vision-language models (VLMs) with multi-sensor data. We design and benchmark a set of four potential solutions as baselines, categorizing them into either top-down or bottom-up approaches based on their input data strategies. While effective, these methods exhibit certain limitations, such as missing novel objects in 3D box estimation or applying rigorous priors, leading to biases towards objects near the camera or of rectangular geometries. To overcome these limitations, we introduce a universal \\textsc{Find n' Propagate} approach for 3D OV tasks, aimed at maximizing the recall of novel objects and propagating this detection capability to more distant areas thereby progressively capturing more. In particular, we utilize a greedy box seeker to search against 3D novel boxes of varying orientations and depth in each generated frustum and ensure the reliability of newly identified boxes by cross alignment and density ranker. Additionally, the inherent bias towards camera-proximal objects is alleviated by the proposed remote simulator, which randomly diversifies pseudo-labeled novel instances in the self-training process, combined with the fusion of base samples in the memory bank. Extensive experiments demonstrate a 53% improvement in novel recall across diverse OV settings, VLMs, and 3D detectors. Notably, we achieve up to a 3.97-fold increase in Average Precision (AP) for novel object classes. The source code is made available at https://github.com/djamahl99/findnpropagate.","url_abs":"https://arxiv.org/abs/2403.13556v2","url_pdf":"https://arxiv.org/pdf/2403.13556v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"find-n-propagate-open-vocabulary-3d-object","repo_url":"https://github.com/djamahl99/findnpropagate","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"3d-object-detection","task_name":"3D Object Detection"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"base","method_name":"BASE"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.13556","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.13556"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/djamahl99/findnpropagate","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"unverified":7},"by_repo_kind":{"official":{"samples":11,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"add872aaafe8385f","entry":"angle2class","repo":"djamahl99/findnpropagate","repo_kind":"official","path":"pcdet/utils/frustum_model_util.py","file_url":"https://github.com/djamahl99/findnpropagate/blob/HEAD/pcdet/utils/frustum_model_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"add872aaafe8385f"}},{"code_sha256_prefix":"f9a9ad78e3489868","entry":"class2angle","repo":"djamahl99/findnpropagate","repo_kind":"official","path":"pcdet/utils/frustum_model_util.py","file_url":"https://github.com/djamahl99/findnpropagate/blob/HEAD/pcdet/utils/frustum_model_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f9a9ad78e3489868"}},{"code_sha256_prefix":"ebb535e31d8e58de","entry":"parse_output_to_tensors","repo":"djamahl99/findnpropagate","repo_kind":"official","path":"pcdet/utils/frustum_model_util.py","file_url":"https://github.com/djamahl99/findnpropagate/blob/HEAD/pcdet/utils/frustum_model_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ebb535e31d8e58de"}},{"code_sha256_prefix":"0d095007c24b14c0","entry":"post_act_block_dense","repo":"djamahl99/findnpropagate","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone_2d.py","file_url":"https://github.com/djamahl99/findnpropagate/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone_2d.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0d095007c24b14c0"}},{"code_sha256_prefix":"44db2351bcc0bffe","entry":"cfg_from_yaml_file","repo":"djamahl99/findnpropagate","repo_kind":"official","path":"pcdet/config.py","file_url":"https://github.com/djamahl99/findnpropagate/blob/HEAD/pcdet/config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"44db2351bcc0bffe"}},{"code_sha256_prefix":"65fe32ede00e7dcb","entry":"compute_fg_mask","repo":"djamahl99/findnpropagate","repo_kind":"official","path":"pcdet/utils/loss_utils.py","file_url":"https://github.com/djamahl99/findnpropagate/blob/HEAD/pcdet/utils/loss_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"65fe32ede00e7dcb"}},{"code_sha256_prefix":"1780d388cc532a6d","entry":"get_corner_loss_lidar","repo":"djamahl99/findnpropagate","repo_kind":"official","path":"pcdet/utils/loss_utils.py","file_url":"https://github.com/djamahl99/findnpropagate/blob/HEAD/pcdet/utils/loss_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1780d388cc532a6d"}},{"code_sha256_prefix":"50e8e8cdfc5129f0","entry":"merge_new_config","repo":"djamahl99/findnpropagate","repo_kind":"official","path":"pcdet/config.py","file_url":"https://github.com/djamahl99/findnpropagate/blob/HEAD/pcdet/config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50e8e8cdfc5129f0"}},{"code_sha256_prefix":"488b91d67a807558","entry":"neg_loss_cornernet","repo":"djamahl99/findnpropagate","repo_kind":"official","path":"pcdet/utils/loss_utils.py","file_url":"https://github.com/djamahl99/findnpropagate/blob/HEAD/pcdet/utils/loss_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"488b91d67a807558"}},{"code_sha256_prefix":"2b4e0558df870bf9","entry":"post_act_block","repo":"djamahl99/findnpropagate","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone.py","file_url":"https://github.com/djamahl99/findnpropagate/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2b4e0558df870bf9"}},{"code_sha256_prefix":"016a882194742fe9","entry":"post_act_block","repo":"djamahl99/findnpropagate","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone_2d.py","file_url":"https://github.com/djamahl99/findnpropagate/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone_2d.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"016a882194742fe9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}