{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/panopticndt-efficient-and-robust-panoptic","title":"PanopticNDT: Efficient and Robust Panoptic Mapping","arxiv_id":"2309.13635","date":"2023-09-24","proceeding":null,"authors":["Daniel Seichter","Benedict Stephan","Söhnke Benedikt Fischedick","Steffen Müller","Leonard Rabes","Horst-Michael Gross"],"abstract":"As the application scenarios of mobile robots are getting more complex and challenging, scene understanding becomes increasingly crucial. A mobile robot that is supposed to operate autonomously in indoor environments must have precise knowledge about what objects are present, where they are, what their spatial extent is, and how they can be reached; i.e., information about free space is also crucial. Panoptic mapping is a powerful instrument providing such information. However, building 3D panoptic maps with high spatial resolution is challenging on mobile robots, given their limited computing capabilities. In this paper, we propose PanopticNDT - an efficient and robust panoptic mapping approach based on occupancy normal distribution transform (NDT) mapping. We evaluate our approach on the publicly available datasets Hypersim and ScanNetV2. The results reveal that our approach can represent panoptic information at a higher level of detail than other state-of-the-art approaches while enabling real-time panoptic mapping on mobile robots. Finally, we prove the real-world applicability of PanopticNDT with qualitative results in a domestic application.","url_abs":"https://arxiv.org/abs/2309.13635v2","url_pdf":"https://arxiv.org/pdf/2309.13635v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"panopticndt-efficient-and-robust-panoptic","repo_url":"https://github.com/tui-nicr/panoptic-mapping","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"panopticndt-efficient-and-robust-panoptic","repo_url":"https://github.com/tui-nicr/emsaformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"panopticndt-efficient-and-robust-panoptic","repo_url":"https://github.com/tui-nicr/emsanet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"panopticndt-efficient-and-robust-panoptic","repo_url":"https://github.com/tui-nicr/nicr-scene-analysis-datasets","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"2d-panoptic-segmentation","task_name":"2D Panoptic Segmentation"},{"task_slug":"3d-panoptic-segmentation","task_name":"3D Panoptic Segmentation"},{"task_slug":"3d-semantic-segmentation","task_name":"3D Semantic Segmentation"},{"task_slug":"panoptic-segmentation","task_name":"Panoptic Segmentation"},{"task_slug":"panoptic-segmentation-panopticndt-instances","task_name":"Panoptic Segmentation (PanopticNDT instances)"},{"task_slug":"scene-classification-unified-classes","task_name":"Scene Classification (unified classes)"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/2d-panoptic-segmentation-on-scannetv2","task":"2D Panoptic Segmentation","dataset":"ScanNetV2","model":"EMSANet (2x ResNet-34 NBt1D, PanopticNDT version)","rank_in_archive_order":1,"of":1,"metrics":{"PQ":"58.22"},"uses_additional_data":false},{"leaderboard":"/sota/3d-semantic-segmentation-on-hypersim","task":"3D Semantic Segmentation","dataset":"Hypersim","model":"PanopticNDT (10cm)","rank_in_archive_order":1,"of":2,"metrics":{"mIoU":"45.43","mIoU (test)":"45.34"},"uses_additional_data":false},{"leaderboard":"/sota/3d-semantic-segmentation-on-hypersim","task":"3D Semantic Segmentation","dataset":"Hypersim","model":"SemanticNDT (10cm)","rank_in_archive_order":2,"of":2,"metrics":{"mIoU":"44.31","mIoU (test)":"44.8"},"uses_additional_data":false},{"leaderboard":"/sota/panoptic-segmentation-on-hypersim","task":"Panoptic Segmentation","dataset":"Hypersim","model":"EMSANet (2x ResNet-34 NBt1D)","rank_in_archive_order":1,"of":1,"metrics":{"PQ":"34.95","PQ (test)":"29.77","mIoU":"49.12","mIoU (test)":"44.66"},"uses_additional_data":false},{"leaderboard":"/sota/panoptic-segmentation-on-nyu-depth-v2","task":"Panoptic Segmentation","dataset":"NYU Depth v2","model":"EMSANet (2x ResNet-34 NBt1D, PanopticNDT version, finetuned)","rank_in_archive_order":1,"of":2,"metrics":{"PQ":"51.15"},"uses_additional_data":false},{"leaderboard":"/sota/panoptic-segmentation-on-scannetv2","task":"Panoptic Segmentation","dataset":"ScanNetV2","model":"PanopticNDT (10cm)","rank_in_archive_order":2,"of":5,"metrics":{"PQ":"59.19"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-hypersim","task":"Semantic Segmentation","dataset":"Hypersim","model":"EMSANet (2x ResNet-34 NBt1D)","rank_in_archive_order":1,"of":5,"metrics":{"mIoU":"49.74","mIoU (test)":"46.66"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-nyu-depth-v2","task":"Semantic Segmentation","dataset":"NYU Depth v2","model":"EMSANet (2x ResNet-34 NBt1D, PanopticNDT version, finetuned)","rank_in_archive_order":9,"of":121,"metrics":{"Mean IoU":"59.02"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-sun-rgbd","task":"Semantic Segmentation","dataset":"SUN-RGBD","model":"EMSANet (2x ResNet-34 NBt1D, PanopticNDT version, finetuned)","rank_in_archive_order":16,"of":44,"metrics":{"Mean IoU":"50.86%"},"uses_additional_data":true},{"leaderboard":"/sota/semantic-segmentation-on-scannet","task":"Semantic Segmentation","dataset":"ScanNet","model":"PanopticNDT (10cm)","rank_in_archive_order":30,"of":45,"metrics":{"test mIoU":"68.1","val mIoU":"68.39"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-scannetv2","task":"Semantic Segmentation","dataset":"ScanNetV2","model":"EMSANet (2x ResNet-34 NBt1D, PanopticNDT version)","rank_in_archive_order":2,"of":12,"metrics":{"Mean IoU":"60.0%","Mean IoU (test)":"60.0%","Mean IoU (val)":"70.99%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.13635","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.13635"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tui-nicr/emsaformer","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tui-nicr/emsanet","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tui-nicr/panoptic-mapping","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tui-nicr/nicr-scene-analysis-datasets","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":12},"by_repo_kind":{"listed":{"samples":12,"ran":0,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d1a05fa41fa6025b","entry":"alloc_buf","repo":"tui-nicr/emsanet","repo_kind":"listed","path":"inference_time_whole_model.py","file_url":"https://github.com/tui-nicr/emsanet/blob/HEAD/inference_time_whole_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d1a05fa41fa6025b"}},{"code_sha256_prefix":"7fcd0b9de3375423","entry":"blend_images","repo":"tui-nicr/emsanet","repo_kind":"listed","path":"emsanet/visualization.py","file_url":"https://github.com/tui-nicr/emsanet/blob/HEAD/emsanet/visualization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7fcd0b9de3375423"}},{"code_sha256_prefix":"dfe2f4daf2bb2966","entry":"compute_class_weights","repo":"tui-nicr/nicr-scene-analysis-datasets","repo_kind":"listed","path":"src/nicr_scene_analysis_datasets/dataset_base/_class_weighting.py","file_url":"https://github.com/tui-nicr/nicr-scene-analysis-datasets/blob/HEAD/src/nicr_scene_analysis_datasets/dataset_base/_class_weighting.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dfe2f4daf2bb2966"}},{"code_sha256_prefix":"3b5ec9012aa4a2a5","entry":"depth_compute_stats","repo":"tui-nicr/nicr-scene-analysis-datasets","repo_kind":"listed","path":"src/nicr_scene_analysis_datasets/dataset_base/_depth_dataset.py","file_url":"https://github.com/tui-nicr/nicr-scene-analysis-datasets/blob/HEAD/src/nicr_scene_analysis_datasets/dataset_base/_depth_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3b5ec9012aa4a2a5"}},{"code_sha256_prefix":"2beb97e864ab24c6","entry":"get_engine","repo":"tui-nicr/emsanet","repo_kind":"listed","path":"inference_time_whole_model.py","file_url":"https://github.com/tui-nicr/emsanet/blob/HEAD/inference_time_whole_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2beb97e864ab24c6"}},{"code_sha256_prefix":"6187963a4f5a9aad","entry":"get_engine_v2","repo":"tui-nicr/emsanet","repo_kind":"listed","path":"inference_time_whole_model.py","file_url":"https://github.com/tui-nicr/emsanet/blob/HEAD/inference_time_whole_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6187963a4f5a9aad"}},{"code_sha256_prefix":"e60a52fe170f233d","entry":"get_lr_scheduler","repo":"tui-nicr/emsanet","repo_kind":"listed","path":"emsanet/lr_scheduler.py","file_url":"https://github.com/tui-nicr/emsanet/blob/HEAD/emsanet/lr_scheduler.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e60a52fe170f233d"}},{"code_sha256_prefix":"1a50127363dc2189","entry":"get_optimizer","repo":"tui-nicr/emsanet","repo_kind":"listed","path":"emsanet/optimizer.py","file_url":"https://github.com/tui-nicr/emsanet/blob/HEAD/emsanet/optimizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1a50127363dc2189"}},{"code_sha256_prefix":"2aa4f9de057628dd","entry":"normalize_embedding","repo":"tui-nicr/nicr-scene-analysis-datasets","repo_kind":"listed","path":"src/nicr_scene_analysis_datasets/auxiliary_data/_dataset.py","file_url":"https://github.com/tui-nicr/nicr-scene-analysis-datasets/blob/HEAD/src/nicr_scene_analysis_datasets/auxiliary_data/_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2aa4f9de057628dd"}},{"code_sha256_prefix":"324bfcd690d8ff8f","entry":"parse_datasets","repo":"tui-nicr/emsaformer","repo_kind":"listed","path":"emsaformer/data.py","file_url":"https://github.com/tui-nicr/emsaformer/blob/HEAD/emsaformer/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"324bfcd690d8ff8f"}},{"code_sha256_prefix":"2e3d9b0804410969","entry":"parse_datasets","repo":"tui-nicr/emsanet","repo_kind":"listed","path":"emsanet/data.py","file_url":"https://github.com/tui-nicr/emsanet/blob/HEAD/emsanet/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2e3d9b0804410969"}},{"code_sha256_prefix":"751415e727584b4d","entry":"time_inference_tensorrt_trtexec","repo":"tui-nicr/emsaformer","repo_kind":"listed","path":"inference_time_whole_model.py","file_url":"https://github.com/tui-nicr/emsaformer/blob/HEAD/inference_time_whole_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"751415e727584b4d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}