{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ep2p-loc-end-to-end-3d-point-to-2d-pixel","title":"EP2P-Loc: End-to-End 3D Point to 2D Pixel Localization for Large-Scale Visual Localization","arxiv_id":"2309.07471","date":"2023-09-14","proceeding":"ICCV 2023 1","authors":["Minjung Kim","Junseo Koo","Gunhee Kim"],"abstract":"Visual localization is the task of estimating a 6-DoF camera pose of a query image within a provided 3D reference map. Thanks to recent advances in various 3D sensors, 3D point clouds are becoming a more accurate and affordable option for building the reference map, but research to match the points of 3D point clouds with pixels in 2D images for visual localization remains challenging. Existing approaches that jointly learn 2D-3D feature matching suffer from low inliers due to representational differences between the two modalities, and the methods that bypass this problem into classification have an issue of poor refinement. In this work, we propose EP2P-Loc, a novel large-scale visual localization method that mitigates such appearance discrepancy and enables end-to-end training for pose estimation. To increase the number of inliers, we propose a simple algorithm to remove invisible 3D points in the image, and find all 2D-3D correspondences without keypoint detection. To reduce memory usage and search complexity, we take a coarse-to-fine approach where we extract patch-level features from 2D images, then perform 2D patch classification on each 3D point, and obtain the exact corresponding 2D pixel coordinates through positional encoding. Finally, for the first time in this task, we employ a differentiable PnP for end-to-end training. In the experiments on newly curated large-scale indoor and outdoor benchmarks based on 2D-3D-S and KITTI, we show that our method achieves the state-of-the-art performance compared to existing visual localization and image-to-point cloud registration methods.","url_abs":"https://arxiv.org/abs/2309.07471v1","url_pdf":"https://arxiv.org/pdf/2309.07471v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ep2p-loc-end-to-end-3d-point-to-2d-pixel","repo_url":"https://github.com/minnjung/ep2p-loc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"3d-feature-matching","task_name":"3D Feature Matching"},{"task_slug":"image-to-point-cloud-registration","task_name":"Image to Point Cloud Registration"},{"task_slug":"keypoint-detection","task_name":"Keypoint Detection"},{"task_slug":"point-cloud-registration","task_name":"Point Cloud Registration"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"},{"task_slug":"visual-localization","task_name":"Visual Localization"}],"methods":[{"method_slug":"pnp","method_name":"PnP"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2309.07471","atlas_url":"https://app.syntology.ai/?focus=2309.07471","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.07471"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/minnjung/ep2p-loc","reach":{"status":"ok"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"bbed012635fcdde5","entry":"collate_fn","repo":"minnjung/ep2p-loc","repo_kind":"official","path":"datasets/dataloader.py","file_url":"https://github.com/minnjung/ep2p-loc/blob/HEAD/datasets/dataloader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bbed012635fcdde5"}},{"code_sha256_prefix":"271d3a4ef1c43133","entry":"get_dataloader","repo":"minnjung/ep2p-loc","repo_kind":"official","path":"datasets/dataloader.py","file_url":"https://github.com/minnjung/ep2p-loc/blob/HEAD/datasets/dataloader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"271d3a4ef1c43133"}},{"code_sha256_prefix":"4485ad1f79199d08","entry":"rgba_to_rgb","repo":"minnjung/ep2p-loc","repo_kind":"official","path":"datasets/dataloader.py","file_url":"https://github.com/minnjung/ep2p-loc/blob/HEAD/datasets/dataloader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4485ad1f79199d08"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}