{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/e2pnet-event-to-point-cloud-registration-with-1","title":"E2PNet: Event to Point Cloud Registration with Spatio-Temporal Representation Learning","arxiv_id":"2311.18433","date":"2023-11-30","proceeding":"NeurIPS 2023 11","authors":["Xiuhong Lin","Changjie Qiu","Zhipeng Cai","Siqi Shen","Yu Zang","Weiquan Liu","Xuesheng Bian","Matthias Müller","Cheng Wang"],"abstract":"Event cameras have emerged as a promising vision sensor in recent years due to their unparalleled temporal resolution and dynamic range. While registration of 2D RGB images to 3D point clouds is a long-standing problem in computer vision, no prior work studies 2D-3D registration for event cameras. To this end, we propose E2PNet, the first learning-based method for event-to-point cloud registration. The core of E2PNet is a novel feature representation network called Event-Points-to-Tensor (EP2T), which encodes event data into a 2D grid-shaped feature tensor. This grid-shaped feature enables matured RGB-based frameworks to be easily used for event-to-point cloud registration, without changing hyper-parameters and the training procedure. EP2T treats the event input as spatio-temporal point clouds. Unlike standard 3D learning architectures that treat all dimensions of point clouds equally, the novel sampling and information aggregation modules in EP2T are designed to handle the inhomogeneity of the spatial and temporal dimensions. Experiments on the MVSEC and VECtor datasets demonstrate the superiority of E2PNet over hand-crafted and other learning-based methods. Compared to RGB-based registration, E2PNet is more robust to extreme illumination or fast motion due to the use of event data. Beyond 2D-3D registration, we also show the potential of EP2T for other vision tasks such as flow estimation, event-to-image reconstruction and object recognition. The source code can be found at: https://github.com/Xmu-qcj/E2PNet.","url_abs":"https://arxiv.org/abs/2311.18433v2","url_pdf":"https://arxiv.org/pdf/2311.18433v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"e2pnet-event-to-point-cloud-registration-with-1","repo_url":"https://github.com/xmu-qcj/e2pnet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-reconstruction","task_name":"Image Reconstruction"},{"task_slug":"object-recognition","task_name":"Object Recognition"},{"task_slug":"point-cloud-registration","task_name":"Point Cloud Registration"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.18433","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.18433"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xmu-qcj/e2pnet","reach":null}],"summary":{"ran":1,"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"official":{"samples":9,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"0faa8f3c4e86bc1f","entry":"STA","repo":"xmu-qcj/e2pnet","repo_kind":"official","path":"EP2T/EP2T.py","file_url":"https://github.com/xmu-qcj/e2pnet/blob/HEAD/EP2T/EP2T.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0faa8f3c4e86bc1f"}},{"code_sha256_prefix":"9b47a4aeedfe3073","entry":"normalize_event_volume","repo":"xmu-qcj/e2pnet","repo_kind":"official","path":"EP2T/EP2T.py","file_url":"https://github.com/xmu-qcj/e2pnet/blob/HEAD/EP2T/EP2T.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9b47a4aeedfe3073"}},{"code_sha256_prefix":"2fbd100a3e967f1b","entry":"Attention","repo":"xmu-qcj/e2pnet","repo_kind":"official","path":"EP2T/EP2T.py","file_url":"https://github.com/xmu-qcj/e2pnet/blob/HEAD/EP2T/EP2T.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2fbd100a3e967f1b"}},{"code_sha256_prefix":"73ce132542682bb9","entry":"EP2T","repo":"xmu-qcj/e2pnet","repo_kind":"official","path":"EP2T/EP2T.py","file_url":"https://github.com/xmu-qcj/e2pnet/blob/HEAD/EP2T/EP2T.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"73ce132542682bb9"}},{"code_sha256_prefix":"48d9b916798535ef","entry":"PointNetAutoencoder","repo":"xmu-qcj/e2pnet","repo_kind":"official","path":"EP2T/EP2T.py","file_url":"https://github.com/xmu-qcj/e2pnet/blob/HEAD/EP2T/EP2T.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"48d9b916798535ef"}},{"code_sha256_prefix":"a916469e7b712bd3","entry":"PointNetEncoder","repo":"xmu-qcj/e2pnet","repo_kind":"official","path":"EP2T/EP2T.py","file_url":"https://github.com/xmu-qcj/e2pnet/blob/HEAD/EP2T/EP2T.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a916469e7b712bd3"}},{"code_sha256_prefix":"b3efafb434123af6","entry":"random_pose","repo":"xmu-qcj/e2pnet","repo_kind":"official","path":"Dataset/data_loader_MVSEC.py","file_url":"https://github.com/xmu-qcj/e2pnet/blob/HEAD/Dataset/data_loader_MVSEC.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3efafb434123af6"}},{"code_sha256_prefix":"d50438ea5f0c80b2","entry":"random_rotation","repo":"xmu-qcj/e2pnet","repo_kind":"official","path":"Dataset/data_loader_MVSEC.py","file_url":"https://github.com/xmu-qcj/e2pnet/blob/HEAD/Dataset/data_loader_MVSEC.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d50438ea5f0c80b2"}},{"code_sha256_prefix":"09fca7a4277ca5d2","entry":"random_translation","repo":"xmu-qcj/e2pnet","repo_kind":"official","path":"Dataset/data_loader_MVSEC.py","file_url":"https://github.com/xmu-qcj/e2pnet/blob/HEAD/Dataset/data_loader_MVSEC.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09fca7a4277ca5d2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}