{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/3d-implicit-transporter-for-temporally","title":"3D Implicit Transporter for Temporally Consistent Keypoint Discovery","arxiv_id":"2309.05098","date":"2023-09-10","proceeding":"ICCV 2023 1","authors":["Chengliang Zhong","Yuhang Zheng","Yupeng Zheng","Hao Zhao","Li Yi","Xiaodong Mu","Ling Wang","Pengfei Li","Guyue Zhou","Chao Yang","Xinliang Zhang","Jian Zhao"],"abstract":"Keypoint-based representation has proven advantageous in various visual and robotic tasks. However, the existing 2D and 3D methods for detecting keypoints mainly rely on geometric consistency to achieve spatial alignment, neglecting temporal consistency. To address this issue, the Transporter method was introduced for 2D data, which reconstructs the target frame from the source frame to incorporate both spatial and temporal information. However, the direct application of the Transporter to 3D point clouds is infeasible due to their structural differences from 2D images. Thus, we propose the first 3D version of the Transporter, which leverages hybrid 3D representation, cross attention, and implicit reconstruction. We apply this new learning system on 3D articulated objects and nonrigid animals (humans and rodents) and show that learned keypoints are spatio-temporally consistent. Additionally, we propose a closed-loop control strategy that utilizes the learned keypoints for 3D object manipulation and demonstrate its superior performance. Codes are available at https://github.com/zhongcl-thu/3D-Implicit-Transporter.","url_abs":"https://arxiv.org/abs/2309.05098v1","url_pdf":"https://arxiv.org/pdf/2309.05098v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"3d-implicit-transporter-for-temporally","repo_url":"https://github.com/zhongcl-thu/3d-implicit-transporter","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.05098","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.05098"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhongcl-thu/3d-implicit-transporter","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":14,"unverified":2},"by_repo_kind":{"official":{"samples":16,"ran":14,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"60f2f4207ae20ab9","entry":"angles2rotation_matrix","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/datasets/augmentation.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/datasets/augmentation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60f2f4207ae20ab9"}},{"code_sha256_prefix":"61969ba9c876c10d","entry":"atomic_rotate_pytorch_batch","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/datasets/augmentation.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/datasets/augmentation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61969ba9c876c10d"}},{"code_sha256_prefix":"8e6b5059f61d84e6","entry":"chamfer_distance_naive","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/nets/common.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/nets/common.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e6b5059f61d84e6"}},{"code_sha256_prefix":"0072de07b144b152","entry":"compute_iou","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/nets/common.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/nets/common.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0072de07b144b152"}},{"code_sha256_prefix":"5f7e1b27b1346b64","entry":"compute_keypoint_location_mean","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/nets/transporter_net.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/nets/transporter_net.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f7e1b27b1346b64"}},{"code_sha256_prefix":"63dac5bcaa4953a7","entry":"cosine","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/losses/pose_loss.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/losses/pose_loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63dac5bcaa4953a7"}},{"code_sha256_prefix":"731d9d7620d9a10c","entry":"gaussian_grid","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/nets/transporter_net.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/nets/transporter_net.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"731d9d7620d9a10c"}},{"code_sha256_prefix":"dcf31906f80d979c","entry":"get_ray_directions_cam","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/nets/geometry.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/nets/geometry.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dcf31906f80d979c"}},{"code_sha256_prefix":"9babaac7293b5582","entry":"make_3d_grid","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/datasets/train_dataset.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/datasets/train_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9babaac7293b5582"}},{"code_sha256_prefix":"f28b37b553b6d834","entry":"mlp","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/nets/model_utils.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/nets/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f28b37b553b6d834"}},{"code_sha256_prefix":"cadd5283381515f2","entry":"query_ball_point","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/nets/model_utils.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/nets/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cadd5283381515f2"}},{"code_sha256_prefix":"54dc24b743feff79","entry":"reflect_vector_on_vector","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/nets/geometry.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/nets/geometry.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54dc24b743feff79"}},{"code_sha256_prefix":"7d6dcfc5a36f8ceb","entry":"rotate","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/datasets/augmentation.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/datasets/augmentation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d6dcfc5a36f8ceb"}},{"code_sha256_prefix":"9eefd295be42d857","entry":"spatial_softmax","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/nets/transporter_net.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/nets/transporter_net.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9eefd295be42d857"}},{"code_sha256_prefix":"f4d8f4c4d0f7da71","entry":"chamfer_distance","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/nets/common.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/nets/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f4d8f4c4d0f7da71"}},{"code_sha256_prefix":"6ddec81b1d23c787","entry":"square_distance","repo":"zhongcl-thu/3D-Implicit-Transporter","repo_kind":"official","path":"core/nets/model_utils.py","file_url":"https://github.com/zhongcl-thu/3D-Implicit-Transporter/blob/HEAD/core/nets/model_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6ddec81b1d23c787"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}