{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multiview-equivariance-improves-3d","title":"Multiview Equivariance Improves 3D Correspondence Understanding with Minimal Feature Finetuning","arxiv_id":"2411.19458","date":"2024-11-29","proceeding":null,"authors":["Yang You","Yixin Li","Congyue Deng","Yue Wang","Leonidas Guibas"],"abstract":"Vision foundation models, particularly the ViT family, have revolutionized image understanding by providing rich semantic features. However, despite their success in 2D comprehension, their abilities on grasping 3D spatial relationships are still unclear. In this work, we evaluate and enhance the 3D awareness of ViT-based models. We begin by systematically assessing their ability to learn 3D equivariant features, specifically examining the consistency of semantic embeddings across different viewpoints. Our findings indicate that improved 3D equivariance leads to better performance on various downstream tasks, including pose estimation, tracking, and semantic transfer. Building on this insight, we propose a simple yet effective finetuning strategy based on 3D correspondences, which significantly enhances the 3D correspondence understanding of existing vision models. Remarkably, even finetuning on a single object for just one iteration results in substantial performance gains. All code and resources will be made publicly available to support further advancements in 3D-aware vision models. Our code is available at https://github.com/qq456cvb/3DCorrEnhance.","url_abs":"https://arxiv.org/abs/2411.19458v1","url_pdf":"https://arxiv.org/pdf/2411.19458v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multiview-equivariance-improves-3d","repo_url":"https://github.com/qq456cvb/3dcorrenhance","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.19458","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.19458"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qq456cvb/3dcorrenhance","reach":null}],"summary":{"ran":2,"ran_honours":2,"ran_fixture":1,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"b6037c922b3d63ac","entry":"AugmentedDataset","repo":"qq456cvb/3dcorrenhance","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/qq456cvb/3dcorrenhance/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b6037c922b3d63ac"}},{"code_sha256_prefix":"ac60e0788f21505e","entry":"_LoRA_qkv","repo":"qq456cvb/3dcorrenhance","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/qq456cvb/3dcorrenhance/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac60e0788f21505e"}},{"code_sha256_prefix":"132d1b5e564f6d5c","entry":"img_coord_2_obj_coord","repo":"qq456cvb/3dcorrenhance","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/qq456cvb/3dcorrenhance/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"132d1b5e564f6d5c"}},{"code_sha256_prefix":"e2c5de7b0aeba434","entry":"interpolate_features","repo":"qq456cvb/3dcorrenhance","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/qq456cvb/3dcorrenhance/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e2c5de7b0aeba434"}},{"code_sha256_prefix":"4b3a13b8aba7b6dc","entry":"sigmoid","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"4b3a13b8aba7b6dc"}},{"code_sha256_prefix":"5987b4adb9004cb6","entry":"FinetuneDINO","repo":"qq456cvb/3dcorrenhance","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/qq456cvb/3dcorrenhance/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5987b4adb9004cb6"}},{"code_sha256_prefix":"9aab6064986132b4","entry":"ObjaverseCorrDataset","repo":"qq456cvb/3dcorrenhance","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/qq456cvb/3dcorrenhance/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9aab6064986132b4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}