{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/grounding-image-matching-in-3d-with-mast3r","title":"Grounding Image Matching in 3D with MASt3R","arxiv_id":"2406.09756","date":"2024-06-14","proceeding":null,"authors":["Vincent Leroy","Yohann Cabon","Jérôme Revaud"],"abstract":"Image Matching is a core component of all best-performing algorithms and pipelines in 3D vision. Yet despite matching being fundamentally a 3D problem, intrinsically linked to camera pose and scene geometry, it is typically treated as a 2D problem. This makes sense as the goal of matching is to establish correspondences between 2D pixel fields, but also seems like a potentially hazardous choice. In this work, we take a different stance and propose to cast matching as a 3D task with DUSt3R, a recent and powerful 3D reconstruction framework based on Transformers. Based on pointmaps regression, this method displayed impressive robustness in matching views with extreme viewpoint changes, yet with limited accuracy. We aim here to improve the matching capabilities of such an approach while preserving its robustness. We thus propose to augment the DUSt3R network with a new head that outputs dense local features, trained with an additional matching loss. We further address the issue of quadratic complexity of dense matching, which becomes prohibitively slow for downstream applications if not carefully treated. We introduce a fast reciprocal matching scheme that not only accelerates matching by orders of magnitude, but also comes with theoretical guarantees and, lastly, yields improved results. Extensive experiments show that our approach, coined MASt3R, significantly outperforms the state of the art on multiple matching tasks. In particular, it beats the best published methods by 30% (absolute improvement) in VCRE AUC on the extremely challenging Map-free localization dataset.","url_abs":"https://arxiv.org/abs/2406.09756v1","url_pdf":"https://arxiv.org/pdf/2406.09756v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"grounding-image-matching-in-3d-with-mast3r","repo_url":"https://github.com/naver/mast3r","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.09756","atlas_url":"https://app.syntology.ai/?focus=2406.09756","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.09756"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/naver/mast3r","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":14,"unverified":3},"by_repo_kind":{"official":{"samples":17,"ran":14,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":17,"samples":[{"code_sha256_prefix":"f3fd659d5f95c771","entry":"apply_log_to_norm","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/losses.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f3fd659d5f95c771"}},{"code_sha256_prefix":"474afb3f38b2c1bc","entry":"batched_triangulate","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/cloud_opt/triangulation.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/cloud_opt/triangulation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"474afb3f38b2c1bc"}},{"code_sha256_prefix":"60d067ba9a3fb4a6","entry":"bruteforce_reciprocal_nns","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/fast_nn.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/fast_nn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"60d067ba9a3fb4a6"}},{"code_sha256_prefix":"f6b7e329975502ed","entry":"convert_dust3r_pairs_naming","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/cloud_opt/sparse_ga.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/cloud_opt/sparse_ga.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f6b7e329975502ed"}},{"code_sha256_prefix":"a28441649e04c270","entry":"gamma_loss","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/cloud_opt/utils/losses.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/cloud_opt/utils/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a28441649e04c270"}},{"code_sha256_prefix":"6e6b3f8c6d2c3d69","entry":"get_similarities","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/losses.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e6b3f8c6d2c3d69"}},{"code_sha256_prefix":"eaadf59dfacc29f6","entry":"how_select_local","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/retrieval/model.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/retrieval/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"eaadf59dfacc29f6"}},{"code_sha256_prefix":"fdeb9a710048befb","entry":"l05_loss","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/cloud_opt/utils/losses.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/cloud_opt/utils/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fdeb9a710048befb"}},{"code_sha256_prefix":"57af2b727ab331be","entry":"l1_loss","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/cloud_opt/utils/losses.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/cloud_opt/utils/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"57af2b727ab331be"}},{"code_sha256_prefix":"6575a76869abe3b3","entry":"merge_corres","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/fast_nn.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/fast_nn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6575a76869abe3b3"}},{"code_sha256_prefix":"68208f1e15cc2916","entry":"pcawhitenlearn_shrinkage","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/retrieval/model.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/retrieval/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"68208f1e15cc2916"}},{"code_sha256_prefix":"2e16111f6a6717ac","entry":"reg_desc","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/catmlp_dpt_head.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/catmlp_dpt_head.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"2e16111f6a6717ac"}},{"code_sha256_prefix":"a2c256044b1ba139","entry":"sel","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/image_pairs.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/image_pairs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a2c256044b1ba139"}},{"code_sha256_prefix":"f8448367aee10bcc","entry":"weighted_spoc","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/retrieval/model.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/retrieval/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f8448367aee10bcc"}},{"code_sha256_prefix":"d07895a2f6ed02f8","entry":"filter_pairs_seq","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/image_pairs.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/image_pairs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d07895a2f6ed02f8"}},{"code_sha256_prefix":"9a09f31d98cc4d88","entry":"load_model","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/model.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"9a09f31d98cc4d88"}},{"code_sha256_prefix":"c030fad7954a65d2","entry":"matches_to_depths","repo":"naver/mast3r","repo_kind":"official","path":"mast3r/cloud_opt/triangulation.py","file_url":"https://github.com/naver/mast3r/blob/HEAD/mast3r/cloud_opt/triangulation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c030fad7954a65d2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}