{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mvsformer-learning-robust-image","title":"MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth","arxiv_id":"2208.02541","date":"2022-08-04","proceeding":null,"authors":["Chenjie Cao","Xinlin Ren","Yanwei Fu"],"abstract":"Feature representation learning is the key recipe for learning-based Multi-View Stereo (MVS). As the common feature extractor of learning-based MVS, vanilla Feature Pyramid Networks (FPNs) suffer from discouraged feature representations for reflection and texture-less areas, which limits the generalization of MVS. Even FPNs worked with pre-trained Convolutional Neural Networks (CNNs) fail to tackle these issues. On the other hand, Vision Transformers (ViTs) have achieved prominent success in many 2D vision tasks. Thus we ask whether ViTs can facilitate feature learning in MVS? In this paper, we propose a pre-trained ViT enhanced MVS network called MVSFormer, which can learn more reliable feature representations benefited by informative priors from ViT. The finetuned MVSFormer with hierarchical ViTs of efficient attention mechanisms can achieve prominent improvement based on FPNs. Besides, the alternative MVSFormer with frozen ViT weights is further proposed. This largely alleviates the training cost with competitive performance strengthened by the attention map from the self-distillation pre-training. MVSFormer can be generalized to various input resolutions with efficient multi-scale training strengthened by gradient accumulation. Moreover, we discuss the merits and drawbacks of classification and regression-based MVS methods, and further propose to unify them with a temperature-based strategy. MVSFormer achieves state-of-the-art performance on the DTU dataset. Particularly, MVSFormer ranks as Top-1 on both intermediate and advanced sets of the highly competitive Tanks-and-Temples leaderboard.","url_abs":"https://arxiv.org/abs/2208.02541v3","url_pdf":"https://arxiv.org/pdf/2208.02541v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mvsformer-learning-robust-image","repo_url":"https://github.com/ewrfcas/mvsformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"point-clouds","task_name":"Point Clouds"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-reconstruction-on-dtu","task":"3D Reconstruction","dataset":"DTU","model":"MVSFormer","rank_in_archive_order":3,"of":24,"metrics":{"Acc":"0.327","Comp":"0.251","Overall":"0.289"},"uses_additional_data":false},{"leaderboard":"/sota/point-clouds-on-tanks-and-temples","task":"Point Clouds","dataset":"Tanks and Temples","model":"MVSFormer","rank_in_archive_order":3,"of":21,"metrics":{"Mean F1 (Advanced)":"40.87","Mean F1 (Intermediate)":"66.37"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2208.02541","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2208.02541"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ewrfcas/mvsformer","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":9,"ran_honours":1,"ran_fixture":1,"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":15,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d17a4985f968317d","entry":"conf_regression","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/module.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/module.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d17a4985f968317d"}},{"code_sha256_prefix":"16a0f9b77eb60de7","entry":"depth_regression","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/module.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/module.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"16a0f9b77eb60de7"}},{"code_sha256_prefix":"55120f2026b56aa2","entry":"drop_path","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/vision_transformer.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/vision_transformer.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"55120f2026b56aa2"}},{"code_sha256_prefix":"79fd467dde213efb","entry":"expand_as","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/warping.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/warping.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"79fd467dde213efb"}},{"code_sha256_prefix":"d94135dc22062b8d","entry":"get_layer_id_for_vit","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/lr_decay.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/lr_decay.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d94135dc22062b8d"}},{"code_sha256_prefix":"c72a4a2293997c31","entry":"init_range","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/module.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/module.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c72a4a2293997c31"}},{"code_sha256_prefix":"2c4f405822853fe6","entry":"lift","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/warping.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/warping.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2c4f405822853fe6"}},{"code_sha256_prefix":"a76fc7fb6958f97c","entry":"make_nograd_func","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"utils.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a76fc7fb6958f97c"}},{"code_sha256_prefix":"b62bec09ceeaca79","entry":"make_recursive_func","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"utils.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b62bec09ceeaca79"}},{"code_sha256_prefix":"904eb124a1c06ae8","entry":"param_groups_lrd","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/lr_decay.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/lr_decay.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"904eb124a1c06ae8"}},{"code_sha256_prefix":"a06b72ce1e7e80f4","entry":"parse_intrinsics","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/warping.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/warping.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a06b72ce1e7e80f4"}},{"code_sha256_prefix":"312f59d39718ea4b","entry":"simple_loss","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/losses.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"312f59d39718ea4b"}},{"code_sha256_prefix":"d9f8f54e0803bc1d","entry":"reg_loss","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/losses.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d9f8f54e0803bc1d"}},{"code_sha256_prefix":"1e4819f1df315d5c","entry":"reg_loss_stage4","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"models/losses.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/models/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1e4819f1df315d5c"}},{"code_sha256_prefix":"8e28f96374f1b285","entry":"tensor2float","repo":"ewrfcas/mvsformer","repo_kind":"official","path":"utils.py","file_url":"https://github.com/ewrfcas/mvsformer/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8e28f96374f1b285"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}