Papers › Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric...

Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation

22 Mar 2024Under review for Transaction 2024 4arXiv:2404.15506archive 2025-07-28

Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Kaixuan Wang, Hao Chen, Gang Yu, Chunhua Shen, Shaojie Shen

We introduce Metric3D v2, a geometric foundation model for zero-shot metric depth and surface normal estimation from a single image, which is crucial for metric 3D recovery. While depth and normal are geometrically related and highly complimentary, they present distinct challenges. SoTA monocular depth methods achieve zero-shot generalization by learning affine-invariant depths, which cannot recover real-world metrics. Meanwhile, SoTA normal estimation methods have limited zero-shot performance due to the lack of large-scale labeled data. To tackle these issues, we propose solutions for both metric depth estimation and surface normal estimation. For metric depth estimation, we show that the key to a zero-shot single-view model lies in resolving the metric ambiguity from various camera models and large-scale data training. We propose a canonical camera space transformation module, which explicitly addresses the ambiguity problem and can be effortlessly plugged into existing monocular models. For surface normal estimation, we propose a joint depth-normal optimization module to distill diverse data knowledge from metric depth, enabling normal estimators to learn beyond normal labels. Equipped with these modules, our depth-normal models can be stably trained with over 16 million of images from thousands of camera models with different-type annotations, resulting in zero-shot generalization to in-the-wild images with unseen camera settings. Our method enables the accurate recovery of metric 3D structures on randomly collected internet images, paving the way for plausible single-image metrology. Our project page is at https://JUGGHM.github.io/Metric3Dv2.

PaperPDFConference PDFCodeCode Syntology ran

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, Syntology's MCP tool lists every function and class Syntology harvested from this paper and whether it ran (how to connect): get_harvested_code_for_paper(arxiv_id="2404.15506")

Code

Syntology Ran 3 of 3 code samples harvested from 1 repository linked to this paper; 0 have no recorded run. Of those that ran: 1 ran · honoured contract; 2 ran · fixture could not drive it.

By repository: official repository: 3 samples from 1 repository, 3 ran. The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

yvanyin/metric3d officialmentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

3 samples harvested; 3 ran; 1 honoured the contract we drafted; 0 have no recorded run. Read from Syntology's graph 2026-09-24; that is when this build read the record, not when the samples ran.

1ran · honoured contract
2ran · fixture could not drive it

Licence: 0 of the 3 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from yvanyin/metric3d. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code.

Each sample ends with its code_sha256, Syntology's identity for that exact code. An agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples from a repository marked official are listed first. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

drop_add_residual_stochastic_depth yvanyin/metric3d/mono/model/backbones/ViT_DINO.py official repository ran · fixture could not drive it BSD-2-Clause (permissive) · 71b3e03a56d81aec · report
drop_path yvanyin/metric3d/mono/model/backbones/ViT_DINO.py official repository ran · fixture could not drive it BSD-2-Clause (permissive) · c157f5b112b3a392 · report
make_2tuple yvanyin/metric3d/mono/model/backbones/ViT_DINO.py official repository ran · honoured contract fingerprinted BSD-2-Clause (permissive) · 3c04490a215299f2 · report

Tasks

Depth EstimationSurface Normal EstimationZero-shot Generalization

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Monocular Depth Estimation IBims-1 Metric3D-v2(L, ZS) δ1.25 0.969 #4 of 4 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Metric3Dv2 (g2, FT, 80m, flip_aug_test) Delta < 1.25 0.989 #4 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Metric3Dv2 (g2, FT, 80m, flip_aug_test) Delta < 1.25^2 0.998 #4 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Metric3Dv2 (g2, FT, 80m, flip_aug_test) Delta < 1.25^3 1.000 #4 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Metric3Dv2 (g2, FT, 80m, flip_aug_test) RMSE 1.766 #4 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Metric3Dv2 (g2, FT, 80m, flip_aug_test) RMSE log 0.060 #4 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Metric3Dv2 (g2, FT, 80m, flip_aug_test) absolute relative error 0.039 #4 of 79 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Metric3Dv2(L, FT) Delta < 1.25 0.989 #6 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Metric3Dv2(L, FT) Delta < 1.25^2 0.998 #6 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Metric3Dv2(L, FT) Delta < 1.25^3 1.000 #6 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Metric3Dv2(L, FT) RMSE 0.183 #6 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Metric3Dv2(L, FT) absolute relative error 0.047 #6 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Metric3Dv2(L, FT) log 10 0.020 #6 of 85 Archive leaderboard report
Surface Normals Estimation IBims-1 Metric3Dv2(g2, ZS) % < 11.25 69.7 #2 of 2 Archive leaderboard report
Surface Normals Estimation IBims-1 Metric3Dv2(g2, ZS) % < 22.5 76.2 #2 of 2 Archive leaderboard report
Surface Normals Estimation IBims-1 Metric3Dv2(g2, ZS) % < 30 78.8 #2 of 2 Archive leaderboard report
Surface Normals Estimation IBims-1 Metric3Dv2(g2, ZS) Mean 19.6 #2 of 2 Archive leaderboard report
Surface Normals Estimation NYU Depth v2 Metric3Dv2(L, FT) % < 11.25 68.8 #1 of 6 Archive leaderboard report
Surface Normals Estimation NYU Depth v2 Metric3Dv2(L, FT) % < 22.5 84.9 #1 of 6 Archive leaderboard report
Surface Normals Estimation NYU Depth v2 Metric3Dv2(L, FT) % < 30 89.8 #1 of 6 Archive leaderboard report
Surface Normals Estimation NYU Depth v2 Metric3Dv2(L, FT) Mean Angle Error 12.0 #1 of 6 Archive leaderboard report
Surface Normals Estimation NYU Depth v2 Metric3Dv2(L, FT) RMSE 19.2 #1 of 6 Archive leaderboard report
Surface Normals Estimation ScanNetV2 Metric3Dv2 (g2, In-domain) % < 11.25 77.8 #1 of 3 Archive leaderboard report
Surface Normals Estimation ScanNetV2 Metric3Dv2 (g2, In-domain) % < 22.5 90.1 #1 of 3 Archive leaderboard report
Surface Normals Estimation ScanNetV2 Metric3Dv2 (g2, In-domain) % < 30 93.5 #1 of 3 Archive leaderboard report
Surface Normals Estimation ScanNetV2 Metric3Dv2 (g2, In-domain) Mean Angle Error 9.2 #1 of 3 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections