Papers › Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

4 Dec 2023CVPR 2024 1arXiv:2312.02145archive 2025-07-28

Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, Konrad Schindler

Monocular depth estimation is a fundamental computer vision task. Recovering 3D depth from a single image is geometrically ill-posed and requires scene understanding, so it is not surprising that the rise of deep learning has led to a breakthrough. The impressive progress of monocular depth estimators has mirrored the growth in model capacity, from relatively modest CNNs to large Transformer architectures. Still, monocular depth estimators tend to struggle when presented with images with unfamiliar content and layout, since their knowledge of the visual world is restricted by the data seen during training, and challenged by zero-shot generalization to new domains. This motivates us to explore whether the extensive priors captured in recent generative diffusion models can enable better, more generalizable depth estimation. We introduce Marigold, a method for affine-invariant monocular depth estimation that is derived from Stable Diffusion and retains its rich prior knowledge. The estimator can be fine-tuned in a couple of days on a single GPU using only synthetic training data. It delivers state-of-the-art performance across a wide range of datasets, including over 20% performance gains in specific cases. Project page: https://marigoldmonodepth.github.io.

PaperPDFConference PDFCodeCode Syntology ran

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, Syntology's MCP tool lists every function and class Syntology harvested from this paper and whether it ran (how to connect): get_harvested_code_for_paper(arxiv_id="2312.02145")

Code

Syntology Ran 14 of 26 code samples harvested from 3 repositories linked to this paper; 12 have no recorded run. Of those that ran: 3 ran · violated contract; 3 ran · our draft was wrong; 8 ran with no contract checked.

By repository: official repository: 11 samples from 1 repository, 9 ran; community (archive-listed): 15 samples from 2 repositories, 5 ran. The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

prs-eth/marigold officialmentioned on GitHubpytorchApache-2.0 report
Magicboomliu/Accelerator-Simple-Template mentioned on GitHubpytorchMIT report
damaggu/tadp mentioned on GitHubpytorchApache-2.0 report
indu1ge/DepthMaster mentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

26 samples harvested; 14 ran; 0 honoured the contract we drafted; 12 have no recorded run. Read from Syntology's graph 2026-09-24; that is when this build read the record, not when the samples ran.

3ran · violated contract
3ran · our draft was wrong
8ran
12unverified

Licence: 0 of the 26 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from 3 repositories linked to this paper, official or community; each sample names its own and says which. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code.

Each sample ends with its code_sha256, Syntology's identity for that exact code. An agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples from a repository marked official are listed first. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

chw2hwc prs-eth/marigold/marigold/util/image_util.py official repository ran fingerprinted Apache-2.0 (permissive) · 1a9761d7649b539b · report
colorize_depth_maps prs-eth/marigold/marigold/util/image_util.py official repository ran Apache-2.0 (permissive) · 1e519d3cea1f46e3 · report
ensemble_depth prs-eth/marigold/marigold/util/ensemble.py official repository ran Apache-2.0 (permissive) · 55702399683243f2 · report
ensemble_iid prs-eth/marigold/marigold/util/ensemble.py official repository ran Apache-2.0 (permissive) · 8dd1994ceb1eb671 · report
ensemble_normals prs-eth/marigold/marigold/util/ensemble.py official repository ran Apache-2.0 (permissive) · 6f901ea1ce90a56a · report
extract_into_tensor prs-eth/marigold/src/trainer/marigold_depth_LCM_trainer.py official repository ran · our draft was wrong Apache-2.0 (permissive) · a15cfd7932844ef9 · report
get_predicted_original_sample prs-eth/marigold/src/trainer/marigold_depth_LCM_trainer.py official repository ran Apache-2.0 (permissive) · 1950491589be0476 · report
resize_max_res prs-eth/marigold/marigold/util/image_util.py official repository ran Apache-2.0 (permissive) · 715b48a5e7426c5e · report
scalings_for_boundary_conditions prs-eth/marigold/src/trainer/marigold_depth_LCM_trainer.py official repository ran · violated contract fingerprinted Apache-2.0 (permissive) · 74a9b4e5e34c0860 · report
find_batch_size prs-eth/marigold/marigold/util/batchsize.py official repository unverified Apache-2.0 (permissive) · 55c27917099f7dc3 · report
get_pred_name prs-eth/marigold/src/dataset/base_depth_dataset.py official repository unverified Apache-2.0 (permissive) · 19b4e7dbd2c70b35 · report
chunk damaggu/tadp/TADP/tadp_seg.py community (archive-listed) ran · our draft was wrong Apache-2.0 (permissive) · 8241c0562bc710fd · report
default damaggu/tadp/TADP/vpd/models.py community (archive-listed) ran · violated contract Apache-2.0 (permissive) · 424012cb37b31172 · report
exists damaggu/tadp/TADP/vpd/models.py community (archive-listed) ran · violated contract Apache-2.0 (permissive) · aa5486a3650902d8 · report
read_text_lines Magicboomliu/Accelerator-Simple-Template/dataloader/utils.py community (archive-listed) ran MIT (permissive) · 78088e741a0c4b80 · report
uniq damaggu/tadp/TADP/vpd/models.py community (archive-listed) ran · our draft was wrong Apache-2.0 (permissive) · 9a299fe5ae09e407 · report
Disparity_Normalization Magicboomliu/Accelerator-Simple-Template/playground/check_depth_est.py community (archive-listed) unverified MIT (permissive) · babc9c3037a5b587 · report
depth2disp Magicboomliu/Accelerator-Simple-Template/dataloader/file_io.py community (archive-listed) unverified MIT (permissive) · 7d50f6c8900b3933 · report
depth2disp_cv Magicboomliu/Accelerator-Simple-Template/dataloader/file_io.py community (archive-listed) unverified MIT (permissive) · 12e95408bfe2ed81 · report
find_batch_size Magicboomliu/Accelerator-Simple-Template/utils/batch_size.py community (archive-listed) unverified MIT (permissive) · 28a8af1b1cd8cb05 · report
find_occ_mask Magicboomliu/Accelerator-Simple-Template/dataloader/utils.py community (archive-listed) unverified MIT (permissive) · bd107ff2d1229ab5 · report
int_list Magicboomliu/Accelerator-Simple-Template/dataloader/utils.py community (archive-listed) unverified MIT (permissive) · 76f433ffb0eb1150 · report
kitti_colormap Magicboomliu/Accelerator-Simple-Template/utils/colormap.py community (archive-listed) unverified MIT (permissive) · 50a3e2b7e8148db0 · report
read_16bit_gt Magicboomliu/Accelerator-Simple-Template/utils/colormap.py community (archive-listed) unverified MIT (permissive) · bf3d969e975a0237 · report
read_img Magicboomliu/Accelerator-Simple-Template/dataloader/file_io.py community (archive-listed) unverified MIT (permissive) · a5edaeb9db495d4c · report
resize_max_res_tensor Magicboomliu/Accelerator-Simple-Template/playground/check_depth_est.py community (archive-listed) unverified MIT (permissive) · ff6b98fbbe52720a · report

Tasks

Depth EstimationMonocular Depth EstimationScene UnderstandingZero-shot Generalization

1 archive task tag without a task page not shown.

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Monocular Depth Estimation ETH3D Marigold Delta < 1.25 0.0960 #6 of 10 Archive leaderboard report
Monocular Depth Estimation ETH3D Marigold absolute relative error 0.065 #6 of 10 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Marigold Delta < 1.25 0.916 #58 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Marigold Delta < 1.25^2 0.987 #58 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Marigold Delta < 1.25^3 0.996 #58 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Marigold RMSE 3.304 #58 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Marigold RMSE log 0.138 #58 of 79 Archive leaderboard report
Monocular Depth Estimation KITTI Eigen split Marigold absolute relative error 0.099 #58 of 79 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Marigold Delta < 1.25 0.964 #10 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Marigold Delta < 1.25^2 0.991 #10 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Marigold Delta < 1.25^3 0.998 #10 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Marigold RMSE 0.224 #10 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Marigold absolute relative error 0.055 #10 of 85 Archive leaderboard report
Monocular Depth Estimation NYU-Depth V2 Marigold log 10 0.024 #10 of 85 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position EncodingsAdamAttentionBPEDense ConnectionsDiffusionDropoutLabel SmoothingLayer NormalizationLinear LayerMulti-Head AttentionPosition-Wise Feed-Forward LayerResidual ConnectionSoftmaxTransformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections