{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/primedepth-efficient-monocular-depth","title":"PrimeDepth: Efficient Monocular Depth Estimation with a Stable Diffusion Preimage","arxiv_id":"2409.09144","date":"2024-09-13","proceeding":null,"authors":["Denis Zavadski","Damjan Kalšan","Carsten Rother"],"abstract":"This work addresses the task of zero-shot monocular depth estimation. A recent advance in this field has been the idea of utilising Text-to-Image foundation models, such as Stable Diffusion. Foundation models provide a rich and generic image representation, and therefore, little training data is required to reformulate them as a depth estimation model that predicts highly-detailed depth maps and has good generalisation capabilities. However, the realisation of this idea has so far led to approaches which are, unfortunately, highly inefficient at test-time due to the underlying iterative denoising process. In this work, we propose a different realisation of this idea and present PrimeDepth, a method that is highly efficient at test time while keeping, or even enhancing, the positive aspects of diffusion-based approaches. Our key idea is to extract from Stable Diffusion a rich, but frozen, image representation by running a single denoising step. This representation, we term preimage, is then fed into a refiner network with an architectural inductive bias, before entering the downstream task. We validate experimentally that PrimeDepth is two orders of magnitude faster than the leading diffusion-based method, Marigold, while being more robust for challenging scenarios and quantitatively marginally superior. Thereby, we reduce the gap to the currently leading data-driven approach, Depth Anything, which is still quantitatively superior, but predicts less detailed depth maps and requires 20 times more labelled data. Due to the complementary nature of our approach, even a simple averaging between PrimeDepth and Depth Anything predictions can improve upon both methods and sets a new state-of-the-art in zero-shot monocular depth estimation. In future, data-driven approaches may also benefit from integrating our preimage.","url_abs":"https://arxiv.org/abs/2409.09144v1","url_pdf":"https://arxiv.org/pdf/2409.09144v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"primedepth-efficient-monocular-depth","repo_url":"https://github.com/vislearn/PrimeDepth","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"monocular-depth-estimation","task_name":"Monocular Depth Estimation"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"zero-shot-generalization","task_name":"Zero-shot Generalization"}],"methods":[{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/monocular-depth-estimation-on-eth3d","task":"Monocular Depth Estimation","dataset":"ETH3D","model":"PrimeDepth","rank_in_archive_order":3,"of":10,"metrics":{"Delta < 1.25":"0.967","absolute relative error":"0.068"},"uses_additional_data":false},{"leaderboard":"/sota/monocular-depth-estimation-on-kitti-eigen","task":"Monocular Depth Estimation","dataset":"KITTI Eigen split","model":"PrimeDepth + Depth Anything","rank_in_archive_order":44,"of":79,"metrics":{"Delta < 1.25":"0.953","absolute relative error":"0.073"},"uses_additional_data":false},{"leaderboard":"/sota/monocular-depth-estimation-on-kitti-eigen","task":"Monocular Depth Estimation","dataset":"KITTI Eigen split","model":"PrimeDepth","rank_in_archive_order":46,"of":79,"metrics":{"Delta < 1.25":"0.937","absolute relative error":"0.079"},"uses_additional_data":false},{"leaderboard":"/sota/monocular-depth-estimation-on-nyu-depth-v2","task":"Monocular Depth Estimation","dataset":"NYU-Depth V2","model":"PrimeDepth + Depth Anything","rank_in_archive_order":5,"of":85,"metrics":{"Delta < 1.25":"0.977","absolute relative error":"0.046"},"uses_additional_data":false},{"leaderboard":"/sota/monocular-depth-estimation-on-nyu-depth-v2","task":"Monocular Depth Estimation","dataset":"NYU-Depth V2","model":"PrimeDepth","rank_in_archive_order":13,"of":85,"metrics":{"Delta < 1.25":"0.966","absolute relative error":"0.058"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.09144","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.09144"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vislearn/PrimeDepth","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":3,"ran_fixture":2,"ran":4,"ran_honours":2,"ran_draft_wrong":2,"unverified":2},"by_repo_kind":{"official":{"samples":15,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"424012cb37b31172","entry":"default","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"424012cb37b31172"}},{"code_sha256_prefix":"4cb732f513d69dfd","entry":"disabled_train","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/models/diffusion/ddpm.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/models/diffusion/ddpm.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb732f513d69dfd"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"cb49209c125de1b4","entry":"get_timestep_embedding","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/modules/diffusionmodules/model.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/modules/diffusionmodules/model.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb49209c125de1b4"}},{"code_sha256_prefix":"b1368330cf0f5642","entry":"isimage","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1368330cf0f5642"}},{"code_sha256_prefix":"d72762b700feee6f","entry":"ismap","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d72762b700feee6f"}},{"code_sha256_prefix":"3bd7e0cdbd131fdb","entry":"make_beta_schedule","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/modules/diffusionmodules/util.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/modules/diffusionmodules/util.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3bd7e0cdbd131fdb"}},{"code_sha256_prefix":"3ee640131c9d4362","entry":"make_ddim_sampling_parameters","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/modules/diffusionmodules/util.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/modules/diffusionmodules/util.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3ee640131c9d4362"}},{"code_sha256_prefix":"0ea4e960ea54514c","entry":"make_ddim_timesteps","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/modules/diffusionmodules/util.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/modules/diffusionmodules/util.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ea4e960ea54514c"}},{"code_sha256_prefix":"3137073275f8c21a","entry":"nonlinearity","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/modules/diffusionmodules/model.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/modules/diffusionmodules/model.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3137073275f8c21a"}},{"code_sha256_prefix":"17eb0a31c91b671c","entry":"normal_kl","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/modules/distributions/distributions.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/modules/distributions/distributions.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17eb0a31c91b671c"}},{"code_sha256_prefix":"9c98c9b6c537a134","entry":"normalization","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/modules/diffusionmodules/labeller.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/modules/diffusionmodules/labeller.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c98c9b6c537a134"}},{"code_sha256_prefix":"9a299fe5ae09e407","entry":"uniq","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a299fe5ae09e407"}},{"code_sha256_prefix":"4f8734a02fc01ca2","entry":"find_denominator","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/modules/diffusionmodules/labeller.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/modules/diffusionmodules/labeller.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4f8734a02fc01ca2"}},{"code_sha256_prefix":"6b4c1a8ce5fb7282","entry":"log_txt_as_img","repo":"vislearn/PrimeDepth","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/vislearn/PrimeDepth/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b4c1a8ce5fb7282"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}