{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ecodepth-effective-conditioning-of-diffusion","title":"ECoDepth: Effective Conditioning of Diffusion Models for Monocular Depth Estimation","arxiv_id":"2403.18807","date":"2024-03-27","proceeding":"CVPR 2024 1","authors":["Suraj Patni","Aradhye Agarwal","Chetan Arora"],"abstract":"In the absence of parallax cues, a learning-based single image depth estimation (SIDE) model relies heavily on shading and contextual cues in the image. While this simplicity is attractive, it is necessary to train such models on large and varied datasets, which are difficult to capture. It has been shown that using embeddings from pre-trained foundational models, such as CLIP, improves zero shot transfer in several applications. Taking inspiration from this, in our paper we explore the use of global image priors generated from a pre-trained ViT model to provide more detailed contextual information. We argue that the embedding vector from a ViT model, pre-trained on a large dataset, captures greater relevant information for SIDE than the usual route of generating pseudo image captions, followed by CLIP based text embeddings. Based on this idea, we propose a new SIDE model using a diffusion backbone which is conditioned on ViT embeddings. Our proposed design establishes a new state-of-the-art (SOTA) for SIDE on NYUv2 dataset, achieving Abs Rel error of 0.059 (14% improvement) compared to 0.069 by the current SOTA (VPD). And on KITTI dataset, achieving Sq Rel error of 0.139 (2% improvement) compared to 0.142 by the current SOTA (GEDepth). For zero-shot transfer with a model trained on NYUv2, we report mean relative improvement of (20%, 23%, 81%, 25%) over NeWCRFs on (Sun-RGBD, iBims1, DIODE, HyperSim) datasets, compared to (16%, 18%, 45%, 9%) by ZoeDepth. The project page is available at https://ecodepth-iitd.github.io","url_abs":"https://arxiv.org/abs/2403.18807v4","url_pdf":"https://arxiv.org/pdf/2403.18807v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ecodepth-effective-conditioning-of-diffusion","repo_url":"https://github.com/aradhye2002/ecodepth","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"depth-prediction","task_name":"Depth Prediction"},{"task_slug":"monocular-depth-estimation","task_name":"Monocular Depth Estimation"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/monocular-depth-estimation-on-kitti-eigen","task":"Monocular Depth Estimation","dataset":"KITTI Eigen split","model":"ECoDepth","rank_in_archive_order":12,"of":79,"metrics":{"Delta < 1.25":"0.979","Delta < 1.25^2":"0.998","Delta < 1.25^3":"1.000","RMSE":"1.966","RMSE log":"0.074","Sq Rel":"0.139","absolute relative error":"0.048"},"uses_additional_data":false},{"leaderboard":"/sota/monocular-depth-estimation-on-nyu-depth-v2","task":"Monocular Depth Estimation","dataset":"NYU-Depth V2","model":"ECoDepth","rank_in_archive_order":14,"of":85,"metrics":{"Delta < 1.25":"0.978","Delta < 1.25^2":"0.997","Delta < 1.25^3":"0.999","RMSE":"0.218","absolute relative error":"0.059","log 10":"0.026"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.18807","atlas_url":"https://app.syntology.ai/?focus=2403.18807","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.18807"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aradhye2002/ecodepth","reach":null}],"summary":{"ran":2,"ran_draft_wrong":3,"ran_violates":2,"ran_honours":2,"ran_fixture":2,"unverified":4},"by_repo_kind":{"official":{"samples":15,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"fbe2310addb47db4","entry":"Decoder","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fbe2310addb47db4"}},{"code_sha256_prefix":"21493ab89ab24490","entry":"EmbeddingAdapter","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"21493ab89ab24490"}},{"code_sha256_prefix":"e92e07aa1bf1be20","entry":"compute_metrics","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e92e07aa1bf1be20"}},{"code_sha256_prefix":"f219540998580dc3","entry":"eigen_crop","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f219540998580dc3"}},{"code_sha256_prefix":"64e27454bc7d936a","entry":"garg_crop","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"64e27454bc7d936a"}},{"code_sha256_prefix":"10c824c6cd3213ab","entry":"get_optimizer","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"10c824c6cd3213ab"}},{"code_sha256_prefix":"73c1d068e8ca81bd","entry":"no_crop","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"73c1d068e8ca81bd"}},{"code_sha256_prefix":"36a4962041ede1c2","entry":"pad","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"36a4962041ede1c2"}},{"code_sha256_prefix":"5d3ab34c6027afa8","entry":"pad_to_make_square","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5d3ab34c6027afa8"}},{"code_sha256_prefix":"46cbbcee59b61483","entry":"silog","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"46cbbcee59b61483"}},{"code_sha256_prefix":"89728325857a7d3e","entry":"unpad","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"89728325857a7d3e"}},{"code_sha256_prefix":"fdf6b86f9b1766b1","entry":"CIDE","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fdf6b86f9b1766b1"}},{"code_sha256_prefix":"6159b9d4ff6618e7","entry":"EcoDepth","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6159b9d4ff6618e7"}},{"code_sha256_prefix":"b710c0ba3e6d33fd","entry":"EcoDepthEncoder","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"b710c0ba3e6d33fd"}},{"code_sha256_prefix":"71964ceaab35f039","entry":"custom_crop","repo":"aradhye2002/ecodepth","repo_kind":"official","path":"model.py","file_url":"https://github.com/aradhye2002/ecodepth/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"71964ceaab35f039"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}