{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/depthcrafter-generating-consistent-long-depth","title":"DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos","arxiv_id":"2409.02095","date":"2024-09-03","proceeding":"CVPR 2025 1","authors":["WenBo Hu","Xiangjun Gao","Xiaoyu Li","Sijie Zhao","Xiaodong Cun","Yong Zhang","Long Quan","Ying Shan"],"abstract":"Estimating video depth in open-world scenarios is challenging due to the diversity of videos in appearance, content motion, camera movement, and length. We present DepthCrafter, an innovative method for generating temporally consistent long depth sequences with intricate details for open-world videos, without requiring any supplementary information such as camera poses or optical flow. The generalization ability to open-world videos is achieved by training the video-to-depth model from a pre-trained image-to-video diffusion model, through our meticulously designed three-stage training strategy. Our training approach enables the model to generate depth sequences with variable lengths at one time, up to 110 frames, and harvest both precise depth details and rich content diversity from realistic and synthetic datasets. We also propose an inference strategy that can process extremely long videos through segment-wise estimation and seamless stitching. Comprehensive evaluations on multiple datasets reveal that DepthCrafter achieves state-of-the-art performance in open-world video depth estimation under zero-shot settings. Furthermore, DepthCrafter facilitates various downstream applications, including depth-based visual effects and conditional video generation.","url_abs":"https://arxiv.org/abs/2409.02095v2","url_pdf":"https://arxiv.org/pdf/2409.02095v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"depthcrafter-generating-consistent-long-depth","repo_url":"https://github.com/Tencent/DepthCrafter","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"monocular-depth-estimation","task_name":"Monocular Depth Estimation"},{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.02095","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.02095"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Tencent/DepthCrafter","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":8,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"c9a4d7f1306c0b7c","entry":"abs_relative_difference","repo":"Tencent/DepthCrafter","repo_kind":"official","path":"benchmark/eval/metric.py","file_url":"https://github.com/Tencent/DepthCrafter/blob/HEAD/benchmark/eval/metric.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c9a4d7f1306c0b7c"}},{"code_sha256_prefix":"031efa299514c3e9","entry":"depth_read","repo":"Tencent/DepthCrafter","repo_kind":"official","path":"benchmark/dataset_extract/dataset_extract_scannet.py","file_url":"https://github.com/Tencent/DepthCrafter/blob/HEAD/benchmark/dataset_extract/dataset_extract_scannet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"031efa299514c3e9"}},{"code_sha256_prefix":"5012368b0af38e0b","entry":"depth_read","repo":"Tencent/DepthCrafter","repo_kind":"official","path":"benchmark/dataset_extract/dataset_extract_bonn.py","file_url":"https://github.com/Tencent/DepthCrafter/blob/HEAD/benchmark/dataset_extract/dataset_extract_bonn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5012368b0af38e0b"}},{"code_sha256_prefix":"4cdb748c2ebad535","entry":"depth_read","repo":"Tencent/DepthCrafter","repo_kind":"official","path":"benchmark/dataset_extract/dataset_extract_kitti.py","file_url":"https://github.com/Tencent/DepthCrafter/blob/HEAD/benchmark/dataset_extract/dataset_extract_kitti.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4cdb748c2ebad535"}},{"code_sha256_prefix":"a7a0355426a4d200","entry":"depth_read","repo":"Tencent/DepthCrafter","repo_kind":"official","path":"benchmark/dataset_extract/dataset_extract_sintel.py","file_url":"https://github.com/Tencent/DepthCrafter/blob/HEAD/benchmark/dataset_extract/dataset_extract_sintel.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a7a0355426a4d200"}},{"code_sha256_prefix":"e94c966db5d2e762","entry":"rmse_linear","repo":"Tencent/DepthCrafter","repo_kind":"official","path":"benchmark/eval/metric.py","file_url":"https://github.com/Tencent/DepthCrafter/blob/HEAD/benchmark/eval/metric.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e94c966db5d2e762"}},{"code_sha256_prefix":"37bf581bc3ccf9f2","entry":"squared_relative_difference","repo":"Tencent/DepthCrafter","repo_kind":"official","path":"benchmark/eval/metric.py","file_url":"https://github.com/Tencent/DepthCrafter/blob/HEAD/benchmark/eval/metric.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"37bf581bc3ccf9f2"}},{"code_sha256_prefix":"a2f8f7129f63f155","entry":"vis_sequence_depth","repo":"Tencent/DepthCrafter","repo_kind":"official","path":"depthcrafter/utils.py","file_url":"https://github.com/Tencent/DepthCrafter/blob/HEAD/depthcrafter/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a2f8f7129f63f155"}},{"code_sha256_prefix":"56c44181ebd50291","entry":"extract_bonn","repo":"Tencent/DepthCrafter","repo_kind":"official","path":"benchmark/dataset_extract/dataset_extract_bonn.py","file_url":"https://github.com/Tencent/DepthCrafter/blob/HEAD/benchmark/dataset_extract/dataset_extract_bonn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"56c44181ebd50291"}},{"code_sha256_prefix":"6211393c309c5e97","entry":"extract_kitti","repo":"Tencent/DepthCrafter","repo_kind":"official","path":"benchmark/dataset_extract/dataset_extract_kitti.py","file_url":"https://github.com/Tencent/DepthCrafter/blob/HEAD/benchmark/dataset_extract/dataset_extract_kitti.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6211393c309c5e97"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}