{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mono-vifi-a-unified-learning-framework-for","title":"Mono-ViFI: A Unified Learning Framework for Self-supervised Single- and Multi-frame Monocular Depth Estimation","arxiv_id":"2407.14126","date":"2024-07-19","proceeding":null,"authors":["Jinfeng Liu","Lingtong Kong","Bo Li","Zerong Wang","Hong Gu","Jinwei Chen"],"abstract":"Self-supervised monocular depth estimation has gathered notable interest since it can liberate training from dependency on depth annotations. In monocular video training case, recent methods only conduct view synthesis between existing camera views, leading to insufficient guidance. To tackle this, we try to synthesize more virtual camera views by flow-based video frame interpolation (VFI), termed as temporal augmentation. For multi-frame inference, to sidestep the problem of dynamic objects encountered by explicit geometry-based methods like ManyDepth, we return to the feature fusion paradigm and design a VFI-assisted multi-frame fusion module to align and aggregate multi-frame features, using motion and occlusion information obtained by the flow-based VFI model. Finally, we construct a unified self-supervised learning framework, named Mono-ViFI, to bilaterally connect single- and multi-frame depth. In this framework, spatial data augmentation through image affine transformation is incorporated for data diversity, along with a triplet depth consistency loss for regularization. The single- and multi-frame models can share weights, making our framework compact and memory-efficient. Extensive experiments demonstrate that our method can bring significant improvements to current advanced architectures. Source code is available at https://github.com/LiuJF1226/Mono-ViFI.","url_abs":"https://arxiv.org/abs/2407.14126v1","url_pdf":"https://arxiv.org/pdf/2407.14126v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mono-vifi-a-unified-learning-framework-for","repo_url":"https://github.com/liujf1226/mono-vifi","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"monocular-depth-estimation","task_name":"Monocular Depth Estimation"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":null,"task_name":"Triplet"},{"task_slug":"video-frame-interpolation","task_name":"Video Frame Interpolation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.14126","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.14126"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/liujf1226/mono-vifi","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":10,"ran_honours":1,"ran_draft_wrong":1,"ran_fixture":1,"unverified":3},"by_repo_kind":{"official":{"samples":16,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a141556f0b9d8a06","entry":"compute_errors","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"evaluate_depth.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/evaluate_depth.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a141556f0b9d8a06"}},{"code_sha256_prefix":"62287188376f0ba0","entry":"disp_to_depth","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"layers.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"62287188376f0ba0"}},{"code_sha256_prefix":"c107ece01fa38df6","entry":"flow_to_color","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"flow_vis.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/flow_vis.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c107ece01fa38df6"}},{"code_sha256_prefix":"ae9446901af6b9a8","entry":"flow_uv_to_colors","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"flow_vis.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/flow_vis.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae9446901af6b9a8"}},{"code_sha256_prefix":"4dc728d7076fe0a9","entry":"get_robust_weight","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"networks/IFRNet.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/networks/IFRNet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4dc728d7076fe0a9"}},{"code_sha256_prefix":"955112f5788539a8","entry":"get_translation_matrix","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"layers.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"955112f5788539a8"}},{"code_sha256_prefix":"8bfc895e86bfc7bc","entry":"load_velodyne_points","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"kitti_utils.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/kitti_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8bfc895e86bfc7bc"}},{"code_sha256_prefix":"1df9a5ffd9b38c34","entry":"pil_loader","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"datasets/mono_dataset.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/datasets/mono_dataset.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1df9a5ffd9b38c34"}},{"code_sha256_prefix":"ff833c099a80a327","entry":"read_calib_file","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"kitti_utils.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/kitti_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff833c099a80a327"}},{"code_sha256_prefix":"3073f3b71dc41f8a","entry":"resize","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"networks/IFRNet.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/networks/IFRNet.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3073f3b71dc41f8a"}},{"code_sha256_prefix":"04e9b96b63844176","entry":"sub2ind","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"kitti_utils.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/kitti_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"04e9b96b63844176"}},{"code_sha256_prefix":"cdc03d6bfc4d3a34","entry":"transformation_from_parameters","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"layers.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cdc03d6bfc4d3a34"}},{"code_sha256_prefix":"a4d5c37ca03cf29f","entry":"warp","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"networks/IFRNet.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/networks/IFRNet.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a4d5c37ca03cf29f"}},{"code_sha256_prefix":"6e1c3c123351222b","entry":"batch_post_process_disparity","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"evaluate_depth.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/evaluate_depth.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e1c3c123351222b"}},{"code_sha256_prefix":"4718c67d38954a25","entry":"load_model","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"evaluate_depth.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/evaluate_depth.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4718c67d38954a25"}},{"code_sha256_prefix":"4818f3b037653060","entry":"load_model","repo":"liujf1226/mono-vifi","repo_kind":"official","path":"evaluate_depth_mf.py","file_url":"https://github.com/liujf1226/mono-vifi/blob/HEAD/evaluate_depth_mf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4818f3b037653060"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}