{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-better-generalization-joint-depth","title":"Towards Better Generalization: Joint Depth-Pose Learning without PoseNet","arxiv_id":"2004.01314","date":"2020-04-03","proceeding":"CVPR 2020 6","authors":["Wang Zhao","Shaohui Liu","Yezhi Shu","Yong-Jin Liu"],"abstract":"In this work, we tackle the essential problem of scale inconsistency for self-supervised joint depth-pose learning. Most existing methods assume that a consistent scale of depth and pose can be learned across all input samples, which makes the learning problem harder, resulting in degraded performance and limited generalization in indoor environments and long-sequence visual odometry application. To address this issue, we propose a novel system that explicitly disentangles scale from the network estimation. Instead of relying on PoseNet architecture, our method recovers relative pose by directly solving fundamental matrix from dense optical flow correspondence and makes use of a two-view triangulation module to recover an up-to-scale 3D structure. Then, we align the scale of the depth prediction with the triangulated point cloud and use the transformed depth map for depth error computation and dense reprojection check. Our whole system can be jointly trained end-to-end. Extensive experiments show that our system not only reaches state-of-the-art performance on KITTI depth and flow estimation, but also significantly improves the generalization ability of existing self-supervised depth-pose learning methods under a variety of challenging scenarios, and achieves state-of-the-art results among self-supervised learning-based methods on KITTI Odometry and NYUv2 dataset. Furthermore, we present some interesting findings on the limitation of PoseNet-based relative pose estimation methods in terms of generalization ability. Code is available at https://github.com/B1ueber2y/TrianFlow.","url_abs":"https://arxiv.org/abs/2004.01314v2","url_pdf":"https://arxiv.org/pdf/2004.01314v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-better-generalization-joint-depth","repo_url":"https://github.com/B1ueber2y/TrianFlow","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"towards-better-generalization-joint-depth","repo_url":"https://github.com/2023-MindSpore-1/ms-code-215/tree/main/PoseNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"towards-better-generalization-joint-depth","repo_url":"https://github.com/MS-Mind/MS-Code-06/tree/main/PoseNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"towards-better-generalization-joint-depth","repo_url":"https://github.com/MindSpore-paper-code-2/code2/tree/main/PoseNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"depth-prediction","task_name":"Depth Prediction"},{"task_slug":"monocular-depth-estimation","task_name":"Monocular Depth Estimation"},{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"visual-odometry","task_name":"Visual Odometry"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/monocular-depth-estimation-on-nyu-depth-v2-4","task":"Monocular Depth Estimation","dataset":"NYU-Depth V2 self-supervised","model":"Zhao et al","rank_in_archive_order":7,"of":8,"metrics":{"Absolute relative error (AbsRel)":"0.189","Root mean square error (RMSE)":"0.686","delta_1":"70.1","delta_2":"91.2","delta_3":"97.8"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2004.01314","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.01314"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/2023-MindSpore-1/ms-code-215/tree/main/PoseNet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-paper-code-2/code2/tree/main/PoseNet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/B1ueber2y/TrianFlow","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MS-Mind/MS-Code-06/tree/main/PoseNet","reach":null}],"summary":{"ran_draft_wrong":1,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"722c49a1e9f7a775","entry":"projection","repo":"B1ueber2y/TrianFlow","repo_kind":"official","path":"infer_vo.py","file_url":"https://github.com/B1ueber2y/TrianFlow/blob/HEAD/infer_vo.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"722c49a1e9f7a775"}},{"code_sha256_prefix":"2fa7f4830a222758","entry":"unprojection","repo":"B1ueber2y/TrianFlow","repo_kind":"official","path":"infer_vo.py","file_url":"https://github.com/B1ueber2y/TrianFlow/blob/HEAD/infer_vo.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2fa7f4830a222758"}},{"code_sha256_prefix":"1635151d9d4ed1fd","entry":"cv_triangulation","repo":"B1ueber2y/TrianFlow","repo_kind":"official","path":"infer_vo.py","file_url":"https://github.com/B1ueber2y/TrianFlow/blob/HEAD/infer_vo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1635151d9d4ed1fd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}