{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/monocular-depth-estimation-and-segmentation","title":"Monocular Depth Estimation and Segmentation for Transparent Object with Iterative Semantic and Geometric Fusion","arxiv_id":"2502.14616","date":"2025-02-20","proceeding":null,"authors":["Jiangyuan Liu","Hongxuan Ma","Yuxin Guo","Yuhao Zhao","Chi Zhang","Wei Sui","Wei Zou"],"abstract":"Transparent object perception is indispensable for numerous robotic tasks. However, accurately segmenting and estimating the depth of transparent objects remain challenging due to complex optical properties. Existing methods primarily delve into only one task using extra inputs or specialized sensors, neglecting the valuable interactions among tasks and the subsequent refinement process, leading to suboptimal and blurry predictions. To address these issues, we propose a monocular framework, which is the first to excel in both segmentation and depth estimation of transparent objects, with only a single-image input. Specifically, we devise a novel semantic and geometric fusion module, effectively integrating the multi-scale information between tasks. In addition, drawing inspiration from human perception of objects, we further incorporate an iterative strategy, which progressively refines initial features for clearer results. Experiments on two challenging synthetic and real-world datasets demonstrate that our model surpasses state-of-the-art monocular, stereo, and multi-view methods by a large margin of about 38.8%-46.2% with only a single RGB input. Codes and models are publicly available at https://github.com/L-J-Yuan/MODEST.","url_abs":"https://arxiv.org/abs/2502.14616v1","url_pdf":"https://arxiv.org/pdf/2502.14616v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"monocular-depth-estimation-and-segmentation","repo_url":"https://github.com/l-j-yuan/modest","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"monocular-depth-estimation","task_name":"Monocular Depth Estimation"},{"task_slug":"transparent-objects","task_name":"Transparent objects"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.14616","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.14616"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/l-j-yuan/modest","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":1,"unverified":12},"by_repo_kind":{"official":{"samples":13,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5f4f4355be28c01e","entry":"isnan","repo":"l-j-yuan/modest","repo_kind":"official","path":"datasets/syntodd.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/datasets/syntodd.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f4f4355be28c01e"}},{"code_sha256_prefix":"e5551f1a55ca490a","entry":"color_img_to_gray","repo":"l-j-yuan/modest","repo_kind":"official","path":"utils/visualize.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/utils/visualize.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5551f1a55ca490a"}},{"code_sha256_prefix":"6b9a82d7ec734a28","entry":"computeMAE","repo":"l-j-yuan/modest","repo_kind":"official","path":"utils/evaluate.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/utils/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b9a82d7ec734a28"}},{"code_sha256_prefix":"6a9c1f914792dbd2","entry":"computeRMSE","repo":"l-j-yuan/modest","repo_kind":"official","path":"utils/evaluate.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/utils/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a9c1f914792dbd2"}},{"code_sha256_prefix":"3604976b3f6cf9a4","entry":"compute_depth_metrics","repo":"l-j-yuan/modest","repo_kind":"official","path":"utils/evaluate.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/utils/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3604976b3f6cf9a4"}},{"code_sha256_prefix":"2700fc830014d4b3","entry":"decompress_datapoint","repo":"l-j-yuan/modest","repo_kind":"official","path":"datasets/syntodd.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/datasets/syntodd.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2700fc830014d4b3"}},{"code_sha256_prefix":"76a7fa32cea967f6","entry":"depth_to_xyz","repo":"l-j-yuan/modest","repo_kind":"official","path":"utils/loss.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/utils/loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"76a7fa32cea967f6"}},{"code_sha256_prefix":"0e8b2c2d2d81f5d1","entry":"get_losses","repo":"l-j-yuan/modest","repo_kind":"official","path":"utils/builder.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/utils/builder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e8b2c2d2d81f5d1"}},{"code_sha256_prefix":"5defdf8118554429","entry":"get_surface_normal","repo":"l-j-yuan/modest","repo_kind":"official","path":"utils/loss.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/utils/loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5defdf8118554429"}},{"code_sha256_prefix":"65701c6ae029f468","entry":"gradient","repo":"l-j-yuan/modest","repo_kind":"official","path":"utils/loss.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/utils/loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65701c6ae029f468"}},{"code_sha256_prefix":"3e4b6c7dd7e0cfe6","entry":"multiview_collate","repo":"l-j-yuan/modest","repo_kind":"official","path":"utils/builder.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/utils/builder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e4b6c7dd7e0cfe6"}},{"code_sha256_prefix":"c8616f3565a0de3a","entry":"vis_depth","repo":"l-j-yuan/modest","repo_kind":"official","path":"utils/visualize.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/utils/visualize.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8616f3565a0de3a"}},{"code_sha256_prefix":"88bdb332e36a05f3","entry":"vis_error_map","repo":"l-j-yuan/modest","repo_kind":"official","path":"utils/visualize.py","file_url":"https://github.com/l-j-yuan/modest/blob/HEAD/utils/visualize.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88bdb332e36a05f3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}