{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-relate-depth-and-semantics-for","title":"Learning to Relate Depth and Semantics for Unsupervised Domain Adaptation","arxiv_id":"2105.07830","date":"2021-05-17","proceeding":"CVPR 2021 1","authors":["Suman Saha","Anton Obukhov","Danda Pani Paudel","Menelaos Kanakis","Yuhua Chen","Stamatios Georgoulis","Luc van Gool"],"abstract":"We present an approach for encoding visual task relationships to improve model performance in an Unsupervised Domain Adaptation (UDA) setting. Semantic segmentation and monocular depth estimation are shown to be complementary tasks; in a multi-task learning setting, a proper encoding of their relationships can further improve performance on both tasks. Motivated by this observation, we propose a novel Cross-Task Relation Layer (CTRL), which encodes task dependencies between the semantic and depth predictions. To capture the cross-task relationships, we propose a neural network architecture that contains task-specific and cross-task refinement heads. Furthermore, we propose an Iterative Self-Learning (ISL) training scheme, which exploits semantic pseudo-labels to provide extra supervision on the target domain. We experimentally observe improvements in both tasks' performance because the complementary information present in these tasks is better captured. Specifically, we show that: (1) our approach improves performance on all tasks when they are complementary and mutually dependent; (2) the CTRL helps to improve both semantic segmentation and depth estimation tasks performance in the challenging UDA setting; (3) the proposed ISL training scheme further improves the semantic segmentation performance. The implementation is available at https://github.com/susaha/ctrl-uda.","url_abs":"https://arxiv.org/abs/2105.07830v2","url_pdf":"https://arxiv.org/pdf/2105.07830v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-relate-depth-and-semantics-for","repo_url":"https://github.com/susaha/ctrl-uda","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"domain-adaptation","task_name":"Domain Adaptation"},{"task_slug":"monocular-depth-estimation","task_name":"Monocular Depth Estimation"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"self-learning","task_name":"Self-Learning"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"unsupervised-domain-adaptation","task_name":"Unsupervised Domain Adaptation"}],"methods":[{"method_slug":"adagrad","method_name":"AdaGrad"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"ctrl","method_name":"CTRL"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gradient-clipping","method_name":"Gradient Clipping"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup","method_name":"Linear Warmup"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"self-learning","method_name":"Self-Learning"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2105.07830","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.07830"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/susaha/ctrl-uda","reach":null}],"summary":{"ran":2,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"6cc02981a5cd9b87","entry":"DepthProb","repo":"susaha/ctrl-uda","repo_kind":"official","path":"ctrl/model/cross_task_relation.py","file_url":"https://github.com/susaha/ctrl-uda/blob/HEAD/ctrl/model/cross_task_relation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6cc02981a5cd9b87"}},{"code_sha256_prefix":"6e6a9b7c2a83a3bc","entry":"Prob2Entropy","repo":"susaha/ctrl-uda","repo_kind":"official","path":"ctrl/model/cross_task_relation.py","file_url":"https://github.com/susaha/ctrl-uda/blob/HEAD/ctrl/model/cross_task_relation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e6a9b7c2a83a3bc"}},{"code_sha256_prefix":"1a561efa18b716f4","entry":"get_bins","repo":"susaha/ctrl-uda","repo_kind":"official","path":"ctrl/model/cross_task_relation.py","file_url":"https://github.com/susaha/ctrl-uda/blob/HEAD/ctrl/model/cross_task_relation.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1a561efa18b716f4"}},{"code_sha256_prefix":"160fa2b32affa0e9","entry":"CTRL","repo":"susaha/ctrl-uda","repo_kind":"official","path":"ctrl/model/cross_task_relation.py","file_url":"https://github.com/susaha/ctrl-uda/blob/HEAD/ctrl/model/cross_task_relation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"160fa2b32affa0e9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}