{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cdtrans-cross-domain-transformer-for","title":"CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation","arxiv_id":"2109.06165","date":"2021-09-13","proceeding":"ICLR 2022 4","authors":["Tongkun Xu","Weihua Chen","Pichao Wang","Fan Wang","Hao Li","Rong Jin"],"abstract":"Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a labeled source domain to a different unlabeled target domain. Most existing UDA methods focus on learning domain-invariant feature representation, either from the domain level or category level, using convolution neural networks (CNNs)-based frameworks. One fundamental problem for the category level based UDA is the production of pseudo labels for samples in target domain, which are usually too noisy for accurate domain alignment, inevitably compromising the UDA performance. With the success of Transformer in various tasks, we find that the cross-attention in Transformer is robust to the noisy input pairs for better feature alignment, thus in this paper Transformer is adopted for the challenging UDA task. Specifically, to generate accurate input pairs, we design a two-way center-aware labeling algorithm to produce pseudo labels for target samples. Along with the pseudo labels, a weight-sharing triple-branch transformer framework is proposed to apply self-attention and cross-attention for source/target feature learning and source-target domain alignment, respectively. Such design explicitly enforces the framework to learn discriminative domain-specific and domain-invariant representations simultaneously. The proposed method is dubbed CDTrans (cross-domain transformer), and it provides one of the first attempts to solve UDA tasks with a pure transformer solution. Experiments show that our proposed method achieves the best performance on public UDA datasets, e.g. VisDA-2017 and DomainNet. Code and models are available at https://github.com/CDTrans/CDTrans.","url_abs":"https://arxiv.org/abs/2109.06165v4","url_pdf":"https://arxiv.org/pdf/2109.06165v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cdtrans-cross-domain-transformer-for","repo_url":"https://github.com/cdtrans/cdtrans","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"cdtrans-cross-domain-transformer-for","repo_url":"https://github.com/Darth-Kronos/Unsupervised-Domain-Adaptation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"domain-adaptation","task_name":"Domain Adaptation"},{"task_slug":"unsupervised-domain-adaptation","task_name":"Unsupervised Domain Adaptation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/domain-adaptation-on-office-31","task":"Domain Adaptation","dataset":"Office-31","model":"CDTrans","rank_in_archive_order":5,"of":40,"metrics":{"Average Accuracy":"92.6"},"uses_additional_data":false},{"leaderboard":"/sota/domain-adaptation-on-office-home","task":"Domain Adaptation","dataset":"Office-Home","model":"CDTrans (DeiT-B)","rank_in_archive_order":10,"of":29,"metrics":{"Accuracy":"80.5"},"uses_additional_data":false},{"leaderboard":"/sota/domain-adaptation-on-visda2017","task":"Domain Adaptation","dataset":"VisDA2017","model":"CDTrans","rank_in_archive_order":11,"of":28,"metrics":{"Accuracy":"88.4"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-office-home","task":"Unsupervised Domain Adaptation","dataset":"Office-Home","model":"CDTrans","rank_in_archive_order":11,"of":20,"metrics":{"Accuracy":"80.5"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2109.06165","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.06165"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Darth-Kronos/Unsupervised-Domain-Adaptation","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cdtrans/cdtrans","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"official":{"samples":8,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e3b2a83f52101f2d","entry":"normalize","repo":"cdtrans/cdtrans","repo_kind":"official","path":"loss/triplet_loss.py","file_url":"https://github.com/cdtrans/cdtrans/blob/HEAD/loss/triplet_loss.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e3b2a83f52101f2d"}},{"code_sha256_prefix":"71b4e1d375d8fd78","entry":"cosine_dist","repo":"cdtrans/cdtrans","repo_kind":"official","path":"loss/triplet_loss.py","file_url":"https://github.com/cdtrans/cdtrans/blob/HEAD/loss/triplet_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"71b4e1d375d8fd78"}},{"code_sha256_prefix":"5315f75bf5367f0a","entry":"euclidean_dist","repo":"cdtrans/cdtrans","repo_kind":"official","path":"loss/triplet_loss.py","file_url":"https://github.com/cdtrans/cdtrans/blob/HEAD/loss/triplet_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5315f75bf5367f0a"}},{"code_sha256_prefix":"2b5e77e0deb72f1b","entry":"make_loss","repo":"cdtrans/cdtrans","repo_kind":"official","path":"loss/make_loss.py","file_url":"https://github.com/cdtrans/cdtrans/blob/HEAD/loss/make_loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b5e77e0deb72f1b"}},{"code_sha256_prefix":"5f1101145f896208","entry":"read_image","repo":"cdtrans/cdtrans","repo_kind":"official","path":"datasets/bases.py","file_url":"https://github.com/cdtrans/cdtrans/blob/HEAD/datasets/bases.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f1101145f896208"}},{"code_sha256_prefix":"ec44d1cea891fdb3","entry":"shear_x","repo":"cdtrans/cdtrans","repo_kind":"official","path":"datasets/autoaugment.py","file_url":"https://github.com/cdtrans/cdtrans/blob/HEAD/datasets/autoaugment.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec44d1cea891fdb3"}},{"code_sha256_prefix":"368f32c4172985b4","entry":"shear_y","repo":"cdtrans/cdtrans","repo_kind":"official","path":"datasets/autoaugment.py","file_url":"https://github.com/cdtrans/cdtrans/blob/HEAD/datasets/autoaugment.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"368f32c4172985b4"}},{"code_sha256_prefix":"aa897f695e05da40","entry":"translate_x_rel","repo":"cdtrans/cdtrans","repo_kind":"official","path":"datasets/autoaugment.py","file_url":"https://github.com/cdtrans/cdtrans/blob/HEAD/datasets/autoaugment.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa897f695e05da40"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}