{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/safe-self-refinement-for-transformer-based","title":"Safe Self-Refinement for Transformer-based Domain Adaptation","arxiv_id":"2204.07683","date":"2022-04-16","proceeding":"CVPR 2022 1","authors":["Tao Sun","Cheng Lu","Tianshuo Zhang","Haibin Ling"],"abstract":"Unsupervised Domain Adaptation (UDA) aims to leverage a label-rich source domain to solve tasks on a related unlabeled target domain. It is a challenging problem especially when a large domain gap lies between the source and target domains. In this paper we propose a novel solution named SSRT (Safe Self-Refinement for Transformer-based domain adaptation), which brings improvement from two aspects. First, encouraged by the success of vision transformers in various vision tasks, we arm SSRT with a transformer backbone. We find that the combination of vision transformer with simple adversarial adaptation surpasses best reported Convolutional Neural Network (CNN)-based results on the challenging DomainNet benchmark, showing its strong transferable feature representation. Second, to reduce the risk of model collapse and improve the effectiveness of knowledge transfer between domains with large gaps, we propose a Safe Self-Refinement strategy. Specifically, SSRT utilizes predictions of perturbed target domain data to refine the model. Since the model capacity of vision transformer is large and predictions in such challenging tasks can be noisy, a safe training mechanism is designed to adaptively adjust learning configuration. Extensive evaluations are conducted on several widely tested UDA benchmarks and SSRT achieves consistently the best performances, including 85.43% on Office-Home, 88.76% on VisDA-2017 and 45.2% on DomainNet.","url_abs":"https://arxiv.org/abs/2204.07683v1","url_pdf":"https://arxiv.org/pdf/2204.07683v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"safe-self-refinement-for-transformer-based","repo_url":"https://github.com/tsun/ssrt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"domain-adaptation","task_name":"Domain Adaptation"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"unsupervised-domain-adaptation","task_name":"Unsupervised Domain Adaptation"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/domain-adaptation-on-office-31","task":"Domain Adaptation","dataset":"Office-31","model":"SSRT-B (ours)","rank_in_archive_order":4,"of":40,"metrics":{"Average Accuracy":"93.5"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-office-home","task":"Unsupervised Domain Adaptation","dataset":"Office-Home","model":"SSRT-B","rank_in_archive_order":7,"of":20,"metrics":{"Accuracy":"85.43"},"uses_additional_data":false},{"leaderboard":"/sota/unsupervised-domain-adaptation-on-visda2017","task":"Unsupervised Domain Adaptation","dataset":"VisDA2017","model":"SSRT-B (ours)","rank_in_archive_order":8,"of":13,"metrics":{"Accuracy":"88.76"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2204.07683","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2204.07683"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tsun/ssrt","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/tsun/SSRT","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"ran_fixture":4,"ran_draft_wrong":2,"unverified":10},"by_repo_kind":{"official":{"samples":18,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"3d0a0d378c8dbc24","entry":"GradientReverseFunction","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d0a0d378c8dbc24"}},{"code_sha256_prefix":"22050c4d1bca63e0","entry":"PatchEmbed","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"22050c4d1bca63e0"}},{"code_sha256_prefix":"ec88df6f428e3f61","entry":"WarmStartGradientReverseLayer","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec88df6f428e3f61"}},{"code_sha256_prefix":"fa2dd3be10cf9804","entry":"_no_grad_trunc_normal_","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fa2dd3be10cf9804"}},{"code_sha256_prefix":"553ee0765abad219","entry":"_ntuple","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"553ee0765abad219"}},{"code_sha256_prefix":"552d614343c6f5b0","entry":"adapt_input_conv","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"552d614343c6f5b0"}},{"code_sha256_prefix":"75e11c702145c5c3","entry":"named_apply","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75e11c702145c5c3"}},{"code_sha256_prefix":"b01c11b0a035cd40","entry":"resize_pos_embed","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b01c11b0a035cd40"}},{"code_sha256_prefix":"02566da69866c48c","entry":"trunc_normal_","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02566da69866c48c"}},{"code_sha256_prefix":"bbf9328517c62d43","entry":"Attention","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bbf9328517c62d43"}},{"code_sha256_prefix":"eb8b342e174c85c9","entry":"Block","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb8b342e174c85c9"}},{"code_sha256_prefix":"be99cca89b88af5e","entry":"SSRT","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"be99cca89b88af5e"}},{"code_sha256_prefix":"0904466d7926c365","entry":"SSRTNet","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0904466d7926c365"}},{"code_sha256_prefix":"17aa99cd0717e1d0","entry":"VT","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17aa99cd0717e1d0"}},{"code_sha256_prefix":"2a0271883efece1e","entry":"VisionTransformer","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a0271883efece1e"}},{"code_sha256_prefix":"ddbeb989cf002f02","entry":"_init_vit_weights","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ddbeb989cf002f02"}},{"code_sha256_prefix":"3c95bc9517c87684","entry":"_load_weights","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c95bc9517c87684"}},{"code_sha256_prefix":"50805d05a18a031e","entry":"lecun_normal_","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"50805d05a18a031e"}},{"code_sha256_prefix":"1a6538cd23fa0b1a","entry":"variance_scaling_","repo":"tsun/SSRT","repo_kind":"official","path":"model/SSRT.py","file_url":"https://github.com/tsun/SSRT/blob/HEAD/model/SSRT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a6538cd23fa0b1a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}