{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/clipartt-light-weight-adaptation-of-clip-to","title":"CLIPArTT: Adaptation of CLIP to New Domains at Test Time","arxiv_id":"2405.00754","date":"2024-05-01","proceeding":null,"authors":["Gustavo Adolfo Vargas Hakim","David Osowiechi","Mehrdad Noori","Milad Cheraghalikhani","Ali Bahri","Moslem Yazdanpanah","Ismail Ben Ayed","Christian Desrosiers"],"abstract":"Pre-trained vision-language models (VLMs), exemplified by CLIP, demonstrate remarkable adaptability across zero-shot classification tasks without additional training. However, their performance diminishes in the presence of domain shifts. In this study, we introduce CLIP Adaptation duRing Test-Time (CLIPArTT), a fully test-time adaptation (TTA) approach for CLIP, which involves automatic text prompts construction during inference for their use as text supervision. Our method employs a unique, minimally invasive text prompt tuning process, wherein multiple predicted classes are aggregated into a single new text prompt, used as \\emph{pseudo label} to re-classify inputs in a transductive manner. Additionally, we pioneer the standardization of TTA benchmarks (e.g., TENT) in the realm of VLMs. Our findings demonstrate that, without requiring additional transformations nor new trainable modules, CLIPArTT enhances performance dynamically across non-corrupted datasets such as CIFAR-100, corrupted datasets like CIFAR-100-C and ImageNet-C, alongside synthetic datasets such as VisDA-C. This research underscores the potential for improving VLMs' adaptability through novel test-time strategies, offering insights for robust performance across varied datasets and environments. The code can be found at: https://github.com/dosowiechi/CLIPArTT.git","url_abs":"https://arxiv.org/abs/2405.00754v2","url_pdf":"https://arxiv.org/pdf/2405.00754v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"clipartt-light-weight-adaptation-of-clip-to","repo_url":"https://github.com/dosowiechi/clipartt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"pseudo-label","task_name":"Pseudo Label"},{"task_slug":"test-time-adaptation","task_name":"Test-time Adaptation"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"},{"task_slug":null,"task_name":"zero-shot-classification"}],"methods":[{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2405.00754","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.00754"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dosowiechi/clipartt","reach":{"status":"ok"}}],"summary":{"ran_fixture":2,"ran_draft_wrong":3,"ran":6,"unverified":4},"by_repo_kind":{"official":{"samples":15,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"b0f936d4d6ae3b8c","entry":"accuracy","repo":"dosowiechi/clipartt","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/utils/utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0f936d4d6ae3b8c"}},{"code_sha256_prefix":"98f385d847636a3e","entry":"basic_clean","repo":"dosowiechi/clipartt","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"98f385d847636a3e"}},{"code_sha256_prefix":"d858deb1b3a2d3ba","entry":"cross_entropy","repo":"dosowiechi/clipartt","repo_kind":"official","path":"models/tent.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/models/tent.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d858deb1b3a2d3ba"}},{"code_sha256_prefix":"473a24407468b75e","entry":"entropy_energy","repo":"dosowiechi/clipartt","repo_kind":"official","path":"models/lame.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/models/lame.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"473a24407468b75e"}},{"code_sha256_prefix":"d919ae32e5e4e616","entry":"get_pairs","repo":"dosowiechi/clipartt","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d919ae32e5e4e616"}},{"code_sha256_prefix":"e7236889a0d75f77","entry":"getprompt","repo":"dosowiechi/clipartt","repo_kind":"official","path":"models/tent.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/models/tent.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e7236889a0d75f77"}},{"code_sha256_prefix":"cc8f4613f74df21b","entry":"prepare_val_data","repo":"dosowiechi/clipartt","repo_kind":"official","path":"utils/prepare_dataset.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/utils/prepare_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cc8f4613f74df21b"}},{"code_sha256_prefix":"b3a7110b4cd7c335","entry":"setup_optimizer","repo":"dosowiechi/clipartt","repo_kind":"official","path":"adapt.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/adapt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3a7110b4cd7c335"}},{"code_sha256_prefix":"3078648c75e3e1e3","entry":"softmax_entropy","repo":"dosowiechi/clipartt","repo_kind":"official","path":"models/tent.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/models/tent.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3078648c75e3e1e3"}},{"code_sha256_prefix":"077971bc206bae40","entry":"test_batch","repo":"dosowiechi/clipartt","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"077971bc206bae40"}},{"code_sha256_prefix":"9542161e9640b858","entry":"whitespace_clean","repo":"dosowiechi/clipartt","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9542161e9640b858"}},{"code_sha256_prefix":"ad2900d99122fdff","entry":"adapt_batch","repo":"dosowiechi/clipartt","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ad2900d99122fdff"}},{"code_sha256_prefix":"38c09396452c23a0","entry":"build_model","repo":"dosowiechi/clipartt","repo_kind":"official","path":"clip/model.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/clip/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"38c09396452c23a0"}},{"code_sha256_prefix":"d2597d43cbd2fbca","entry":"laplacian_optimization","repo":"dosowiechi/clipartt","repo_kind":"official","path":"models/lame.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/models/lame.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d2597d43cbd2fbca"}},{"code_sha256_prefix":"2da7ec0975be872a","entry":"load","repo":"dosowiechi/clipartt","repo_kind":"official","path":"clip/clip.py","file_url":"https://github.com/dosowiechi/clipartt/blob/HEAD/clip/clip.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2da7ec0975be872a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}