{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/frustratingly-easy-test-time-adaptation-of","title":"Frustratingly Easy Test-Time Adaptation of Vision-Language Models","arxiv_id":"2405.18330","date":"2024-05-28","proceeding":null,"authors":["Matteo Farina","Gianni Franchi","Giovanni Iacca","Massimiliano Mancini","Elisa Ricci"],"abstract":"Vision-Language Models seamlessly discriminate among arbitrary semantic categories, yet they still suffer from poor generalization when presented with challenging examples. For this reason, Episodic Test-Time Adaptation (TTA) strategies have recently emerged as powerful techniques to adapt VLMs in the presence of a single unlabeled image. The recent literature on TTA is dominated by the paradigm of prompt tuning by Marginal Entropy Minimization, which, relying on online backpropagation, inevitably slows down inference while increasing memory. In this work, we theoretically investigate the properties of this approach and unveil that a surprisingly strong TTA method lies dormant and hidden within it. We term this approach ZERO (TTA with \"zero\" temperature), whose design is both incredibly effective and frustratingly simple: augment N times, predict, retain the most confident predictions, and marginalize after setting the Softmax temperature to zero. Remarkably, ZERO requires a single batched forward pass through the vision encoder only and no backward passes. We thoroughly evaluate our approach following the experimental protocol established in the literature and show that ZERO largely surpasses or compares favorably w.r.t. the state-of-the-art while being almost 10x faster and 13x more memory-friendly than standard Test-Time Prompt Tuning. Thanks to its simplicity and comparatively negligible computation, ZERO can serve as a strong baseline for future work in this field. The code is available at https://github.com/FarinaMatteo/zero.","url_abs":"https://arxiv.org/abs/2405.18330v2","url_pdf":"https://arxiv.org/pdf/2405.18330v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"frustratingly-easy-test-time-adaptation-of","repo_url":"https://github.com/farinamatteo/zero","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"test-time-adaptation","task_name":"Test-time Adaptation"}],"methods":[{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2405.18330","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.18330"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/farinamatteo/zero","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"ran":1,"unverified":4},"by_repo_kind":{"official":{"samples":8,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"98f385d847636a3e","entry":"basic_clean","repo":"farinamatteo/zero","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/farinamatteo/zero/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98f385d847636a3e"}},{"code_sha256_prefix":"23f0fc9cbefff039","entry":"confidence_filter","repo":"farinamatteo/zero","repo_kind":"official","path":"ttas/base.py","file_url":"https://github.com/farinamatteo/zero/blob/HEAD/ttas/base.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"23f0fc9cbefff039"}},{"code_sha256_prefix":"d919ae32e5e4e616","entry":"get_pairs","repo":"farinamatteo/zero","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/farinamatteo/zero/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d919ae32e5e4e616"}},{"code_sha256_prefix":"9542161e9640b858","entry":"whitespace_clean","repo":"farinamatteo/zero","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/farinamatteo/zero/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9542161e9640b858"}},{"code_sha256_prefix":"48ac2bd4dcbfb717","entry":"build_model","repo":"farinamatteo/zero","repo_kind":"official","path":"maple_clip/model.py","file_url":"https://github.com/farinamatteo/zero/blob/HEAD/maple_clip/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48ac2bd4dcbfb717"}},{"code_sha256_prefix":"aa56b568a90f8516","entry":"build_model","repo":"farinamatteo/zero","repo_kind":"official","path":"clip/model.py","file_url":"https://github.com/farinamatteo/zero/blob/HEAD/clip/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa56b568a90f8516"}},{"code_sha256_prefix":"fbf8c0143d9c48e3","entry":"load","repo":"farinamatteo/zero","repo_kind":"official","path":"maple_clip/clip.py","file_url":"https://github.com/farinamatteo/zero/blob/HEAD/maple_clip/clip.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fbf8c0143d9c48e3"}},{"code_sha256_prefix":"88d0e8acec4aead0","entry":"load","repo":"farinamatteo/zero","repo_kind":"official","path":"clip/clip.py","file_url":"https://github.com/farinamatteo/zero/blob/HEAD/clip/clip.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88d0e8acec4aead0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}