{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/test-time-adaptation-with-binary-feedback","title":"Test-Time Adaptation with Binary Feedback","arxiv_id":"2505.18514","date":"2025-05-24","proceeding":null,"authors":["Taeckyung Lee","Sorn Chottananurak","Junsu Kim","Jinwoo Shin","Taesik Gong","Sung-Ju Lee"],"abstract":"Deep learning models perform poorly when domain shifts exist between training and test data. Test-time adaptation (TTA) is a paradigm to mitigate this issue by adapting pre-trained models using only unlabeled test samples. However, existing TTA methods can fail under severe domain shifts, while recent active TTA approaches requiring full-class labels are impractical due to high labeling costs. To address this issue, we introduce a new setting of TTA with binary feedback. This setting uses a few binary feedback inputs from annotators to indicate whether model predictions are correct, thereby significantly reducing the labeling burden of annotators. Under the setting, we propose BiTTA, a novel dual-path optimization framework that leverages reinforcement learning to balance binary feedback-guided adaptation on uncertain samples with agreement-based self-adaptation on confident predictions. Experiments show BiTTA achieves 13.3%p accuracy improvements over state-of-the-art baselines, demonstrating its effectiveness in handling severe distribution shifts with minimal labeling effort. The source code is available at https://github.com/taeckyung/BiTTA.","url_abs":"https://arxiv.org/abs/2505.18514v1","url_pdf":"https://arxiv.org/pdf/2505.18514v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"test-time-adaptation-with-binary-feedback","repo_url":"https://github.com/taeckyung/bitta","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"test-time-adaptation","task_name":"Test-time Adaptation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2505.18514","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.18514"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/taeckyung/BiTTA","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/taeckyung/bitta","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"ran_draft_wrong":2,"ran_honours":1,"ran_fixture":1,"unverified":8},"by_repo_kind":{"official":{"samples":13,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7d3f76502e1e286c","entry":"get_dirnames","repo":"taeckyung/BiTTA","repo_kind":"official","path":"process_cifar.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/process_cifar.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d3f76502e1e286c"}},{"code_sha256_prefix":"8e952e7f9820216a","entry":"load_data","repo":"taeckyung/BiTTA","repo_kind":"official","path":"process_cifar.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/process_cifar.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e952e7f9820216a"}},{"code_sha256_prefix":"d41e65573feeba36","entry":"load_image","repo":"taeckyung/BiTTA","repo_kind":"official","path":"data_loader/DOMAINNET126Dataset.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/data_loader/DOMAINNET126Dataset.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d41e65573feeba36"}},{"code_sha256_prefix":"92bc8db96a48bcb0","entry":"reshape","repo":"taeckyung/BiTTA","repo_kind":"official","path":"process_cifar.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/process_cifar.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"92bc8db96a48bcb0"}},{"code_sha256_prefix":"c631ff42fed59f21","entry":"stage_path","repo":"taeckyung/BiTTA","repo_kind":"official","path":"download_pacs.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/download_pacs.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c631ff42fed59f21"}},{"code_sha256_prefix":"dfdeebd2ab319782","entry":"ResNet18","repo":"taeckyung/BiTTA","repo_kind":"official","path":"models/ResNet.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/models/ResNet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dfdeebd2ab319782"}},{"code_sha256_prefix":"0db2a6b94d888b0e","entry":"ResNet34","repo":"taeckyung/BiTTA","repo_kind":"official","path":"models/ResNet.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/models/ResNet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0db2a6b94d888b0e"}},{"code_sha256_prefix":"e4bc08da680e42e6","entry":"ResNet50","repo":"taeckyung/BiTTA","repo_kind":"official","path":"models/ResNet.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/models/ResNet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4bc08da680e42e6"}},{"code_sha256_prefix":"b5d3af53cad842fb","entry":"extract_arguments_from_file","repo":"taeckyung/BiTTA","repo_kind":"official","path":"compare_arguments.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/compare_arguments.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b5d3af53cad842fb"}},{"code_sha256_prefix":"6510eca45388fb0f","entry":"extract_default","repo":"taeckyung/BiTTA","repo_kind":"official","path":"compare_arguments.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/compare_arguments.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6510eca45388fb0f"}},{"code_sha256_prefix":"3da48796a1ca07b5","entry":"extract_type","repo":"taeckyung/BiTTA","repo_kind":"official","path":"compare_arguments.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/compare_arguments.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3da48796a1ca07b5"}},{"code_sha256_prefix":"63b9f4d0c7cc2cc4","entry":"get_avg_online_acc","repo":"taeckyung/BiTTA","repo_kind":"official","path":"print_acc.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/print_acc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63b9f4d0c7cc2cc4"}},{"code_sha256_prefix":"1ba2c7237a527239","entry":"process_path","repo":"taeckyung/BiTTA","repo_kind":"official","path":"print_acc.py","file_url":"https://github.com/taeckyung/BiTTA/blob/HEAD/print_acc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ba2c7237a527239"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}