{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficiently-learning-at-test-time-active-fine","title":"Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs","arxiv_id":"2410.08020","date":"2024-10-10","proceeding":null,"authors":["Jonas Hübotter","Sascha Bongni","Ido Hakimi","Andreas Krause"],"abstract":"Recent efforts in fine-tuning language models often rely on automatic data selection, commonly using Nearest Neighbors retrieval from large datasets. However, we theoretically show that this approach tends to select redundant data, limiting its effectiveness or even hurting performance. To address this, we introduce SIFT, a data selection algorithm designed to reduce uncertainty about the model's response given a prompt, which unifies ideas from retrieval and active learning. Whereas Nearest Neighbor retrieval typically fails in the presence of information duplication, SIFT accounts for information duplication and optimizes the overall information gain of the selected examples. We focus our evaluations on fine-tuning at test-time for prompt-specific language modeling on the Pile dataset, and show that SIFT consistently outperforms Nearest Neighbor retrieval, with minimal computational overhead. Moreover, we show that our uncertainty estimates can predict the performance gain of test-time fine-tuning, and use this to develop an adaptive algorithm that invests test-time compute proportional to realized performance gains. We provide the $\\texttt{activeft}$ (Active Fine-Tuning) library which can be used as a drop-in replacement for Nearest Neighbor retrieval.","url_abs":"https://arxiv.org/abs/2410.08020v2","url_pdf":"https://arxiv.org/pdf/2410.08020v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficiently-learning-at-test-time-active-fine","repo_url":"https://github.com/jonhue/activeft","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"active-learning","task_name":"Active Learning"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[{"method_slug":"focus","method_name":"Focus"},{"method_slug":null,"method_name":"Library"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Test-Time Fine-Tuning with SIFT + Llama-3.2 (3B)","rank_in_archive_order":1,"of":39,"metrics":{"Bits per byte":"0.557"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Test-Time Fine-Tuning with SIFT + Phi-3 (3.8B)","rank_in_archive_order":2,"of":39,"metrics":{"Bits per byte":"0.595"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Test-Time Fine-Tuning with SIFT + Llama-3.2 (1B)","rank_in_archive_order":3,"of":39,"metrics":{"Bits per byte":"0.606"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Gemma-2 27B","rank_in_archive_order":4,"of":39,"metrics":{"Bits per byte":"0.629"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Llama-3.2 3B","rank_in_archive_order":6,"of":39,"metrics":{"Bits per byte":"0.640"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Phi-3 14B","rank_in_archive_order":8,"of":39,"metrics":{"Bits per byte":"0.651"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Gemma-2 9B","rank_in_archive_order":9,"of":39,"metrics":{"Bits per byte":"0.670"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Phi-3 7B","rank_in_archive_order":10,"of":39,"metrics":{"Bits per byte":"0.678"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Phi-3 3.8B","rank_in_archive_order":11,"of":39,"metrics":{"Bits per byte":"0.679"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Llama-3.2 1B","rank_in_archive_order":12,"of":39,"metrics":{"Bits per byte":"0.697"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Gemma-2 2B","rank_in_archive_order":14,"of":39,"metrics":{"Bits per byte":"0.721"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Llama-3.2-Instruct 3B","rank_in_archive_order":15,"of":39,"metrics":{"Bits per byte":"0.737"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Test-Time Fine-Tuning with SIFT + GPT-2 (774M)","rank_in_archive_order":17,"of":39,"metrics":{"Bits per byte":"0.762"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Llama-3.2-Instruct 1B","rank_in_archive_order":19,"of":39,"metrics":{"Bits per byte":"0.807"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-the-pile","task":"Language Modelling","dataset":"The Pile","model":"Test-Time Fine-Tuning with SIFT + GPT-2 (124M)","rank_in_archive_order":21,"of":39,"metrics":{"Bits per byte":"0.862"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.08020","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.08020"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jonhue/activeft","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6f4236e0cc55fd13","entry":"mini_batch_wrapper","repo":"jonhue/activeft","repo_kind":"official","path":"activeft/utils.py","file_url":"https://github.com/jonhue/activeft/blob/HEAD/activeft/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f4236e0cc55fd13"}},{"code_sha256_prefix":"c67783cbc48b7cf3","entry":"mini_batch_wrapper_non_cat","repo":"jonhue/activeft","repo_kind":"official","path":"activeft/utils.py","file_url":"https://github.com/jonhue/activeft/blob/HEAD/activeft/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c67783cbc48b7cf3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}