{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/data-whisperer-efficient-data-selection-for","title":"Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning","arxiv_id":"2505.12212","date":"2025-05-18","proceeding":null,"authors":["Shaobo Wang","ZiMing Wang","Xiangqi Jin","Jize Wang","Jiajun Zhang","Kaixin Li","Zichen Wen","Zhong Li","Conghui He","Xuming Hu","Linfeng Zhang"],"abstract":"Fine-tuning large language models (LLMs) on task-specific data is essential for their effective deployment. As dataset sizes grow, efficiently selecting optimal subsets for training becomes crucial to balancing performance and computational costs. Traditional data selection methods often require fine-tuning a scoring model on the target dataset, which is time-consuming and resource-intensive, or rely on heuristics that fail to fully leverage the model's predictive capabilities. To address these challenges, we propose Data Whisperer, an efficient, training-free, attention-based method that leverages few-shot in-context learning with the model to be fine-tuned. Comprehensive evaluations were conducted on both raw and synthetic datasets across diverse tasks and models. Notably, Data Whisperer achieves superior performance compared to the full GSM8K dataset on the Llama-3-8B-Instruct model, using just 10% of the data, and outperforms existing methods with a 3.1-point improvement and a 7.4$\\times$ speedup.","url_abs":"https://arxiv.org/abs/2505.12212v1","url_pdf":"https://arxiv.org/pdf/2505.12212v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-whisperer-efficient-data-selection-for","repo_url":"https://github.com/gszfwsb/Data-Whisperer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"gsm8k","task_name":"GSM8K"},{"task_slug":"in-context-learning","task_name":"In-Context Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2505.12212","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.12212"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/gszfwsb/Data-Whisperer","reach":null}],"summary":{"ran_honours":1,"ran_violates":1,"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":7,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"4f59514e4f6d1658","entry":"calculate_rouge_l","repo":"gszfwsb/Data-Whisperer","repo_kind":"official","path":"pruning/datawhisperer_gsm_pruner.py","file_url":"https://github.com/gszfwsb/Data-Whisperer/blob/HEAD/pruning/datawhisperer_gsm_pruner.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4f59514e4f6d1658"}},{"code_sha256_prefix":"7d4e720c5eec0e6f","entry":"exact_match","repo":"gszfwsb/Data-Whisperer","repo_kind":"official","path":"pruning/datawhisperer_gsm_pruner.py","file_url":"https://github.com/gszfwsb/Data-Whisperer/blob/HEAD/pruning/datawhisperer_gsm_pruner.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7d4e720c5eec0e6f"}},{"code_sha256_prefix":"f33868b25affe264","entry":"load_json","repo":"gszfwsb/Data-Whisperer","repo_kind":"official","path":"pruning/datawhisperer_gsm_pruner.py","file_url":"https://github.com/gszfwsb/Data-Whisperer/blob/HEAD/pruning/datawhisperer_gsm_pruner.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f33868b25affe264"}},{"code_sha256_prefix":"19e30102a0907bdf","entry":"timer_decorator","repo":"gszfwsb/Data-Whisperer","repo_kind":"official","path":"pruning/datawhisperer_gsm_pruner.py","file_url":"https://github.com/gszfwsb/Data-Whisperer/blob/HEAD/pruning/datawhisperer_gsm_pruner.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19e30102a0907bdf"}},{"code_sha256_prefix":"3ece9c8f9bd7db5b","entry":"DataWhisperer_GSM_Pruner","repo":"gszfwsb/Data-Whisperer","repo_kind":"official","path":"pruning/datawhisperer_gsm_pruner.py","file_url":"https://github.com/gszfwsb/Data-Whisperer/blob/HEAD/pruning/datawhisperer_gsm_pruner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3ece9c8f9bd7db5b"}},{"code_sha256_prefix":"1e53a03eb50382c7","entry":"Pruner","repo":"gszfwsb/Data-Whisperer","repo_kind":"official","path":"pruning/datawhisperer_gsm_pruner.py","file_url":"https://github.com/gszfwsb/Data-Whisperer/blob/HEAD/pruning/datawhisperer_gsm_pruner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1e53a03eb50382c7"}},{"code_sha256_prefix":"246757f426cf0aa8","entry":"save_json","repo":"gszfwsb/Data-Whisperer","repo_kind":"official","path":"pruning/datawhisperer_gsm_pruner.py","file_url":"https://github.com/gszfwsb/Data-Whisperer/blob/HEAD/pruning/datawhisperer_gsm_pruner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"246757f426cf0aa8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}