{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/helpful-or-harmful-data-fine-tuning-free","title":"Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictions","arxiv_id":"2406.04606","date":"2024-06-07","proceeding":null,"authors":["Jingtan Wang","Xiaoqiang Lin","Rui Qiao","Chuan-Sheng Foo","Bryan Kian Hsiang Low"],"abstract":"The increasing complexity of foundational models underscores the necessity for explainability, particularly for fine-tuning, the most widely used training method for adapting models to downstream tasks. Instance attribution, one type of explanation, attributes the model prediction to each training example by an instance score. However, the robustness of instance scores, specifically towards dataset resampling, has been overlooked. To bridge this gap, we propose a notion of robustness on the sign of the instance score. We theoretically and empirically demonstrate that the popular leave-one-out-based methods lack robustness, while the Shapley value behaves significantly better, but at a higher computational cost. Accordingly, we introduce an efficient fine-tuning-free approximation of the Shapley value (FreeShap) for instance attribution based on the neural tangent kernel. We empirically demonstrate that FreeShap outperforms other methods for instance attribution and other data-centric applications such as data removal, data selection, and wrong label detection, and further generalize our scale to large language models (LLMs). Our code is available at https://github.com/JTWang2000/FreeShap.","url_abs":"https://arxiv.org/abs/2406.04606v1","url_pdf":"https://arxiv.org/pdf/2406.04606v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"helpful-or-harmful-data-fine-tuning-free","repo_url":"https://github.com/jtwang2000/freeshap","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.04606","atlas_url":"https://app.syntology.ai/?focus=2406.04606","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.04606"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/JTWang2000/FreeShap","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jtwang2000/freeshap","reach":{"status":"ok"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"fb9020626aee2101","entry":"powerset","repo":"JTWang2000/FreeShap","repo_kind":"official","path":"vinfo/dvutils/utils.py","file_url":"https://github.com/JTWang2000/FreeShap/blob/HEAD/vinfo/dvutils/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fb9020626aee2101"}},{"code_sha256_prefix":"87695b224d461a5e","entry":"slice_data","repo":"JTWang2000/FreeShap","repo_kind":"official","path":"vinfo/entks/ntk.py","file_url":"https://github.com/JTWang2000/FreeShap/blob/HEAD/vinfo/entks/ntk.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"87695b224d461a5e"}},{"code_sha256_prefix":"7f45ae25efe8c0fa","entry":"softmax","repo":"JTWang2000/FreeShap","repo_kind":"official","path":"vinfo/dvutils/utils.py","file_url":"https://github.com/JTWang2000/FreeShap/blob/HEAD/vinfo/dvutils/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7f45ae25efe8c0fa"}},{"code_sha256_prefix":"69503969b65275a2","entry":"to_cuda","repo":"JTWang2000/FreeShap","repo_kind":"official","path":"vinfo/entks/ntk.py","file_url":"https://github.com/JTWang2000/FreeShap/blob/HEAD/vinfo/entks/ntk.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"69503969b65275a2"}},{"code_sha256_prefix":"687e768b80894e09","entry":"tqdm_joblib","repo":"JTWang2000/FreeShap","repo_kind":"official","path":"vinfo/utils.py","file_url":"https://github.com/JTWang2000/FreeShap/blob/HEAD/vinfo/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"687e768b80894e09"}},{"code_sha256_prefix":"b2e76acdef785c7b","entry":"worker","repo":"JTWang2000/FreeShap","repo_kind":"official","path":"vinfo/dvutils/Data_Shapley.py","file_url":"https://github.com/JTWang2000/FreeShap/blob/HEAD/vinfo/dvutils/Data_Shapley.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b2e76acdef785c7b"}},{"code_sha256_prefix":"27cd0c00003f7ff6","entry":"compute_ntk","repo":"JTWang2000/FreeShap","repo_kind":"official","path":"vinfo/entks/ntk.py","file_url":"https://github.com/JTWang2000/FreeShap/blob/HEAD/vinfo/entks/ntk.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27cd0c00003f7ff6"}},{"code_sha256_prefix":"5dc3d84f2410ba41","entry":"create_exp_dir","repo":"JTWang2000/FreeShap","repo_kind":"official","path":"vinfo/dvutils/utils.py","file_url":"https://github.com/JTWang2000/FreeShap/blob/HEAD/vinfo/dvutils/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5dc3d84f2410ba41"}},{"code_sha256_prefix":"5faef15ca416cd0d","entry":"get_conversion_dict","repo":"JTWang2000/FreeShap","repo_kind":"official","path":"vinfo/utils.py","file_url":"https://github.com/JTWang2000/FreeShap/blob/HEAD/vinfo/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5faef15ca416cd0d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}