{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/what-is-your-data-worth-to-gpt-llm-scale-data","title":"What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions","arxiv_id":"2405.13954","date":"2024-05-22","proceeding":null,"authors":["Sang Keun Choe","Hwijeen Ahn","Juhan Bae","Kewen Zhao","Minsoo Kang","Youngseog Chung","Adithya Pratapa","Willie Neiswanger","Emma Strubell","Teruko Mitamura","Jeff Schneider","Eduard Hovy","Roger Grosse","Eric Xing"],"abstract":"Large language models (LLMs) are trained on a vast amount of human-written data, but data providers often remain uncredited. In response to this issue, data valuation (or data attribution), which quantifies the contribution or value of each data to the model output, has been discussed as a potential solution. Nevertheless, applying existing data valuation methods to recent LLMs and their vast training datasets has been largely limited by prohibitive compute and memory costs. In this work, we focus on influence functions, a popular gradient-based data valuation method, and significantly improve its scalability with an efficient gradient projection strategy called LoGra that leverages the gradient structure in backpropagation. We then provide a theoretical motivation of gradient projection approaches to influence functions to promote trust in the data valuation process. Lastly, we lower the barrier to implementing data valuation systems by introducing LogIX, a software package that can transform existing training code into data valuation code with minimal effort. In our data valuation experiments, LoGra achieves competitive accuracy against more expensive baselines while showing up to 6,500x improvement in throughput and 5x reduction in GPU memory usage when applied to Llama3-8B-Instruct and the 1B-token dataset.","url_abs":"https://arxiv.org/abs/2405.13954v1","url_pdf":"https://arxiv.org/pdf/2405.13954v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"what-is-your-data-worth-to-gpt-llm-scale-data","repo_url":"https://github.com/logix-project/logix","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"data-valuation","task_name":"Data Valuation"},{"task_slug":null,"task_name":"GPU"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.13954","atlas_url":"https://app.syntology.ai/?focus=2405.13954","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.13954"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/logix-project/logix","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":3,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6048bc83e834e377","entry":"cross_dot_product","repo":"logix-project/logix","repo_kind":"official","path":"logix/analysis/influence_function_utils.py","file_url":"https://github.com/logix-project/logix/blob/HEAD/logix/analysis/influence_function_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6048bc83e834e377"}},{"code_sha256_prefix":"d3e53200c9f0c7cb","entry":"get_rank","repo":"logix-project/logix","repo_kind":"official","path":"logix/utils.py","file_url":"https://github.com/logix-project/logix/blob/HEAD/logix/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d3e53200c9f0c7cb"}},{"code_sha256_prefix":"96b9c8cb4aa4b385","entry":"to_numpy","repo":"logix-project/logix","repo_kind":"official","path":"logix/utils.py","file_url":"https://github.com/logix-project/logix/blob/HEAD/logix/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"96b9c8cb4aa4b385"}},{"code_sha256_prefix":"fa24fec8e4cbb089","entry":"get_repr_dim","repo":"logix-project/logix","repo_kind":"official","path":"logix/utils.py","file_url":"https://github.com/logix-project/logix/blob/HEAD/logix/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fa24fec8e4cbb089"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}