{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adversarial-attacks-on-data-attribution","title":"Adversarial Attacks on Data Attribution","arxiv_id":"2409.05657","date":"2024-09-09","proceeding":null,"authors":["Xinhe Wang","Pingbang Hu","Junwei Deng","Jiaqi W. Ma"],"abstract":"Data attribution aims to quantify the contribution of individual training data points to the outputs of an AI model, which has been used to measure the value of training data and compensate data providers. Given the impact on financial decisions and compensation mechanisms, a critical question arises concerning the adversarial robustness of data attribution methods. However, there has been little to no systematic research addressing this issue. In this work, we aim to bridge this gap by detailing a threat model with clear assumptions about the adversary's goal and capabilities and proposing principled adversarial attack methods on data attribution. We present two methods, Shadow Attack and Outlier Attack, which generate manipulated datasets to inflate the compensation adversarially. The Shadow Attack leverages knowledge about the data distribution in the AI applications, and derives adversarial perturbations through \"shadow training\", a technique commonly used in membership inference attacks. In contrast, the Outlier Attack does not assume any knowledge about the data distribution and relies solely on black-box queries to the target model's predictions. It exploits an inductive bias present in many data attribution methods - outlier data points are more likely to be influential - and employs adversarial examples to generate manipulated datasets. Empirically, in image classification and text generation tasks, the Shadow Attack can inflate the data-attribution-based compensation by at least 200%, while the Outlier Attack achieves compensation inflation ranging from 185% to as much as 643%. Our implementation is ready at https://github.com/TRAIS-Lab/adversarial-attack-data-attribution.","url_abs":"https://arxiv.org/abs/2409.05657v4","url_pdf":"https://arxiv.org/pdf/2409.05657v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adversarial-attacks-on-data-attribution","repo_url":"https://github.com/trais-lab/adversarial-attack-data-attribution","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"adversarial-attack","task_name":"Adversarial Attack"},{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"inductive-bias","task_name":"Inductive Bias"},{"task_slug":"text-generation","task_name":"Text Generation"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.05657","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.05657"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/TRAIS-Lab/adversarial-attackdata-attribution","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/trais-lab/adversarial-attack-data-attribution","reach":null}],"summary":{"ran_draft_wrong":4,"ran_fixture":1,"ran_honours":1},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"549400e6d4070a65","entry":"subset_to_tensors","repo":"trais-lab/adversarial-attack-data-attribution","repo_kind":"official","path":"shadow_attack.py","file_url":"https://github.com/trais-lab/adversarial-attack-data-attribution/blob/HEAD/shadow_attack.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"549400e6d4070a65"}},{"code_sha256_prefix":"05d83c2629374306","entry":"adversarial_perturbation_with_loss","repo":"trais-lab/adversarial-attack-data-attribution","repo_kind":"official","path":"shadow_attack.py","file_url":"https://github.com/trais-lab/adversarial-attack-data-attribution/blob/HEAD/shadow_attack.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"05d83c2629374306"}},{"code_sha256_prefix":"e54cc8dba56e3913","entry":"central_difference_gradient_sign","repo":"trais-lab/adversarial-attack-data-attribution","repo_kind":"official","path":"outlier_attack.py","file_url":"https://github.com/trais-lab/adversarial-attack-data-attribution/blob/HEAD/outlier_attack.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e54cc8dba56e3913"}},{"code_sha256_prefix":"c77ff39bbd037f2b","entry":"fgsm_attack_blackbox","repo":"trais-lab/adversarial-attack-data-attribution","repo_kind":"official","path":"outlier_attack.py","file_url":"https://github.com/trais-lab/adversarial-attack-data-attribution/blob/HEAD/outlier_attack.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c77ff39bbd037f2b"}},{"code_sha256_prefix":"b2b118a6c404640d","entry":"load_models_from_state_dicts","repo":"trais-lab/adversarial-attack-data-attribution","repo_kind":"official","path":"shadow_attack.py","file_url":"https://github.com/trais-lab/adversarial-attack-data-attribution/blob/HEAD/shadow_attack.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b2b118a6c404640d"}},{"code_sha256_prefix":"d395f57eae436d2e","entry":"simba","repo":"trais-lab/adversarial-attack-data-attribution","repo_kind":"official","path":"outlier_attack.py","file_url":"https://github.com/trais-lab/adversarial-attack-data-attribution/blob/HEAD/outlier_attack.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d395f57eae436d2e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}