{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2606-00160","title":"DataShield: Safety-degrading Data Filtering for LLM Benign Instruction Fine-Tuning","arxiv_id":"2606.00160","date":"2026-05-29","proceeding":null,"authors":["Junbo Zhang","Qianli Zhou","Xinyang Deng","Wen Jiang","Jie Pan","Jinbiao Zhu"],"abstract":"Large language models (LLMs) suffer from degraded safety capabilities even when fine-tuned with benign datasets. However, existing methods for identifying safety-degrading samples in benign datasets suffer from high computational costs and significant noise issues. In this paper, we propose DataShield to efficiently and effectively identify potential safety-degrading samples. Our key intuition is based on the observation that benign fine-tuning increases the overall response compliance of LLMs. DataShield's key technical insight is to quantify each sample's contribution to the model's compliance behavior as its safety degradation score. DataShield consists of three core components: (1) Compliance Vector Extraction, which captures the LLM's compliance behavior tendency; (2) a novel Compliance-Aware Score (CAS), which automatically identifies the optimal safety-critical layer; and (3) Safety-degrading Sample Filtering, which quantifies the projection shift of training data along the compliance direction. Extensive experimental evaluation on Llama3-8B, Llama3.1-8B, and Qwen2.5-7B using the Alpaca and Dolly benign datasets validates our method's effectiveness in identifying high-risk and low-risk data subsets. We also observe that open-ended question answering is more likely to trigger safety degradation, and corresponding responses tend to be longer. We hope this work can provide new insights into data-centric defense methods. The source code is available at: https://github.com/ZJunBo/DataShield.","url_abs":"https://arxiv.org/abs/2606.00160","url_pdf":"https://arxiv.org/pdf/2606.00160","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2606.00160","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2606.00160"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ZJunBo/DataShield","reach":null}],"summary":{"ran_draft_wrong":4,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"c987b20e048caf01","entry":"extract_text_pairs","repo":"ZJunBo/DataShield","repo_kind":"found_in_text","path":"datashield/compute_projections.py","file_url":"https://github.com/ZJunBo/DataShield/blob/HEAD/datashield/compute_projections.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c987b20e048caf01"}},{"code_sha256_prefix":"27b6012e992f0750","entry":"load_records","repo":"ZJunBo/DataShield","repo_kind":"found_in_text","path":"datashield/compute_projections.py","file_url":"https://github.com/ZJunBo/DataShield/blob/HEAD/datashield/compute_projections.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27b6012e992f0750"}},{"code_sha256_prefix":"46ba8cd07ff9e0f3","entry":"load_vectors","repo":"ZJunBo/DataShield","repo_kind":"found_in_text","path":"datashield/compute_projections.py","file_url":"https://github.com/ZJunBo/DataShield/blob/HEAD/datashield/compute_projections.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"46ba8cd07ff9e0f3"}},{"code_sha256_prefix":"4ca5ec3965dafbde","entry":"projection","repo":"ZJunBo/DataShield","repo_kind":"found_in_text","path":"datashield/compute_projections.py","file_url":"https://github.com/ZJunBo/DataShield/blob/HEAD/datashield/compute_projections.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4ca5ec3965dafbde"}},{"code_sha256_prefix":"c3dcb7df28035d7d","entry":"run","repo":"ZJunBo/DataShield","repo_kind":"found_in_text","path":"datashield/compute_projections.py","file_url":"https://github.com/ZJunBo/DataShield/blob/HEAD/datashield/compute_projections.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c3dcb7df28035d7d"}},{"code_sha256_prefix":"625c919ec20a08b4","entry":"save_records","repo":"ZJunBo/DataShield","repo_kind":"found_in_text","path":"datashield/compute_projections.py","file_url":"https://github.com/ZJunBo/DataShield/blob/HEAD/datashield/compute_projections.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"625c919ec20a08b4"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}