{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2601-01966","title":"Refinement Provenance Inference: Detecting LLM-Refined Training Prompts from Model Behavior","arxiv_id":"2601.01966","date":"2026-01-05","proceeding":null,"authors":["Bo Yin","Qi Li","Runpeng Yu","Xinchao Wang"],"abstract":"Instruction tuning increasingly relies on LLM-based prompt refinement, where prompts in the training corpus are selectively rewritten by an external refiner to improve clarity and instruction alignment. This motivates an instance-level audit problem: for a fine-tuned model and a training prompt-response pair, can we infer whether the model was trained on the original prompt or its LLM-refined version within a mixed corpus? This matters for dataset governance and dispute resolution when training data are contested. However, it is non-trivial in practice: refined and raw instances are interleaved in the training corpus with unknown, source-dependent mixture ratios, making it harder to develop provenance methods that generalize across models and training setups. In this paper, we formalize this audit task as Refinement Provenance Inference (RPI) and show that prompt refinement yields stable, detectable shifts in teacher-forced token distributions, even when semantic differences are not obvious. Building on this phenomenon, we propose RePro, a logit-based provenance framework that fuses teacher-forced likelihood features with logit-ranking signals. During training, RePro learns a transferable representation via shadow fine-tuning, and uses a lightweight linear head to infer provenance on unseen victims without training-data access. Empirically, RePro consistently attains strong performance and transfers well across refiners, suggesting that it exploits refiner-agnostic distribution shifts rather than rewrite-style artifacts.","url_abs":"https://arxiv.org/abs/2601.01966","url_pdf":"https://arxiv.org/pdf/2601.01966","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2601.01966","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2601.01966"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/YinBo0927/RePro","reach":null}],"summary":{"ran":5,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"03a626a2b907b17e","entry":"AttackerArtifacts","repo":"YinBo0927/RePro","repo_kind":"found_in_text","path":"repro/attacker.py","file_url":"https://github.com/YinBo0927/RePro/blob/HEAD/repro/attacker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"03a626a2b907b17e"}},{"code_sha256_prefix":"6407c57856c65394","entry":"LinearHead","repo":"YinBo0927/RePro","repo_kind":"found_in_text","path":"repro/attacker.py","file_url":"https://github.com/YinBo0927/RePro/blob/HEAD/repro/attacker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6407c57856c65394"}},{"code_sha256_prefix":"1da9bbef0ddba734","entry":"MLPEncoder","repo":"YinBo0927/RePro","repo_kind":"found_in_text","path":"repro/attacker.py","file_url":"https://github.com/YinBo0927/RePro/blob/HEAD/repro/attacker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1da9bbef0ddba734"}},{"code_sha256_prefix":"d91949fe33a1cb79","entry":"ProjectionHead","repo":"YinBo0927/RePro","repo_kind":"found_in_text","path":"repro/attacker.py","file_url":"https://github.com/YinBo0927/RePro/blob/HEAD/repro/attacker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d91949fe33a1cb79"}},{"code_sha256_prefix":"798db8cfc97bae25","entry":"Standardizer","repo":"YinBo0927/RePro","repo_kind":"found_in_text","path":"repro/attacker.py","file_url":"https://github.com/YinBo0927/RePro/blob/HEAD/repro/attacker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"798db8cfc97bae25"}},{"code_sha256_prefix":"0945a27516e8d776","entry":"compute_standardizer","repo":"YinBo0927/RePro","repo_kind":"found_in_text","path":"repro/attacker.py","file_url":"https://github.com/YinBo0927/RePro/blob/HEAD/repro/attacker.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0945a27516e8d776"}},{"code_sha256_prefix":"2c8f2775bd4f3adb","entry":"supervised_contrastive_loss","repo":"YinBo0927/RePro","repo_kind":"found_in_text","path":"repro/attacker.py","file_url":"https://github.com/YinBo0927/RePro/blob/HEAD/repro/attacker.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2c8f2775bd4f3adb"}},{"code_sha256_prefix":"d256fb42ad8a989a","entry":"train_attacker","repo":"YinBo0927/RePro","repo_kind":"found_in_text","path":"repro/attacker.py","file_url":"https://github.com/YinBo0927/RePro/blob/HEAD/repro/attacker.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d256fb42ad8a989a"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}