{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/revealing-weaknesses-in-text-watermarking","title":"Revealing Weaknesses in Text Watermarking Through Self-Information Rewrite Attacks","arxiv_id":"2505.05190","date":"2025-05-08","proceeding":null,"authors":["Yixin Cheng","Hongcheng Guo","Yangming Li","Leonid Sigal"],"abstract":"Text watermarking aims to subtly embed statistical signals into text by controlling the Large Language Model (LLM)'s sampling process, enabling watermark detectors to verify that the output was generated by the specified model. The robustness of these watermarking algorithms has become a key factor in evaluating their effectiveness. Current text watermarking algorithms embed watermarks in high-entropy tokens to ensure text quality. In this paper, we reveal that this seemingly benign design can be exploited by attackers, posing a significant risk to the robustness of the watermark. We introduce a generic efficient paraphrasing attack, the Self-Information Rewrite Attack (SIRA), which leverages the vulnerability by calculating the self-information of each token to identify potential pattern tokens and perform targeted attack. Our work exposes a widely prevalent vulnerability in current watermarking algorithms. The experimental results show SIRA achieves nearly 100% attack success rates on seven recent watermarking methods with only 0.88 USD per million tokens cost. Our approach does not require any access to the watermark algorithms or the watermarked LLM and can seamlessly transfer to any LLM as the attack model, even mobile-level models. Our findings highlight the urgent need for more robust watermarking.","url_abs":"https://arxiv.org/abs/2505.05190v2","url_pdf":"https://arxiv.org/pdf/2505.05190v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"revealing-weaknesses-in-text-watermarking","repo_url":"https://github.com/allencheng97/self-information-rewrite-attack","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"large-language-model","task_name":"Large Language Model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2505.05190","atlas_url":"https://app.syntology.ai/?focus=2505.05190","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.05190"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/allencheng97/self-information-rewrite-attack","reach":null}],"summary":{"ran":2,"ran_draft_wrong":3,"unverified":5},"by_repo_kind":{"official":{"samples":10,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"3bffc91ce336d28f","entry":"ResidualBlock","repo":"allencheng97/self-information-rewrite-attack","repo_kind":"official","path":"watermark/sir/sir.py","file_url":"https://github.com/allencheng97/self-information-rewrite-attack/blob/HEAD/watermark/sir/sir.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3bffc91ce336d28f"}},{"code_sha256_prefix":"311b2486ffe7e54a","entry":"TransformModel","repo":"allencheng97/self-information-rewrite-attack","repo_kind":"official","path":"watermark/sir/sir.py","file_url":"https://github.com/allencheng97/self-information-rewrite-attack/blob/HEAD/watermark/sir/sir.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"311b2486ffe7e54a"}},{"code_sha256_prefix":"5ce8d8f0a23c979c","entry":"fill_attack_prompt","repo":"allencheng97/self-information-rewrite-attack","repo_kind":"official","path":"scripts/attack.py","file_url":"https://github.com/allencheng97/self-information-rewrite-attack/blob/HEAD/scripts/attack.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5ce8d8f0a23c979c"}},{"code_sha256_prefix":"56808bc9d9bbabec","entry":"fill_attack_prompt","repo":"allencheng97/self-information-rewrite-attack","repo_kind":"official","path":"scripts/attack_onestep.py","file_url":"https://github.com/allencheng97/self-information-rewrite-attack/blob/HEAD/scripts/attack_onestep.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"56808bc9d9bbabec"}},{"code_sha256_prefix":"7031b621735d8047","entry":"fill_parapharse_prompt","repo":"allencheng97/self-information-rewrite-attack","repo_kind":"official","path":"scripts/attack_onestep.py","file_url":"https://github.com/allencheng97/self-information-rewrite-attack/blob/HEAD/scripts/attack_onestep.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7031b621735d8047"}},{"code_sha256_prefix":"53374535d9028f3b","entry":"SIR","repo":"allencheng97/self-information-rewrite-attack","repo_kind":"official","path":"watermark/sir/sir.py","file_url":"https://github.com/allencheng97/self-information-rewrite-attack/blob/HEAD/watermark/sir/sir.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"53374535d9028f3b"}},{"code_sha256_prefix":"ee2297c2b0078380","entry":"SIRConfig","repo":"allencheng97/self-information-rewrite-attack","repo_kind":"official","path":"watermark/sir/sir.py","file_url":"https://github.com/allencheng97/self-information-rewrite-attack/blob/HEAD/watermark/sir/sir.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ee2297c2b0078380"}},{"code_sha256_prefix":"9d7bc12eaa7263d6","entry":"SIRLogitsProcessor","repo":"allencheng97/self-information-rewrite-attack","repo_kind":"official","path":"watermark/sir/sir.py","file_url":"https://github.com/allencheng97/self-information-rewrite-attack/blob/HEAD/watermark/sir/sir.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9d7bc12eaa7263d6"}},{"code_sha256_prefix":"f892499193069494","entry":"SIRUtils","repo":"allencheng97/self-information-rewrite-attack","repo_kind":"official","path":"watermark/sir/sir.py","file_url":"https://github.com/allencheng97/self-information-rewrite-attack/blob/HEAD/watermark/sir/sir.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f892499193069494"}},{"code_sha256_prefix":"8320582fa0e6577a","entry":"create_directory_for_file","repo":"allencheng97/self-information-rewrite-attack","repo_kind":"official","path":"watermark/sir/sir.py","file_url":"https://github.com/allencheng97/self-information-rewrite-attack/blob/HEAD/watermark/sir/sir.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8320582fa0e6577a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}