{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-better-statistical-understanding-of","title":"Towards Better Statistical Understanding of Watermarking LLMs","arxiv_id":"2403.13027","date":"2024-03-19","proceeding":null,"authors":["Zhongze Cai","Shang Liu","Hanzhao Wang","Huaiyang Zhong","Xiaocheng Li"],"abstract":"In this paper, we study the problem of watermarking large language models (LLMs). We consider the trade-off between model distortion and detection ability and formulate it as a constrained optimization problem based on the green-red algorithm of Kirchenbauer et al. (2023a). We show that the optimal solution to the optimization problem enjoys a nice analytical property which provides a better understanding and inspires the algorithm design for the watermarking process. We develop an online dual gradient ascent watermarking algorithm in light of this optimization formulation and prove its asymptotic Pareto optimality between model distortion and detection ability. Such a result guarantees an averaged increased green list probability and henceforth detection ability explicitly (in contrast to previous results). Moreover, we provide a systematic discussion on the choice of the model distortion metrics for the watermarking problem. We justify our choice of KL divergence and present issues with the existing criteria of ``distortion-free'' and perplexity. Finally, we empirically evaluate our algorithms on extensive datasets against benchmark algorithms.","url_abs":"https://arxiv.org/abs/2403.13027v1","url_pdf":"https://arxiv.org/pdf/2403.13027v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-better-statistical-understanding-of","repo_url":"https://github.com/zhongzecai/dualga","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.13027","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.13027"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhongzecai/dualga","reach":{"status":"ok"}}],"summary":{"ran":18,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":23,"ran":19,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":23,"samples":[{"code_sha256_prefix":"88258837d760b644","entry":"DGt","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/DualGD_generator.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/DualGD_generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"88258837d760b644"}},{"code_sha256_prefix":"bcc348526121329b","entry":"KLt","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/DualGD_generator.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/DualGD_generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bcc348526121329b"}},{"code_sha256_prefix":"5213b646c1f83e4a","entry":"add_idx","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/data_processing.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/data_processing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5213b646c1f83e4a"}},{"code_sha256_prefix":"3e107246e54539b5","entry":"additive_prf","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/alternative_prf_schemes.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/alternative_prf_schemes.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3e107246e54539b5"}},{"code_sha256_prefix":"428e9dd9cdeca201","entry":"concat_rows","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/evaluation.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"428e9dd9cdeca201"}},{"code_sha256_prefix":"4ea6b15c91ca13a0","entry":"conditional_no_check_args","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/evaluation.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4ea6b15c91ca13a0"}},{"code_sha256_prefix":"8880355eca496e49","entry":"deletion_attack","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/attack_light.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/attack_light.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8880355eca496e49"}},{"code_sha256_prefix":"a621beb4ebc978ae","entry":"denom_func","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/DualGD_generator.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/DualGD_generator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a621beb4ebc978ae"}},{"code_sha256_prefix":"c88510de06c5a48b","entry":"dict_remove","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/data_processing.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/data_processing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c88510de06c5a48b"}},{"code_sha256_prefix":"2b0288f4b606671b","entry":"insertion_attack","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/attack_light.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/attack_light.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2b0288f4b606671b"}},{"code_sha256_prefix":"19ee4683eda506c6","entry":"list_of_strings","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/DualGD_generate.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/DualGD_generate.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19ee4683eda506c6"}},{"code_sha256_prefix":"571045e954da6d27","entry":"list_of_strings","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/MinACF_eval.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/MinACF_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"571045e954da6d27"}},{"code_sha256_prefix":"142c8b133ed9f266","entry":"list_relative_jsonl_file_paths","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/ppl.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/ppl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"142c8b133ed9f266"}},{"code_sha256_prefix":"ee2e7209ff4d37fa","entry":"load_full_results","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/MinACF_eval.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/MinACF_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ee2e7209ff4d37fa"}},{"code_sha256_prefix":"ab84746bf94016d2","entry":"multiplicative_prf","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/alternative_prf_schemes.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/alternative_prf_schemes.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab84746bf94016d2"}},{"code_sha256_prefix":"97bace61e91ab919","entry":"seeding_scheme_lookup","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/alternative_prf_schemes.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/alternative_prf_schemes.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"97bace61e91ab919"}},{"code_sha256_prefix":"f6febe281ce78bb3","entry":"single_insertion","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/attack_advanced.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/attack_advanced.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f6febe281ce78bb3"}},{"code_sha256_prefix":"3310b2d9bf130318","entry":"substitution_attack","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/attack_light.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/attack_light.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3310b2d9bf130318"}},{"code_sha256_prefix":"b9619a7b2e06f6ff","entry":"triple_insertion_single_len","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/attack_advanced.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/attack_advanced.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b9619a7b2e06f6ff"}},{"code_sha256_prefix":"0ff1af55ee3a1d09","entry":"gen_watermarked_n_unwatermarked","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/generating.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/generating.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0ff1af55ee3a1d09"}},{"code_sha256_prefix":"0b925ea9412f4d08","entry":"generate_dipper_paraphrases","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/attack_advanced.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/attack_advanced.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0b925ea9412f4d08"}},{"code_sha256_prefix":"86b01270fe49c4d1","entry":"load_hf_dataset","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/data_processing.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/data_processing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"86b01270fe49c4d1"}},{"code_sha256_prefix":"ff61ef7b3942bc85","entry":"load_tokenizer","repo":"zhongzecai/dualga","repo_kind":"official","path":"WatermarkAlgorithm/utils/evaluation.py","file_url":"https://github.com/zhongzecai/dualga/blob/HEAD/WatermarkAlgorithm/utils/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ff61ef7b3942bc85"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}