{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/synthetic-text-generation-with-differential","title":"Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe","arxiv_id":"2210.14348","date":"2022-10-25","proceeding":null,"authors":["Xiang Yue","Huseyin A. Inan","Xuechen Li","Girish Kumar","Julia McAnallen","Hoda Shajari","Huan Sun","David Levitan","Robert Sim"],"abstract":"Privacy concerns have attracted increasing attention in data-driven products due to the tendency of machine learning models to memorize sensitive training data. Generating synthetic versions of such data with a formal privacy guarantee, such as differential privacy (DP), provides a promising path to mitigating these privacy concerns, but previous approaches in this direction have typically failed to produce synthetic data of high quality. In this work, we show that a simple and practical recipe in the text domain is effective: simply fine-tuning a pretrained generative language model with DP enables the model to generate useful synthetic text with strong privacy protection. Through extensive empirical analyses on both benchmark and private customer data, we demonstrate that our method produces synthetic text that is competitive in terms of utility with its non-private counterpart, meanwhile providing strong protection against potential privacy leakages.","url_abs":"https://arxiv.org/abs/2210.14348v3","url_pdf":"https://arxiv.org/pdf/2210.14348v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"synthetic-text-generation-with-differential","repo_url":"https://github.com/microsoft/dp-transformers","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2210.14348","atlas_url":"https://app.syntology.ai/?focus=2210.14348","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.14348"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/microsoft/dp-transformers","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"eed0eaf234273f91","entry":"calc_perplexity","repo":"microsoft/dp-transformers","repo_kind":"official","path":"research/synthetic-text-generation-with-DP/generate-text.py","file_url":"https://github.com/microsoft/dp-transformers/blob/HEAD/research/synthetic-text-generation-with-DP/generate-text.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eed0eaf234273f91"}},{"code_sha256_prefix":"67563e08a8fbe297","entry":"convert_model","repo":"microsoft/dp-transformers","repo_kind":"official","path":"research/synthetic-text-generation-with-DP/generate-text.py","file_url":"https://github.com/microsoft/dp-transformers/blob/HEAD/research/synthetic-text-generation-with-DP/generate-text.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67563e08a8fbe297"}},{"code_sha256_prefix":"3bdbafd8411ba32d","entry":"create_author_mapping","repo":"microsoft/dp-transformers","repo_kind":"official","path":"src/dp_transformers/dp_utils.py","file_url":"https://github.com/microsoft/dp-transformers/blob/HEAD/src/dp_transformers/dp_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3bdbafd8411ba32d"}},{"code_sha256_prefix":"575843dbe0cdffe7","entry":"main_preprocess_function","repo":"microsoft/dp-transformers","repo_kind":"official","path":"research/fine_tune_llm_w_qlora/data_utils.py","file_url":"https://github.com/microsoft/dp-transformers/blob/HEAD/research/fine_tune_llm_w_qlora/data_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"575843dbe0cdffe7"}},{"code_sha256_prefix":"cc6277c4af092dcc","entry":"force_causal_attention","repo":"microsoft/dp-transformers","repo_kind":"official","path":"src/dp_transformers/module_modification.py","file_url":"https://github.com/microsoft/dp-transformers/blob/HEAD/src/dp_transformers/module_modification.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc6277c4af092dcc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}