{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/beyond-memorization-violating-privacy-via","title":"Beyond Memorization: Violating Privacy Via Inference with Large Language Models","arxiv_id":"2310.07298","date":"2023-10-11","proceeding":null,"authors":["Robin Staab","Mark Vero","Mislav Balunović","Martin Vechev"],"abstract":"Current privacy research on large language models (LLMs) primarily focuses on the issue of extracting memorized training data. At the same time, models' inference capabilities have increased drastically. This raises the key question of whether current LLMs could violate individuals' privacy by inferring personal attributes from text given at inference time. In this work, we present the first comprehensive study on the capabilities of pretrained LLMs to infer personal attributes from text. We construct a dataset consisting of real Reddit profiles, and show that current LLMs can infer a wide range of personal attributes (e.g., location, income, sex), achieving up to $85\\%$ top-1 and $95\\%$ top-3 accuracy at a fraction of the cost ($100\\times$) and time ($240\\times$) required by humans. As people increasingly interact with LLM-powered chatbots across all aspects of life, we also explore the emerging threat of privacy-invasive chatbots trying to extract personal information through seemingly benign questions. Finally, we show that common mitigations, i.e., text anonymization and model alignment, are currently ineffective at protecting user privacy against LLM inference. Our findings highlight that current LLMs can infer personal data at a previously unattainable scale. In the absence of working defenses, we advocate for a broader discussion around LLM privacy implications beyond memorization, striving for a wider privacy protection.","url_abs":"https://arxiv.org/abs/2310.07298v2","url_pdf":"https://arxiv.org/pdf/2310.07298v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"beyond-memorization-violating-privacy-via","repo_url":"https://github.com/eth-sri/llmprivacy","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"beyond-memorization-violating-privacy-via","repo_url":"https://github.com/eth-sri/synthpai","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"memorization","task_name":"Memorization"},{"task_slug":"text-anonymization","task_name":"Text Anonymization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2310.07298","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.07298"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eth-sri/llmprivacy","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eth-sri/synthpai","reach":null}],"summary":{"ran_draft_wrong":2,"ran":10,"ran_honours":1},"by_repo_kind":{"official":{"samples":11,"ran":11,"repositories":1},"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"c7ebc428e1566486","entry":"adult_filter","repo":"eth-sri/llmprivacy","repo_kind":"official","path":"src/acs/folk_utils.py","file_url":"https://github.com/eth-sri/llmprivacy/blob/HEAD/src/acs/folk_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7ebc428e1566486"}},{"code_sha256_prefix":"d244c2276b806bad","entry":"birthday_interests","repo":"eth-sri/llmprivacy","repo_kind":"official","path":"src/acs/prompt_helpers.py","file_url":"https://github.com/eth-sri/llmprivacy/blob/HEAD/src/acs/prompt_helpers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d244c2276b806bad"}},{"code_sha256_prefix":"de827ce9167acceb","entry":"build_user_bot_system_prompt","repo":"eth-sri/llmprivacy","repo_kind":"official","path":"src/chat/user_bot_system_prompt_builder.py","file_url":"https://github.com/eth-sri/llmprivacy/blob/HEAD/src/chat/user_bot_system_prompt_builder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de827ce9167acceb"}},{"code_sha256_prefix":"3886295c0fa57d40","entry":"check_if_guess","repo":"eth-sri/llmprivacy","repo_kind":"official","path":"src/chat/chat_parser.py","file_url":"https://github.com/eth-sri/llmprivacy/blob/HEAD/src/chat/chat_parser.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3886295c0fa57d40"}},{"code_sha256_prefix":"63e3f3bd138014ac","entry":"compare_ages","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"63e3f3bd138014ac"}},{"code_sha256_prefix":"b1dacac750646f60","entry":"extract_chat","repo":"eth-sri/llmprivacy","repo_kind":"official","path":"src/chat/chat_parser.py","file_url":"https://github.com/eth-sri/llmprivacy/blob/HEAD/src/chat/chat_parser.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1dacac750646f60"}},{"code_sha256_prefix":"a533533aba00cb14","entry":"extract_predictions","repo":"eth-sri/llmprivacy","repo_kind":"official","path":"src/chat/chat_parser.py","file_url":"https://github.com/eth-sri/llmprivacy/blob/HEAD/src/chat/chat_parser.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a533533aba00cb14"}},{"code_sha256_prefix":"396503cafa45b9e2","entry":"get_acs_cat","repo":"eth-sri/llmprivacy","repo_kind":"official","path":"src/acs/folk_constants.py","file_url":"https://github.com/eth-sri/llmprivacy/blob/HEAD/src/acs/folk_constants.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"396503cafa45b9e2"}},{"code_sha256_prefix":"eae830ace5a03415","entry":"get_inv_acs_cat","repo":"eth-sri/llmprivacy","repo_kind":"official","path":"src/acs/folk_constants.py","file_url":"https://github.com/eth-sri/llmprivacy/blob/HEAD/src/acs/folk_constants.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eae830ace5a03415"}},{"code_sha256_prefix":"a3f3bb2859d03c81","entry":"get_ord","repo":"eth-sri/llmprivacy","repo_kind":"official","path":"src/acs/folk_utils.py","file_url":"https://github.com/eth-sri/llmprivacy/blob/HEAD/src/acs/folk_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3f3bb2859d03c81"}},{"code_sha256_prefix":"987fa5b0a23f7735","entry":"get_spaces","repo":"eth-sri/llmprivacy","repo_kind":"official","path":"src/chat/eval_chat.py","file_url":"https://github.com/eth-sri/llmprivacy/blob/HEAD/src/chat/eval_chat.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"987fa5b0a23f7735"}},{"code_sha256_prefix":"f109daac4a53d3a3","entry":"get_usernames","repo":"eth-sri/synthpai","repo_kind":"listed","path":"src/thread/generate_online_profiles.py","file_url":"https://github.com/eth-sri/synthpai/blob/HEAD/src/thread/generate_online_profiles.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f109daac4a53d3a3"}},{"code_sha256_prefix":"e035f70b2a617317","entry":"sample_return","repo":"eth-sri/llmprivacy","repo_kind":"official","path":"src/acs/prompt_helpers.py","file_url":"https://github.com/eth-sri/llmprivacy/blob/HEAD/src/acs/prompt_helpers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e035f70b2a617317"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}