{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/privacylens-evaluating-privacy-norm-awareness","title":"PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action","arxiv_id":"2409.00138","date":"2024-08-29","proceeding":null,"authors":["Yijia Shao","Tianshi Li","Weiyan Shi","Yanchen Liu","Diyi Yang"],"abstract":"As language models (LMs) are widely utilized in personalized communication scenarios (e.g., sending emails, writing social media posts) and endowed with a certain level of agency, ensuring they act in accordance with the contextual privacy norms becomes increasingly critical. However, quantifying the privacy norm awareness of LMs and the emerging privacy risk in LM-mediated communication is challenging due to (1) the contextual and long-tailed nature of privacy-sensitive cases, and (2) the lack of evaluation approaches that capture realistic application scenarios. To address these challenges, we propose PrivacyLens, a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories, enabling multi-level evaluation of privacy leakage in LM agents' actions. We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds. Using this dataset, we reveal a discrepancy between LM performance in answering probing questions and their actual behavior when executing user instructions in an agent setup. State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions. We also demonstrate the dynamic nature of PrivacyLens by extending each seed into multiple trajectories to red-team LM privacy leakage risk. Dataset and code are available at https://github.com/SALT-NLP/PrivacyLens.","url_abs":"https://arxiv.org/abs/2409.00138v3","url_pdf":"https://arxiv.org/pdf/2409.00138v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"privacylens-evaluating-privacy-norm-awareness","repo_url":"https://github.com/salt-nlp/privacylens","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2409.00138","atlas_url":"https://app.syntology.ai/?focus=2409.00138","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.00138"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/SALT-NLP/PrivacyLens","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/salt-nlp/privacylens","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":9,"ran_draft_wrong":4},"by_repo_kind":{"official":{"samples":13,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"abe0fa181096220a","entry":"build_instruction_generation_prompt","repo":"salt-nlp/privacylens","repo_kind":"official","path":"data_construction/format_vignette_for_trajectory_simulation.py","file_url":"https://github.com/salt-nlp/privacylens/blob/HEAD/data_construction/format_vignette_for_trajectory_simulation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"abe0fa181096220a"}},{"code_sha256_prefix":"21e4de4c0eededee","entry":"build_seed2vignette_template","repo":"salt-nlp/privacylens","repo_kind":"official","path":"data_construction/seed_to_vignette.py","file_url":"https://github.com/salt-nlp/privacylens/blob/HEAD/data_construction/seed_to_vignette.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"21e4de4c0eededee"}},{"code_sha256_prefix":"c56cba76d2b331d1","entry":"load_json","repo":"salt-nlp/privacylens","repo_kind":"official","path":"helper/inspect_data.py","file_url":"https://github.com/salt-nlp/privacylens/blob/HEAD/helper/inspect_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c56cba76d2b331d1"}},{"code_sha256_prefix":"bc753da2fbd7dbb7","entry":"post_process","repo":"SALT-NLP/PrivacyLens","repo_kind":"official","path":"evaluation/get_final_action.py","file_url":"https://github.com/SALT-NLP/PrivacyLens/blob/HEAD/evaluation/get_final_action.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc753da2fbd7dbb7"}},{"code_sha256_prefix":"0e201f4f96ed959a","entry":"prepare_prompt_to_evaluate_helpfulness","repo":"salt-nlp/privacylens","repo_kind":"official","path":"evaluation/evaluate_final_action.py","file_url":"https://github.com/salt-nlp/privacylens/blob/HEAD/evaluation/evaluate_final_action.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e201f4f96ed959a"}},{"code_sha256_prefix":"923a15549e1d97f4","entry":"prepare_prompt_to_identify_leakage","repo":"salt-nlp/privacylens","repo_kind":"official","path":"evaluation/evaluate_final_action.py","file_url":"https://github.com/salt-nlp/privacylens/blob/HEAD/evaluation/evaluate_final_action.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"923a15549e1d97f4"}},{"code_sha256_prefix":"4b16c77d829248d2","entry":"prepare_prompt_to_identify_sensitive_information","repo":"salt-nlp/privacylens","repo_kind":"official","path":"evaluation/evaluate_final_action.py","file_url":"https://github.com/salt-nlp/privacylens/blob/HEAD/evaluation/evaluate_final_action.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4b16c77d829248d2"}},{"code_sha256_prefix":"cefb3c272dc7c713","entry":"prepare_seed_probing_prompt","repo":"SALT-NLP/PrivacyLens","repo_kind":"official","path":"evaluation/probing.py","file_url":"https://github.com/SALT-NLP/PrivacyLens/blob/HEAD/evaluation/probing.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cefb3c272dc7c713"}},{"code_sha256_prefix":"f3c42f0c80510056","entry":"prepare_vignette_probing_prompt","repo":"SALT-NLP/PrivacyLens","repo_kind":"official","path":"evaluation/probing.py","file_url":"https://github.com/SALT-NLP/PrivacyLens/blob/HEAD/evaluation/probing.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f3c42f0c80510056"}},{"code_sha256_prefix":"9725802be067e765","entry":"print_api_usage","repo":"salt-nlp/privacylens","repo_kind":"official","path":"helper/utils.py","file_url":"https://github.com/salt-nlp/privacylens/blob/HEAD/helper/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9725802be067e765"}},{"code_sha256_prefix":"41bfcf1b7980d8c2","entry":"process_trajectory","repo":"SALT-NLP/PrivacyLens","repo_kind":"official","path":"data_construction/format_trajectory.py","file_url":"https://github.com/SALT-NLP/PrivacyLens/blob/HEAD/data_construction/format_trajectory.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41bfcf1b7980d8c2"}},{"code_sha256_prefix":"28fb684d6e720c67","entry":"retry","repo":"salt-nlp/privacylens","repo_kind":"official","path":"helper/utils.py","file_url":"https://github.com/salt-nlp/privacylens/blob/HEAD/helper/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"28fb684d6e720c67"}},{"code_sha256_prefix":"ca248793a28dcb8b","entry":"test_no_explicit_word","repo":"salt-nlp/privacylens","repo_kind":"official","path":"data_construction/seed_to_vignette.py","file_url":"https://github.com/salt-nlp/privacylens/blob/HEAD/data_construction/seed_to_vignette.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca248793a28dcb8b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}