{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/clipping-privacy-identity-inference-attacks","title":"Does CLIP Know My Face?","arxiv_id":"2209.07341","date":"2022-09-15","proceeding":null,"authors":["Dominik Hintersdorf","Lukas Struppek","Manuel Brack","Felix Friedrich","Patrick Schramowski","Kristian Kersting"],"abstract":"With the rise of deep learning in various applications, privacy concerns around the protection of training data have become a critical area of research. Whereas prior studies have focused on privacy risks in single-modal models, we introduce a novel method to assess privacy for multi-modal models, specifically vision-language models like CLIP. The proposed Identity Inference Attack (IDIA) reveals whether an individual was included in the training data by querying the model with images of the same person. Letting the model choose from a wide variety of possible text labels, the model reveals whether it recognizes the person and, therefore, was used for training. Our large-scale experiments on CLIP demonstrate that individuals used for training can be identified with very high accuracy. We confirm that the model has learned to associate names with depicted individuals, implying the existence of sensitive information that can be extracted by adversaries. Our results highlight the need for stronger privacy protection in large-scale models and suggest that IDIAs can be used to prove the unauthorized use of data for training and to enforce privacy laws.","url_abs":"https://arxiv.org/abs/2209.07341v4","url_pdf":"https://arxiv.org/pdf/2209.07341v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"clipping-privacy-identity-inference-attacks","repo_url":"https://github.com/d0mih/clipping_privacy","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"clipping-privacy-identity-inference-attacks","repo_url":"https://github.com/d0mih/does-clip-know-my-face","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"clipping-privacy-identity-inference-attacks","repo_url":"https://github.com/yangyucheng000/Paper-4/tree/main/DOEmindspore","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"inference-attack","task_name":"Inference Attack"}],"methods":[{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2209.07341","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.07341"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/d0mih/clipping_privacy","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yangyucheng000/Paper-4/tree/main/DOEmindspore","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/d0mih/does-clip-know-my-face","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"53a51218bc9d7ab8","entry":"get_class_from_module","repo":"d0mih/clipping_privacy","repo_kind":"official","path":"facescrub_training/utils/get_class_from_module.py","file_url":"https://github.com/d0mih/clipping_privacy/blob/HEAD/facescrub_training/utils/get_class_from_module.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53a51218bc9d7ab8"}},{"code_sha256_prefix":"2b64126e67ef7d8b","entry":"get_filtered_names","repo":"d0mih/clipping_privacy","repo_kind":"official","path":"laion400m_experiments/00_search_laion_captions.py","file_url":"https://github.com/d0mih/clipping_privacy/blob/HEAD/laion400m_experiments/00_search_laion_captions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b64126e67ef7d8b"}},{"code_sha256_prefix":"df4617cd4504000b","entry":"read_actor_files","repo":"d0mih/clipping_privacy","repo_kind":"official","path":"laion400m_experiments/00_download_laion_european_non_members.py","file_url":"https://github.com/d0mih/clipping_privacy/blob/HEAD/laion400m_experiments/00_download_laion_european_non_members.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df4617cd4504000b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}