{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/do-ssl-models-have-deja-vu-a-case-of-1","title":"Do SSL Models Have Déjà Vu? A Case of Unintended Memorization in Self-supervised Learning","arxiv_id":"2304.13850","date":"2023-04-26","proceeding":"NeurIPS 2023 11","authors":["Casey Meehan","Florian Bordes","Pascal Vincent","Kamalika Chaudhuri","Chuan Guo"],"abstract":"Self-supervised learning (SSL) algorithms can produce useful image representations by learning to associate different parts of natural images with one another. However, when taken to the extreme, SSL models can unintendedly memorize specific parts in individual training samples rather than learning semantically meaningful associations. In this work, we perform a systematic study of the unintended memorization of image-specific information in SSL models -- which we refer to as d\\'ej\\`a vu memorization. Concretely, we show that given the trained model and a crop of a training image containing only the background (e.g., water, sky, grass), it is possible to infer the foreground object with high accuracy or even visually reconstruct it. Furthermore, we show that d\\'ej\\`a vu memorization is common to different SSL algorithms, is exacerbated by certain design choices, and cannot be detected by conventional techniques for evaluating representation quality. Our study of d\\'ej\\`a vu memorization reveals previously unknown privacy risks in SSL models, as well as suggests potential practical mitigation strategies. Code is available at https://github.com/facebookresearch/DejaVu.","url_abs":"https://arxiv.org/abs/2304.13850v3","url_pdf":"https://arxiv.org/pdf/2304.13850v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"do-ssl-models-have-deja-vu-a-case-of-1","repo_url":"https://github.com/facebookresearch/dejavu","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"memorization","task_name":"Memorization"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2304.13850","atlas_url":"https://app.syntology.ai/?focus=2304.13850","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.13850"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/dejavu","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_violates":1,"ran_draft_wrong":1,"ran_fixture":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"28c640db8f721c0d","entry":"exclude_bias_and_norm","repo":"facebookresearch/dejavu","repo_kind":"official","path":"train_SSL.py","file_url":"https://github.com/facebookresearch/dejavu/blob/HEAD/train_SSL.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"28c640db8f721c0d"}},{"code_sha256_prefix":"fc2c59422af9d93e","entry":"gather_center","repo":"facebookresearch/dejavu","repo_kind":"official","path":"train_SSL.py","file_url":"https://github.com/facebookresearch/dejavu/blob/HEAD/train_SSL.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fc2c59422af9d93e"}},{"code_sha256_prefix":"3e30d88eaef01190","entry":"off_diagonal","repo":"facebookresearch/dejavu","repo_kind":"official","path":"train_SSL.py","file_url":"https://github.com/facebookresearch/dejavu/blob/HEAD/train_SSL.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3e30d88eaef01190"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}