{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/finding-nemo-localizing-neurons-responsible","title":"Finding NeMo: Localizing Neurons Responsible For Memorization in Diffusion Models","arxiv_id":"2406.02366","date":"2024-06-04","proceeding":null,"authors":["Dominik Hintersdorf","Lukas Struppek","Kristian Kersting","Adam Dziedzic","Franziska Boenisch"],"abstract":"Diffusion models (DMs) produce very detailed and high-quality images. Their power results from extensive training on large amounts of data, usually scraped from the internet without proper attribution or consent from content creators. Unfortunately, this practice raises privacy and intellectual property concerns, as DMs can memorize and later reproduce their potentially sensitive or copyrighted training images at inference time. Prior efforts prevent this issue by either changing the input to the diffusion process, thereby preventing the DM from generating memorized samples during inference, or removing the memorized data from training altogether. While those are viable solutions when the DM is developed and deployed in a secure and constantly monitored environment, they hold the risk of adversaries circumventing the safeguards and are not effective when the DM itself is publicly released. To solve the problem, we introduce NeMo, the first method to localize memorization of individual data samples down to the level of neurons in DMs' cross-attention layers. Through our experiments, we make the intriguing finding that in many cases, single neurons are responsible for memorizing particular training samples. By deactivating these memorization neurons, we can avoid the replication of training data at inference time, increase the diversity in the generated outputs, and mitigate the leakage of private and copyrighted data. In this way, our NeMo contributes to a more responsible deployment of DMs.","url_abs":"https://arxiv.org/abs/2406.02366v3","url_pdf":"https://arxiv.org/pdf/2406.02366v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"finding-nemo-localizing-neurons-responsible","repo_url":"https://github.com/ml-research/localizing_memorization_in_diffusion_models","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"memorization","task_name":"Memorization"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.02366","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.02366"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ml-research/localizing_memorization_in_diffusion_models","reach":null}],"summary":{"ran":1,"ran_honours":1,"ran_fixture":1,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"b1d5f7f04d03657f","entry":"CollectActivationsLinearNoMean","repo":"ml-research/localizing_memorization_in_diffusion_models","repo_kind":"official","path":"utils/activation_detection.py","file_url":"https://github.com/ml-research/localizing_memorization_in_diffusion_models/blob/HEAD/utils/activation_detection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1d5f7f04d03657f"}},{"code_sha256_prefix":"5cf0f548a7148c08","entry":"get_num_neurons_per_layer","repo":"ml-research/localizing_memorization_in_diffusion_models","repo_kind":"official","path":"4_generate_images.py","file_url":"https://github.com/ml-research/localizing_memorization_in_diffusion_models/blob/HEAD/4_generate_images.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5cf0f548a7148c08"}},{"code_sha256_prefix":"302a044fe46029e6","entry":"make_image_grid","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"302a044fe46029e6"}},{"code_sha256_prefix":"92ce689c38a1516e","entry":"prepare_diffusion_inputs","repo":"ml-research/localizing_memorization_in_diffusion_models","repo_kind":"official","path":"utils/activation_detection.py","file_url":"https://github.com/ml-research/localizing_memorization_in_diffusion_models/blob/HEAD/utils/activation_detection.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"92ce689c38a1516e"}},{"code_sha256_prefix":"faf67b17312a8edc","entry":"str_to_list","repo":"ml-research/localizing_memorization_in_diffusion_models","repo_kind":"official","path":"4_generate_images.py","file_url":"https://github.com/ml-research/localizing_memorization_in_diffusion_models/blob/HEAD/4_generate_images.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"faf67b17312a8edc"}},{"code_sha256_prefix":"36cd98a1efde20c8","entry":"collect_activations","repo":"ml-research/localizing_memorization_in_diffusion_models","repo_kind":"official","path":"utils/activation_detection.py","file_url":"https://github.com/ml-research/localizing_memorization_in_diffusion_models/blob/HEAD/utils/activation_detection.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"36cd98a1efde20c8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}