{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-30642","title":"Diffusion Models Preferentially Memorize Prototypical Examples or: Why Does My Diffusion Model Love Slop?","arxiv_id":"2605.30642","date":"2026-05-28","proceeding":"ICML","authors":["Marta Aparicio Rodriguez","Anastasia Borovykh","Grigorios A. Pavliotis","Daniel J. Korchinski"],"abstract":"Generative models have a persistent limitation: their tendency to memorize training data can create legal liabilities and erode creative diversity. Understanding which samples are memorized in whole or in part, and under what conditions, therefore remains an important open problem. Here we answer the question \"Are atypical or rare samples memorized first?\" in the negative. We train diffusion models on strings generated according to the production rules of the Random Hierarchy Model (RHM), and find that samples composed of common substrings are preferentially memorized. This holds true even if the training data consists of entirely unique samples, indicating that deduplication at the data point level does not provide a meaningful privacy guarantee. Correspondingly we predict, then observe, delayed memorization for fat-tailed datasets (i.e., those with more atypical samples). This effect is amplified when fat-tails are introduced into high-level production rules. These together suggest that dataset diversity, particularly at higher levels of abstraction, plays an important role in staving off memorization. Finally, we identify an intermediate regime of partial memorization in which common substrings are learned first and subsequently overproduced during generation. If training is stopped in this regime, models will exhibit the reversion-to-the-mean blandness often derided as \"slop\".","url_abs":"https://arxiv.org/abs/2605.30642","url_pdf":"https://arxiv.org/pdf/2605.30642","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.30642","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.30642"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/martaaparod/memorization-in-diffusion-models","reach":null}],"summary":{"ran":13,"unverified":6},"by_repo_kind":{"found_in_text":{"samples":19,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ea9c6216fde96f44","entry":"build_training_tensor","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_celeba/evals/memorization.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_celeba/evals/memorization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"ea9c6216fde96f44"}},{"code_sha256_prefix":"ed2b679e56cc65e3","entry":"compare_with_trainset","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_rhm/diffusion/evaluate_model.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_rhm/diffusion/evaluate_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"ed2b679e56cc65e3"}},{"code_sha256_prefix":"a3da630747220f19","entry":"compute_knn","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_celeba/evals/memorization.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_celeba/evals/memorization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"a3da630747220f19"}},{"code_sha256_prefix":"cf279b1c29f0f84a","entry":"compute_pseudolikelihood","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_celeba/evals/spin_modelling.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_celeba/evals/spin_modelling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"cf279b1c29f0f84a"}},{"code_sha256_prefix":"e48a6e99e136f221","entry":"convert_labels_to_spins","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_celeba/evals/spin_modelling.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_celeba/evals/spin_modelling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e48a6e99e136f221"}},{"code_sha256_prefix":"537d6f0f1026e237","entry":"create_temp_image_dir","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_celeba/evals/compute_fid.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_celeba/evals/compute_fid.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"537d6f0f1026e237"}},{"code_sha256_prefix":"8fc6eff3b3ddc322","entry":"fit_ising_to_spins","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_celeba/evals/spin_modelling.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_celeba/evals/spin_modelling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"8fc6eff3b3ddc322"}},{"code_sha256_prefix":"23a0f0bfe8bb0faf","entry":"generate_labels","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_rhm/datasets/random_hierarchy_model_probability.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_rhm/datasets/random_hierarchy_model_probability.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"23a0f0bfe8bb0faf"}},{"code_sha256_prefix":"1850b19759b64aea","entry":"get_compatible_positions","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_rhm/belief_propagation/BP_utils.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_rhm/belief_propagation/BP_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"1850b19759b64aea"}},{"code_sha256_prefix":"8e8cc463ce85db65","entry":"hierarhical_copies","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_rhm/diffusion/evaluate_model.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_rhm/diffusion/evaluate_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"8e8cc463ce85db65"}},{"code_sha256_prefix":"bf5065959dbc94ef","entry":"init_optimizer","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_rhm/init.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_rhm/init.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"bf5065959dbc94ef"}},{"code_sha256_prefix":"56e2bee4fcc76b15","entry":"predict_attributes","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_celeba/evals/classifier.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_celeba/evals/classifier.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"56e2bee4fcc76b15"}},{"code_sha256_prefix":"a9ce17e0553626bc","entry":"save_samples_as_images","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_celeba/evals/compute_fid.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_celeba/evals/compute_fid.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"a9ce17e0553626bc"}},{"code_sha256_prefix":"422c9d116468f228","entry":"BP_countcorrect_upward","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_rhm/belief_propagation/BP_utils.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_rhm/belief_propagation/BP_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"422c9d116468f228"}},{"code_sha256_prefix":"b6003bcf7aa4b396","entry":"compute_d3pm_loss_per_time","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_rhm/diffusion/evaluate_model.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_rhm/diffusion/evaluate_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"b6003bcf7aa4b396"}},{"code_sha256_prefix":"50b3dd68a266d4d5","entry":"compute_mem","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_celeba/evals/memorization.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_celeba/evals/memorization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"50b3dd68a266d4d5"}},{"code_sha256_prefix":"9989ba91f384f161","entry":"load_multilabel_classifier","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_celeba/evals/classifier.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_celeba/evals/classifier.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"9989ba91f384f161"}},{"code_sha256_prefix":"caa0da4ea1171c73","entry":"sample_data_from_labels","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_rhm/datasets/random_hierarchy_model_probability.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_rhm/datasets/random_hierarchy_model_probability.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"caa0da4ea1171c73"}},{"code_sha256_prefix":"b2239ae7b9eff145","entry":"sample_rules","repo":"martaaparod/memorization-in-diffusion-models","repo_kind":"found_in_text","path":"diffusion_rhm/datasets/random_hierarchy_model_probability.py","file_url":"https://github.com/martaaparod/memorization-in-diffusion-models/blob/HEAD/diffusion_rhm/datasets/random_hierarchy_model_probability.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"b2239ae7b9eff145"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}