{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/toward-effective-protection-against-diffusion","title":"Toward effective protection against diffusion based mimicry through score distillation","arxiv_id":"2311.12832","date":"2023-10-02","proceeding":null,"authors":["Haotian Xue","Chumeng Liang","Xiaoyu Wu","Yongxin Chen"],"abstract":"While generative diffusion models excel in producing high-quality images, they can also be misused to mimic authorized images, posing a significant threat to AI systems. Efforts have been made to add calibrated perturbations to protect images from diffusion-based mimicry pipelines. However, most of the existing methods are too ineffective and even impractical to be used by individual users due to their high computation and memory requirements. In this work, we present novel findings on attacking latent diffusion models (LDM) and propose new plug-and-play strategies for more effective protection. In particular, we explore the bottleneck in attacking an LDM, discovering that the encoder module rather than the denoiser module is the vulnerable point. Based on this insight, we present our strategy using Score Distillation Sampling (SDS) to double the speed of protection and reduce memory occupation by half without compromising its strength. Additionally, we provide a robust protection strategy by counterintuitively minimizing the semantic loss, which can assist in generating more natural perturbations. Finally, we conduct extensive experiments to substantiate our findings and comprehensively evaluate our newly proposed strategies. We hope our insights and protective measures can contribute to better defense against malicious diffusion-based mimicry, advancing the development of secure AI systems. The code is available in https://github.com/xavihart/Diff-Protect","url_abs":"https://arxiv.org/abs/2311.12832v2","url_pdf":"https://arxiv.org/pdf/2311.12832v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"toward-effective-protection-against-diffusion","repo_url":"https://github.com/xavihart/diff-protect","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2311.12832","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.12832"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/xavihart/Diff-Protect","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/lllyasviel/AdverseCleaner","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xavihart/diff-protect","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":2,"ran_violates":2,"ran":8,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":14,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"fe5dd5258046898c","entry":"always","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/ldm/modules/x_transformer.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/ldm/modules/x_transformer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fe5dd5258046898c"}},{"code_sha256_prefix":"424012cb37b31172","entry":"default","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/ldm/modules/attention.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"424012cb37b31172"}},{"code_sha256_prefix":"04a65b3be15e5c5c","entry":"draw_bound","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/utils.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"04a65b3be15e5c5c"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/ldm/modules/attention.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"a76f0c8fb0d739ed","entry":"get_dir_name_from_config","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/inpaint.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/inpaint.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a76f0c8fb0d739ed"}},{"code_sha256_prefix":"b1368330cf0f5642","entry":"isimage","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/ldm/util.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b1368330cf0f5642"}},{"code_sha256_prefix":"d72762b700feee6f","entry":"ismap","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/ldm/util.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d72762b700feee6f"}},{"code_sha256_prefix":"ac9f10cafae6f7f7","entry":"load_image_from_path","repo":"xavihart/diff-protect","repo_kind":"official","path":"code/diff_mist.py","file_url":"https://github.com/xavihart/diff-protect/blob/HEAD/code/diff_mist.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ac9f10cafae6f7f7"}},{"code_sha256_prefix":"19a8b4a09402db75","entry":"load_png","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/utils.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19a8b4a09402db75"}},{"code_sha256_prefix":"15598bf90ab2e955","entry":"normalized_","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/eval_gen.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/eval_gen.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"15598bf90ab2e955"}},{"code_sha256_prefix":"775763779edd8692","entry":"recover_image","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/utils.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"775763779edd8692"}},{"code_sha256_prefix":"a8979ef14445d81f","entry":"target_model","repo":"xavihart/diff-protect","repo_kind":"official","path":"code/diff_mist.py","file_url":"https://github.com/xavihart/diff-protect/blob/HEAD/code/diff_mist.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a8979ef14445d81f"}},{"code_sha256_prefix":"9a299fe5ae09e407","entry":"uniq","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/ldm/modules/attention.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9a299fe5ae09e407"}},{"code_sha256_prefix":"f9bd2e83191afad1","entry":"log_txt_as_img","repo":"xavihart/Diff-Protect","repo_kind":"official","path":"code/ldm/util.py","file_url":"https://github.com/xavihart/Diff-Protect/blob/HEAD/code/ldm/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f9bd2e83191afad1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}