{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/understanding-and-improving-adversarial","title":"Targeted Attack Improves Protection against Unauthorized Diffusion Customization","arxiv_id":"2310.04687","date":"2023-10-07","proceeding":null,"authors":["BoYang Zheng","Chumeng Liang","Xiaoyu Wu"],"abstract":"Diffusion models build a new milestone for image generation yet raising public concerns, for they can be fine-tuned on unauthorized images for customization. Protection based on adversarial attacks rises to encounter this unauthorized diffusion customization, by adding protective watermarks to images and poisoning diffusion models. However, current protection, leveraging untargeted attacks, does not appear to be effective enough. In this paper, we propose a simple yet effective improvement for the protection against unauthorized diffusion customization by introducing targeted attacks. We show that by carefully selecting the target, targeted attacks significantly outperform untargeted attacks in poisoning diffusion models and degrading the customization image quality. Extensive experiments validate the superiority of our method on two mainstream customization methods of diffusion models, compared to existing protections. To explain the surprising success of targeted attacks, we delve into the mechanism of attack-based protections and propose a hypothesis based on our observation, which enhances the comprehension of attack-based protections. To the best of our knowledge, we are the first to both reveal the vulnerability of diffusion models to targeted attacks and leverage targeted attacks to enhance protection against unauthorized diffusion customization. Our code is available on GitHub: \\url{https://github.com/psyker-team/mist-v2}.","url_abs":"https://arxiv.org/abs/2310.04687v4","url_pdf":"https://arxiv.org/pdf/2310.04687v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"understanding-and-improving-adversarial","repo_url":"https://github.com/psyker-team/mist-v2","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"understanding-and-improving-adversarial","repo_url":"https://github.com/caradryanliang/improvedadvdm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"adversarial-attack","task_name":"Adversarial Attack"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"misinformation","task_name":"Misinformation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"latent-diffusion-model","method_name":"Latent Diffusion Model"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.04687","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.04687"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/psyker-team/mist-v2","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/caradryanliang/improvedadvdm","reach":{"status":"ok"}}],"summary":{"ran":10,"ran_draft_wrong":2,"ran_violates":2,"unverified":10},"by_repo_kind":{"official":{"samples":22,"ran":13,"repositories":2},"community":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"79482e8ecae87aaa","entry":"attack_forward","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"benchmarks/photoguard+.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/benchmarks/photoguard%2B.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"79482e8ecae87aaa"}},{"code_sha256_prefix":"229d634611bdc645","entry":"collate_fn","repo":"vinairesearch/anti-dreambooth","repo_kind":"community","path":"train_dreambooth.py","file_url":"https://github.com/vinairesearch/anti-dreambooth/blob/HEAD/train_dreambooth.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"229d634611bdc645"}},{"code_sha256_prefix":"424012cb37b31172","entry":"default","repo":"psyker-team/mist-v2","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/psyker-team/mist-v2/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"424012cb37b31172"}},{"code_sha256_prefix":"7f5f265e8a1e513c","entry":"eval_step","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"evaluation.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7f5f265e8a1e513c"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"psyker-team/mist-v2","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/psyker-team/mist-v2/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"06e4ea094029c19d","entry":"get_images_from_path","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"evaluation.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"06e4ea094029c19d"}},{"code_sha256_prefix":"1a71962efc4d5ae4","entry":"get_images_from_path","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"i2i.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/i2i.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1a71962efc4d5ae4"}},{"code_sha256_prefix":"d793962f13865771","entry":"inpainting_dataloader","repo":"psyker-team/mist-v2","repo_kind":"official","path":"lora_diffusion/cli_lora_pti.py","file_url":"https://github.com/psyker-team/mist-v2/blob/HEAD/lora_diffusion/cli_lora_pti.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d793962f13865771"}},{"code_sha256_prefix":"d72762b700feee6f","entry":"ismap","repo":"psyker-team/mist-v2","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/psyker-team/mist-v2/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d72762b700feee6f"}},{"code_sha256_prefix":"686a5a7fc55acb4e","entry":"load_data","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"attacks/aspl_lora.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/attacks/aspl_lora.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"686a5a7fc55acb4e"}},{"code_sha256_prefix":"a3135d69d30aaba2","entry":"load_data","repo":"psyker-team/mist-v2","repo_kind":"official","path":"attacks/mist.py","file_url":"https://github.com/psyker-team/mist-v2/blob/HEAD/attacks/mist.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a3135d69d30aaba2"}},{"code_sha256_prefix":"aa63f591fb2e9bd7","entry":"text2img_dataloader","repo":"psyker-team/mist-v2","repo_kind":"official","path":"lora_diffusion/cli_lora_pti.py","file_url":"https://github.com/psyker-team/mist-v2/blob/HEAD/lora_diffusion/cli_lora_pti.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa63f591fb2e9bd7"}},{"code_sha256_prefix":"cd576e2f55b71b75","entry":"transform_to_tensor","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"evaluation.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cd576e2f55b71b75"}},{"code_sha256_prefix":"9a299fe5ae09e407","entry":"uniq","repo":"psyker-team/mist-v2","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/psyker-team/mist-v2/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9a299fe5ae09e407"}},{"code_sha256_prefix":"32af1c12fd826ef2","entry":"autocast","repo":"psyker-team/mist-v2","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/psyker-team/mist-v2/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"32af1c12fd826ef2"}},{"code_sha256_prefix":"93c799084cc33809","entry":"compute_grad","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"benchmarks/photoguard+.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/benchmarks/photoguard%2B.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"93c799084cc33809"}},{"code_sha256_prefix":"2f00b39db5a81466","entry":"import_model_class_from_model_name_or_path","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"attacks/aspl_lora.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/attacks/aspl_lora.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2f00b39db5a81466"}},{"code_sha256_prefix":"f9bd2e83191afad1","entry":"log_txt_as_img","repo":"psyker-team/mist-v2","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/psyker-team/mist-v2/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f9bd2e83191afad1"}},{"code_sha256_prefix":"573f16fb8e34edf6","entry":"parse_args","repo":"vinairesearch/anti-dreambooth","repo_kind":"community","path":"train_dreambooth.py","file_url":"https://github.com/vinairesearch/anti-dreambooth/blob/HEAD/train_dreambooth.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"573f16fb8e34edf6"}},{"code_sha256_prefix":"9bd2fe7bf5cdf878","entry":"parse_args","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"attacks/aspl_lora.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/attacks/aspl_lora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9bd2fe7bf5cdf878"}},{"code_sha256_prefix":"65aaa57a4aeee3ff","entry":"parse_args","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"attacks/aspl_lora_m.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/attacks/aspl_lora_m.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"65aaa57a4aeee3ff"}},{"code_sha256_prefix":"fd7f8599a9b1bb7d","entry":"parse_args","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"attacks/target_antidb.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/attacks/target_antidb.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fd7f8599a9b1bb7d"}},{"code_sha256_prefix":"a12d53df966bda20","entry":"parse_args","repo":"caradryanliang/improvedadvdm","repo_kind":"official","path":"benchmarks/advdm.py","file_url":"https://github.com/caradryanliang/improvedadvdm/blob/HEAD/benchmarks/advdm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a12d53df966bda20"}},{"code_sha256_prefix":"a56c4df534ae4170","entry":"parse_args","repo":"psyker-team/mist-v2","repo_kind":"official","path":"attacks/mist.py","file_url":"https://github.com/psyker-team/mist-v2/blob/HEAD/attacks/mist.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a56c4df534ae4170"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}