{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/prompt-agnostic-adversarial-perturbation-for","title":"Prompt-Agnostic Adversarial Perturbation for Customized Diffusion Models","arxiv_id":"2408.10571","date":"2024-08-20","proceeding":null,"authors":["Cong Wan","Yuhang He","Xiang Song","Yihong Gong"],"abstract":"Diffusion models have revolutionized customized text-to-image generation, allowing for efficient synthesis of photos from personal data with textual descriptions. However, these advancements bring forth risks including privacy breaches and unauthorized replication of artworks. Previous researches primarily center around using prompt-specific methods to generate adversarial examples to protect personal images, yet the effectiveness of existing methods is hindered by constrained adaptability to different prompts. In this paper, we introduce a Prompt-Agnostic Adversarial Perturbation (PAP) method for customized diffusion models. PAP first models the prompt distribution using a Laplace Approximation, and then produces prompt-agnostic perturbations by maximizing a disturbance expectation based on the modeled distribution. This approach effectively tackles the prompt-agnostic attacks, leading to improved defense stability. Extensive experiments in face privacy and artistic style protection, demonstrate the superior generalization of PAP in comparison to existing techniques. Our project page is available at https://github.com/vancyland/Prompt-Agnostic-Adversarial-Perturbation-for-Customized-Diffusion-Models.github.io.","url_abs":"https://arxiv.org/abs/2408.10571v4","url_pdf":"https://arxiv.org/pdf/2408.10571v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"prompt-agnostic-adversarial-perturbation-for","repo_url":"https://github.com/vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"prompt-agnostic-adversarial-perturbation-for","repo_url":"https://github.com/vancyland/vancyland.github.io-project-PAP","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"text-to-image-generation-1","task_name":"Text to Image Generation"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2408.10571","atlas_url":"https://app.syntology.ai/?focus=2408.10571","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.10571"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vancyland/vancyland.github.io-project-PAP","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io","reach":{"status":"ok","spdx":"GPL-3.0"}}],"summary":{"ran_draft_wrong":1,"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"229d634611bdc645","entry":"collate_fn","repo":"vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io","repo_kind":"official","path":"train_dreambooth.py","file_url":"https://github.com/vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io/blob/HEAD/train_dreambooth.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"229d634611bdc645"}},{"code_sha256_prefix":"686a5a7fc55acb4e","entry":"load_data","repo":"vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io","repo_kind":"official","path":"attack/papv1.py","file_url":"https://github.com/vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io/blob/HEAD/attack/papv1.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"686a5a7fc55acb4e"}},{"code_sha256_prefix":"2f00b39db5a81466","entry":"import_model_class_from_model_name_or_path","repo":"vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io","repo_kind":"official","path":"train_dreambooth.py","file_url":"https://github.com/vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io/blob/HEAD/train_dreambooth.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"2f00b39db5a81466"}},{"code_sha256_prefix":"d0510f45e1460c48","entry":"parse_args","repo":"vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io","repo_kind":"official","path":"train_dreambooth.py","file_url":"https://github.com/vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io/blob/HEAD/train_dreambooth.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"d0510f45e1460c48"}},{"code_sha256_prefix":"66e7662142d90765","entry":"parse_args","repo":"vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io","repo_kind":"official","path":"attack/papv1.py","file_url":"https://github.com/vancyland/prompt-agnostic-adversarial-perturbation-for-customized-diffusion-models.github.io/blob/HEAD/attack/papv1.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"66e7662142d90765"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}