{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/advpaint-protecting-images-from-inpainting","title":"AdvPaint: Protecting Images from Inpainting Manipulation via Adversarial Attention Disruption","arxiv_id":"2503.10081","date":"2025-03-13","proceeding":null,"authors":["Joonsung Jeon","Woo Jae Kim","Suhyeon Ha","Sooel Son","Sung-Eui Yoon"],"abstract":"The outstanding capability of diffusion models in generating high-quality images poses significant threats when misused by adversaries. In particular, we assume malicious adversaries exploiting diffusion models for inpainting tasks, such as replacing a specific region with a celebrity. While existing methods for protecting images from manipulation in diffusion-based generative models have primarily focused on image-to-image and text-to-image tasks, the challenge of preventing unauthorized inpainting has been rarely addressed, often resulting in suboptimal protection performance. To mitigate inpainting abuses, we propose ADVPAINT, a novel defensive framework that generates adversarial perturbations that effectively disrupt the adversary's inpainting tasks. ADVPAINT targets the self- and cross-attention blocks in a target diffusion inpainting model to distract semantic understanding and prompt interactions during image generation. ADVPAINT also employs a two-stage perturbation strategy, dividing the perturbation region based on an enlarged bounding box around the object, enhancing robustness across diverse masks of varying shapes and sizes. Our experimental results demonstrate that ADVPAINT's perturbations are highly effective in disrupting the adversary's inpainting tasks, outperforming existing methods; ADVPAINT attains over a 100-point increase in FID and substantial decreases in precision.","url_abs":"https://arxiv.org/abs/2503.10081v1","url_pdf":"https://arxiv.org/pdf/2503.10081v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"advpaint-protecting-images-from-inpainting","repo_url":"https://github.com/joonsungjeon/advpaint","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"pixel-prediction","method_name":"Inpainting"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2503.10081","atlas_url":"https://app.syntology.ai/?focus=2503.10081","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.10081"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/JoonsungJeon/AdvPaint","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/joonsungjeon/advpaint","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"90fadec0aec03c1b","entry":"collate_fn","repo":"joonsungjeon/advpaint","repo_kind":"official","path":"AdvPaint.py","file_url":"https://github.com/joonsungjeon/advpaint/blob/HEAD/AdvPaint.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"90fadec0aec03c1b"}},{"code_sha256_prefix":"b194fd93d187cadf","entry":"hook_fn","repo":"joonsungjeon/advpaint","repo_kind":"official","path":"AdvPaint.py","file_url":"https://github.com/joonsungjeon/advpaint/blob/HEAD/AdvPaint.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b194fd93d187cadf"}},{"code_sha256_prefix":"99a4a64e057a4ce4","entry":"register_cross_attention_hook","repo":"joonsungjeon/advpaint","repo_kind":"official","path":"AdvPaint.py","file_url":"https://github.com/joonsungjeon/advpaint/blob/HEAD/AdvPaint.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99a4a64e057a4ce4"}},{"code_sha256_prefix":"0cf3b50f33f2d43b","entry":"pgd_SelfQKV_And_Cross_Xadv","repo":"joonsungjeon/advpaint","repo_kind":"official","path":"AdvPaint.py","file_url":"https://github.com/joonsungjeon/advpaint/blob/HEAD/AdvPaint.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0cf3b50f33f2d43b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}