{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-rectifying-diffusion-sampling-with","title":"Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance","arxiv_id":"2403.17377","date":"2024-03-26","proceeding":null,"authors":["Donghoon Ahn","Hyoungwon Cho","Jaewon Min","Wooseok Jang","Jungwoo Kim","SeonHwa Kim","Hyun Hee Park","Kyong Hwan Jin","Seungryong Kim"],"abstract":"Recent studies have demonstrated that diffusion models are capable of generating high-quality samples, but their quality heavily depends on sampling guidance techniques, such as classifier guidance (CG) and classifier-free guidance (CFG). These techniques are often not applicable in unconditional generation or in various downstream tasks such as image restoration. In this paper, we propose a novel sampling guidance, called Perturbed-Attention Guidance (PAG), which improves diffusion sample quality across both unconditional and conditional settings, achieving this without requiring additional training or the integration of external modules. PAG is designed to progressively enhance the structure of samples throughout the denoising process. It involves generating intermediate samples with degraded structure by substituting selected self-attention maps in diffusion U-Net with an identity matrix, by considering the self-attention mechanisms' ability to capture structural information, and guiding the denoising process away from these degraded samples. In both ADM and Stable Diffusion, PAG surprisingly improves sample quality in conditional and even unconditional scenarios. Moreover, PAG significantly improves the baseline performance in various downstream tasks where existing guidances such as CG or CFG cannot be fully utilized, including ControlNet with empty prompts and image restoration such as inpainting and deblurring.","url_abs":"https://arxiv.org/abs/2403.17377v1","url_pdf":"https://arxiv.org/pdf/2403.17377v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-rectifying-diffusion-sampling-with","repo_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"self-rectifying-diffusion-sampling-with","repo_url":"https://github.com/susunghong/seg-sdxl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"self-rectifying-diffusion-sampling-with","repo_url":"https://github.com/Min-Jaewon/PSLD_PAG","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"deblurring","task_name":"Deblurring"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-restoration","task_name":"Image Restoration"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"pag","method_name":"PAG"},{"method_slug":"pag","method_name":"Perturbed-Attention Guidance"}],"datasets_introduced":[],"methods_introduced":[{"slug":"pag","name":"PAG","full_name":"Perturbed-Attention Guidance"},{"slug":"perturbed-attention-guidance","name":"Perturbed-Attention Guidance","full_name":"Perturbed-Attention Guidance"}],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.17377","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.17377"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/susunghong/seg-sdxl","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Min-Jaewon/PSLD_PAG","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"ran":7,"ran_honours":3,"unverified":3},"by_repo_kind":{"official":{"samples":16,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8ab9a97df8161f31","entry":"center_crop_arr","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/image_datasets.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/image_datasets.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8ab9a97df8161f31"}},{"code_sha256_prefix":"cfd76fd0d89574a4","entry":"approx_standard_normal_cdf","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/losses.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/losses.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cfd76fd0d89574a4"}},{"code_sha256_prefix":"e2497fcb1db5da74","entry":"attention_from_qkv","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/unet.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/unet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e2497fcb1db5da74"}},{"code_sha256_prefix":"2ab2316ac6fdd869","entry":"betas_for_alpha_bar","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/gaussian_diffusion.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2ab2316ac6fdd869"}},{"code_sha256_prefix":"cd33283d615fb3d7","entry":"discretized_gaussian_log_likelihood","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/losses.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/losses.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd33283d615fb3d7"}},{"code_sha256_prefix":"a086d6286a40b889","entry":"get_named_beta_schedule","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/gaussian_diffusion.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a086d6286a40b889"}},{"code_sha256_prefix":"e41367ad14ff58fd","entry":"get_param_groups_and_shapes","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/fp16_util.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/fp16_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e41367ad14ff58fd"}},{"code_sha256_prefix":"44b1aa06ec13a1ff","entry":"get_short_layer_names_str","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"classifier_sample.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/classifier_sample.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"44b1aa06ec13a1ff"}},{"code_sha256_prefix":"e20dd5102da3b050","entry":"make_master_params","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/fp16_util.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/fp16_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e20dd5102da3b050"}},{"code_sha256_prefix":"bcd8b4acab199405","entry":"make_output_format","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/logger.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/logger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bcd8b4acab199405"}},{"code_sha256_prefix":"cf2798b666b231ca","entry":"normal_kl","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/losses.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/losses.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf2798b666b231ca"}},{"code_sha256_prefix":"25d9a8be93bc84f5","entry":"save_input_hook","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/unet.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/unet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"25d9a8be93bc84f5"}},{"code_sha256_prefix":"64fff1e30802b815","entry":"unflatten_master_params","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/fp16_util.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/fp16_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"64fff1e30802b815"}},{"code_sha256_prefix":"e515a67f7f32e76d","entry":"mpi_weighted_mean","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/logger.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/logger.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e515a67f7f32e76d"}},{"code_sha256_prefix":"0c6607473a4c4c55","entry":"profile","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/logger.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/logger.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c6607473a4c4c55"}},{"code_sha256_prefix":"53d5be4fdbcfac23","entry":"random_crop_arr","repo":"KU-CVLAB/Perturbed-Attention-Guidance","repo_kind":"official","path":"guided_diffusion/image_datasets.py","file_url":"https://github.com/KU-CVLAB/Perturbed-Attention-Guidance/blob/HEAD/guided_diffusion/image_datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53d5be4fdbcfac23"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}