{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/2505-11245","title":"Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models","arxiv_id":"2505.11245","date":"2025-05-16","proceeding":null,"authors":["Fu-Yun Wang","Yunhao Shui","Jingtan Piao","Keqiang Sun","Hongsheng Li"],"abstract":"Diffusion models have made substantial advances in image generation, yet models trained on large, unfiltered datasets often yield outputs misaligned with human preferences. Numerous methods have been proposed to fine-tune pre-trained diffusion models, achieving notable improvements in aligning generated outputs with human preferences. However, we argue that existing preference alignment methods neglect the critical role of handling unconditional/negative-conditional outputs, leading to a diminished capacity to avoid generating undesirable outcomes. This oversight limits the efficacy of classifier-free guidance~(CFG), which relies on the contrast between conditional generation and unconditional/negative-conditional generation to optimize output quality. In response, we propose a straightforward but versatile effective approach that involves training a model specifically attuned to negative preferences. This method does not require new training strategies or datasets but rather involves minor modifications to existing techniques. Our approach integrates seamlessly with models such as SD1.5, SDXL, video diffusion models and models that have undergone preference optimization, consistently enhancing their alignment with human preferences.","url_abs":"https://arxiv.org/abs/2505.11245v1","url_pdf":"https://arxiv.org/pdf/2505.11245v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"2505-11245","repo_url":"https://github.com/g-u-n/diffusion-npo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2505.11245","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.11245"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/G-U-N/Diffusion-NPO","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/g-u-n/diffusion-npo","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"92a985b9cf3cd4ee","entry":"_get_variance","repo":"g-u-n/diffusion-npo","repo_kind":"official","path":"RL-SPO-NPO/spo/custom_diffusers/ddim_with_logprob.py","file_url":"https://github.com/g-u-n/diffusion-npo/blob/HEAD/RL-SPO-NPO/spo/custom_diffusers/ddim_with_logprob.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"92a985b9cf3cd4ee"}},{"code_sha256_prefix":"5923770195ed4c9e","entry":"_left_broadcast","repo":"g-u-n/diffusion-npo","repo_kind":"official","path":"RL-SPO-NPO/spo/custom_diffusers/ddim_with_logprob.py","file_url":"https://github.com/g-u-n/diffusion-npo/blob/HEAD/RL-SPO-NPO/spo/custom_diffusers/ddim_with_logprob.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5923770195ed4c9e"}},{"code_sha256_prefix":"b4fa76cf341de453","entry":"ddim_step_with_logprob","repo":"g-u-n/diffusion-npo","repo_kind":"official","path":"RL-SPO-NPO/spo/custom_diffusers/ddim_with_logprob.py","file_url":"https://github.com/g-u-n/diffusion-npo/blob/HEAD/RL-SPO-NPO/spo/custom_diffusers/ddim_with_logprob.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b4fa76cf341de453"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}