{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/stochastic-conditional-diffusion-models-for","title":"Stochastic Conditional Diffusion Models for Robust Semantic Image Synthesis","arxiv_id":"2402.16506","date":"2024-02-26","proceeding":null,"authors":["Juyeon Ko","Inho Kong","Dogyun Park","Hyunwoo J. Kim"],"abstract":"Semantic image synthesis (SIS) is a task to generate realistic images corresponding to semantic maps (labels). However, in real-world applications, SIS often encounters noisy user inputs. To address this, we propose Stochastic Conditional Diffusion Model (SCDM), which is a robust conditional diffusion model that features novel forward and generation processes tailored for SIS with noisy labels. It enhances robustness by stochastically perturbing the semantic label maps through Label Diffusion, which diffuses the labels with discrete diffusion. Through the diffusion of labels, the noisy and clean semantic maps become similar as the timestep increases, eventually becoming identical at $t=T$. This facilitates the generation of an image close to a clean image, enabling robust generation. Furthermore, we propose a class-wise noise schedule to differentially diffuse the labels depending on the class. We demonstrate that the proposed method generates high-quality samples through extensive experiments and analyses on benchmark datasets, including a novel experimental setup simulating human errors during real-world applications. Code is available at https://github.com/mlvlab/SCDM.","url_abs":"https://arxiv.org/abs/2402.16506v3","url_pdf":"https://arxiv.org/pdf/2402.16506v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"stochastic-conditional-diffusion-models-for","repo_url":"https://github.com/mlvlab/scdm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"conditional-image-generation","task_name":"Conditional Image Generation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-to-image-translation","task_name":"Image-to-Image Translation"},{"task_slug":"noisy-semantic-image-synthesis","task_name":"Noisy Semantic Image Synthesis"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[{"slug":"noisy-ade20k-ds","name":"noisy-ADE20K-DS","full_name":""},{"slug":"noisy-ade20k-edge","name":"noisy-ADE20K-Edge","full_name":""},{"slug":"noisy-ade20k-random","name":"noisy-ADE20K-Random","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/conditional-image-generation-on-celebamask-hq","task":"Conditional Image Generation","dataset":"CelebAMask-HQ","model":"SCDM","rank_in_archive_order":1,"of":1,"metrics":{"FID":"17.4","LPIPS":"0.418","mIoU":"77.2"},"uses_additional_data":false},{"leaderboard":"/sota/image-to-image-translation-on-ade20k-labels","task":"Image-to-Image Translation","dataset":"ADE20K Labels-to-Photos","model":"SCDM","rank_in_archive_order":3,"of":16,"metrics":{"FID":"26.9","LPIPS":"0.530","mIoU":"49.4"},"uses_additional_data":false},{"leaderboard":"/sota/image-to-image-translation-on-coco-stuff","task":"Image-to-Image Translation","dataset":"COCO-Stuff Labels-to-Photos","model":"SCDM","rank_in_archive_order":3,"of":15,"metrics":{"FID":"15.3","LPIPS":"0.519","mIoU":"38.1"},"uses_additional_data":false},{"leaderboard":"/sota/noisy-semantic-image-synthesis-on-noisy","task":"Noisy Semantic Image Synthesis","dataset":"noisy-ADE20K-DS","model":"SCDM","rank_in_archive_order":1,"of":1,"metrics":{"FID":"32.4","mIoU":"44.7"},"uses_additional_data":false},{"leaderboard":"/sota/noisy-semantic-image-synthesis-on-noisy-1","task":"Noisy Semantic Image Synthesis","dataset":"noisy-ADE20K-Edge","model":"SCDM","rank_in_archive_order":1,"of":1,"metrics":{"FID":"31.2","mIoU":"40.1"},"uses_additional_data":false},{"leaderboard":"/sota/noisy-semantic-image-synthesis-on-noisy-2","task":"Noisy Semantic Image Synthesis","dataset":"noisy-ADE20K-Random","model":"SCDM","rank_in_archive_order":1,"of":1,"metrics":{"FID":"28.1","mIoU":"45.1"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.16506","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.16506"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/mlvlab/SCDM","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mlvlab/scdm","reach":null}],"summary":{"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2093f8abdf7b5c3e","entry":"SPADEGroupNorm","repo":"mlvlab/scdm","repo_kind":"official","path":"guided_diffusion/unet.py","file_url":"https://github.com/mlvlab/scdm/blob/HEAD/guided_diffusion/unet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2093f8abdf7b5c3e"}},{"code_sha256_prefix":"f857d1d316bcb04b","entry":"CondTimestepBlock","repo":"mlvlab/scdm","repo_kind":"official","path":"guided_diffusion/unet.py","file_url":"https://github.com/mlvlab/scdm/blob/HEAD/guided_diffusion/unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f857d1d316bcb04b"}},{"code_sha256_prefix":"5823d8bc8de0e785","entry":"SCDResBlock","repo":"mlvlab/scdm","repo_kind":"official","path":"guided_diffusion/unet.py","file_url":"https://github.com/mlvlab/scdm/blob/HEAD/guided_diffusion/unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5823d8bc8de0e785"}},{"code_sha256_prefix":"0def50a351111624","entry":"calculate_frechet_distance","repo":"mlvlab/SCDM","repo_kind":"official","path":"evaluations/fid/tests_with_FID.py","file_url":"https://github.com/mlvlab/SCDM/blob/HEAD/evaluations/fid/tests_with_FID.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0def50a351111624"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}