{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-the-stability-of-diffusion-models","title":"Improving the Stability and Efficiency of Diffusion Models for Content Consistent Super-Resolution","arxiv_id":"2401.00877","date":"2023-12-30","proceeding":null,"authors":["Lingchen Sun","Rongyuan Wu","Jie Liang","Zhengqiang Zhang","Hongwei Yong","Lei Zhang"],"abstract":"The generative priors of pre-trained latent diffusion models (DMs) have demonstrated great potential to enhance the visual quality of image super-resolution (SR) results. However, the noise sampling process in DMs introduces randomness in the SR outputs, and the generated contents can differ a lot with different noise samples. The multi-step diffusion process can be accelerated by distilling methods, but the generative capacity is difficult to control. To address these issues, we analyze the respective advantages of DMs and generative adversarial networks (GANs) and propose to partition the generative SR process into two stages, where the DM is employed for reconstructing image structures and the GAN is employed for improving fine-grained details. Specifically, we propose a non-uniform timestep sampling strategy in the first stage. A single timestep sampling is first applied to extract the coarse information from the input image, then a few reverse steps are used to reconstruct the main structures. In the second stage, we finetune the decoder of the pre-trained variational auto-encoder by adversarial GAN training for deterministic detail enhancement. Once trained, our proposed method, namely content consistent super-resolution (CCSR),allows flexible use of different diffusion steps in the inference stage without re-training. Extensive experiments show that with 2 or even 1 diffusion step, CCSR can significantly improve the content consistency of SR outputs while keeping high perceptual quality. Codes and models can be found at \\href{https://github.com/csslc/CCSR}{https://github.com/csslc/CCSR}.","url_abs":"https://arxiv.org/abs/2401.00877v2","url_pdf":"https://arxiv.org/pdf/2401.00877v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improving-the-stability-of-diffusion-models","repo_url":"https://github.com/csslc/ccsr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"improving-the-stability-of-diffusion-models","repo_url":"https://github.com/cswry/osediff","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-super-resolution","task_name":"Image Super-Resolution"},{"task_slug":"super-resolution","task_name":"Super-Resolution"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.00877","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.00877"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/csslc/ccsr","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cswry/osediff","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_honours":2,"ran":7,"ran_fixture":1,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":12,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"8bc0bbc72282d3f8","entry":"parse_float_list","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":2,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"8bc0bbc72282d3f8"}},{"code_sha256_prefix":"b2d5ad9f0fef3e14","entry":"parse_int_list","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":2,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"b2d5ad9f0fef3e14"}},{"code_sha256_prefix":"d5662fd75b69cda4","entry":"DiffAugment","repo":"csslc/ccsr","repo_kind":"official","path":"models/DiffAugment.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/models/DiffAugment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d5662fd75b69cda4"}},{"code_sha256_prefix":"a091d19d483bafe2","entry":"format_time","repo":"csslc/ccsr","repo_kind":"official","path":"ADD/dnnlib/util.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/ADD/dnnlib/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a091d19d483bafe2"}},{"code_sha256_prefix":"556dd2c77ff90670","entry":"format_time_brief","repo":"csslc/ccsr","repo_kind":"official","path":"ADD/dnnlib/util.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/ADD/dnnlib/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"556dd2c77ff90670"}},{"code_sha256_prefix":"92726cd76c52caca","entry":"forward_flex","repo":"csslc/ccsr","repo_kind":"official","path":"models/vit_utils.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/models/vit_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"92726cd76c52caca"}},{"code_sha256_prefix":"b9e2f79c13245bdc","entry":"get_activation","repo":"csslc/ccsr","repo_kind":"official","path":"models/vit_utils.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/models/vit_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b9e2f79c13245bdc"}},{"code_sha256_prefix":"bd5a05ce886da4ce","entry":"image_grid","repo":"csslc/ccsr","repo_kind":"official","path":"train_ccsr_stage1.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/train_ccsr_stage1.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bd5a05ce886da4ce"}},{"code_sha256_prefix":"fa3b3cc4f07c4a0f","entry":"parse_str_list","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"fa3b3cc4f07c4a0f"}},{"code_sha256_prefix":"37b50aac63f46854","entry":"rand_brightness","repo":"csslc/ccsr","repo_kind":"official","path":"models/DiffAugment.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/models/DiffAugment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"37b50aac63f46854"}},{"code_sha256_prefix":"db65cf4b9c6f357e","entry":"rand_saturation","repo":"csslc/ccsr","repo_kind":"official","path":"models/DiffAugment.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/models/DiffAugment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"db65cf4b9c6f357e"}},{"code_sha256_prefix":"9d31d2c4cd16bb2d","entry":"ask_yes_no","repo":"csslc/ccsr","repo_kind":"official","path":"ADD/dnnlib/util.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/ADD/dnnlib/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9d31d2c4cd16bb2d"}},{"code_sha256_prefix":"5f2544125330bf84","entry":"forward_vit","repo":"csslc/ccsr","repo_kind":"official","path":"models/vit_utils.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/models/vit_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5f2544125330bf84"}},{"code_sha256_prefix":"2f00b39db5a81466","entry":"import_model_class_from_model_name_or_path","repo":"csslc/ccsr","repo_kind":"official","path":"train_ccsr_stage1.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/train_ccsr_stage1.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2f00b39db5a81466"}},{"code_sha256_prefix":"da94debb8019ad46","entry":"zero_module","repo":"csslc/ccsr","repo_kind":"official","path":"models/controlnet.py","file_url":"https://github.com/csslc/ccsr/blob/HEAD/models/controlnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"da94debb8019ad46"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}