{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficientdm-efficient-quantization-aware-fine","title":"EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models","arxiv_id":"2310.03270","date":"2023-10-05","proceeding":null,"authors":["Yefei He","Jing Liu","Weijia Wu","Hong Zhou","Bohan Zhuang"],"abstract":"Diffusion models have demonstrated remarkable capabilities in image synthesis and related generative tasks. Nevertheless, their practicality for real-world applications is constrained by substantial computational costs and latency issues. Quantization is a dominant way to compress and accelerate diffusion models, where post-training quantization (PTQ) and quantization-aware training (QAT) are two main approaches, each bearing its own properties. While PTQ exhibits efficiency in terms of both time and data usage, it may lead to diminished performance in low bit-width. On the other hand, QAT can alleviate performance degradation but comes with substantial demands on computational and data resources. In this paper, we introduce a data-free and parameter-efficient fine-tuning framework for low-bit diffusion models, dubbed EfficientDM, to achieve QAT-level performance with PTQ-like efficiency. Specifically, we propose a quantization-aware variant of the low-rank adapter (QALoRA) that can be merged with model weights and jointly quantized to low bit-width. The fine-tuning process distills the denoising capabilities of the full-precision model into its quantized counterpart, eliminating the requirement for training data. We also introduce scale-aware optimization and temporal learned step-size quantization to further enhance performance. Extensive experimental results demonstrate that our method significantly outperforms previous PTQ-based diffusion models while maintaining similar time and data efficiency. Specifically, there is only a 0.05 sFID increase when quantizing both weights and activations of LDM-4 to 4-bit on ImageNet 256x256. Compared to QAT-based methods, our EfficientDM also boasts a 16.2x faster quantization speed with comparable generation quality. Code is available at \\href{https://github.com/ThisisBillhe/EfficientDM}{this hrl}.","url_abs":"https://arxiv.org/abs/2310.03270v4","url_pdf":"https://arxiv.org/pdf/2310.03270v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficientdm-efficient-quantization-aware-fine","repo_url":"https://github.com/thisisbillhe/efficientdm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"quantization","task_name":"Quantization"},{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[{"method_slug":"adapter","method_name":"Adapter"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.03270","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.03270"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ThisisBillhe/EfficientDM","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thisisbillhe/efficientdm","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":4,"ran":3,"ran_violates":2,"ran_fixture":1,"unverified":4},"by_repo_kind":{"official":{"samples":14,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"675da8641c602b03","entry":"round_ste","repo":"ThisisBillhe/EfficientDM","repo_kind":"official","path":"quant_scripts/quant_layer.py","file_url":"https://github.com/ThisisBillhe/EfficientDM/blob/HEAD/quant_scripts/quant_layer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"675da8641c602b03"}},{"code_sha256_prefix":"f9c070e6f07834d6","entry":"UniformAffineQuantizer","repo":"thisisbillhe/efficientdm","repo_kind":"official","path":"quant_scripts/quant_layer.py","file_url":"https://github.com/thisisbillhe/efficientdm/blob/HEAD/quant_scripts/quant_layer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9c070e6f07834d6"}},{"code_sha256_prefix":"fe5dd5258046898c","entry":"always","repo":"ThisisBillhe/EfficientDM","repo_kind":"official","path":"ldm/modules/x_transformer.py","file_url":"https://github.com/ThisisBillhe/EfficientDM/blob/HEAD/ldm/modules/x_transformer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe5dd5258046898c"}},{"code_sha256_prefix":"424012cb37b31172","entry":"default","repo":"ThisisBillhe/EfficientDM","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/ThisisBillhe/EfficientDM/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"424012cb37b31172"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"ThisisBillhe/EfficientDM","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/ThisisBillhe/EfficientDM/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"8dc011d26607845d","entry":"get_train_samples","repo":"ThisisBillhe/EfficientDM","repo_kind":"official","path":"quant_scripts/train_efficientdm.py","file_url":"https://github.com/ThisisBillhe/EfficientDM/blob/HEAD/quant_scripts/train_efficientdm.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8dc011d26607845d"}},{"code_sha256_prefix":"b1368330cf0f5642","entry":"isimage","repo":"ThisisBillhe/EfficientDM","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/ThisisBillhe/EfficientDM/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1368330cf0f5642"}},{"code_sha256_prefix":"d72762b700feee6f","entry":"ismap","repo":"ThisisBillhe/EfficientDM","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/ThisisBillhe/EfficientDM/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d72762b700feee6f"}},{"code_sha256_prefix":"08c6ea7576220a13","entry":"lp_loss","repo":"ThisisBillhe/EfficientDM","repo_kind":"official","path":"quant_scripts/quant_layer.py","file_url":"https://github.com/ThisisBillhe/EfficientDM/blob/HEAD/quant_scripts/quant_layer.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"08c6ea7576220a13"}},{"code_sha256_prefix":"9a299fe5ae09e407","entry":"uniq","repo":"ThisisBillhe/EfficientDM","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/ThisisBillhe/EfficientDM/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a299fe5ae09e407"}},{"code_sha256_prefix":"16a46c30de77204f","entry":"QuantModule_intnlora","repo":"thisisbillhe/efficientdm","repo_kind":"official","path":"quant_scripts/quant_layer.py","file_url":"https://github.com/thisisbillhe/efficientdm/blob/HEAD/quant_scripts/quant_layer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"16a46c30de77204f"}},{"code_sha256_prefix":"b800779b8d0bedcb","entry":"TemporalActivationQuantizer","repo":"thisisbillhe/efficientdm","repo_kind":"official","path":"quant_scripts/quant_layer.py","file_url":"https://github.com/thisisbillhe/efficientdm/blob/HEAD/quant_scripts/quant_layer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b800779b8d0bedcb"}},{"code_sha256_prefix":"80ae154054c233d5","entry":"getStep","repo":"ThisisBillhe/EfficientDM","repo_kind":"official","path":"ldm/globalvar.py","file_url":"https://github.com/ThisisBillhe/EfficientDM/blob/HEAD/ldm/globalvar.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"80ae154054c233d5"}},{"code_sha256_prefix":"f9bd2e83191afad1","entry":"log_txt_as_img","repo":"ThisisBillhe/EfficientDM","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/ThisisBillhe/EfficientDM/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9bd2e83191afad1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}