{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/score-matching-model-for-unbounded-data-score-1","title":"Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimation","arxiv_id":"2106.05527","date":"2021-06-10","proceeding":null,"authors":["Dongjun Kim","Seungjae Shin","Kyungwoo Song","Wanmo Kang","Il-Chul Moon"],"abstract":"Recent advances in diffusion models bring state-of-the-art performance on image generation tasks. However, empirical results from previous research in diffusion models imply an inverse correlation between density estimation and sample generation performances. This paper investigates with sufficient empirical evidence that such inverse correlation happens because density estimation is significantly contributed by small diffusion time, whereas sample generation mainly depends on large diffusion time. However, training a score network well across the entire diffusion time is demanding because the loss scale is significantly imbalanced at each diffusion time. For successful training, therefore, we introduce Soft Truncation, a universally applicable training technique for diffusion models, that softens the fixed and static truncation hyperparameter into a random variable. In experiments, Soft Truncation achieves state-of-the-art performance on CIFAR-10, CelebA, CelebA-HQ 256x256, and STL-10 datasets.","url_abs":"https://arxiv.org/abs/2106.05527v5","url_pdf":"https://arxiv.org/pdf/2106.05527v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"score-matching-model-for-unbounded-data-score-1","repo_url":"https://github.com/Kim-Dongjun/Soft-Truncation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"density-estimation","task_name":"Density Estimation"},{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-celeba-64x64","task":"Image Generation","dataset":"CelebA 64x64","model":"DDPM++ (VP, FID) + ST","rank_in_archive_order":9,"of":39,"metrics":{"FID":"1.9","bits/dimension":"2.1"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-celeba-64x64","task":"Image Generation","dataset":"CelebA 64x64","model":"DDPM++ (VP, NLL) + ST","rank_in_archive_order":17,"of":39,"metrics":{"FID":"2.9","bits/dimension":"1.96"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-celeba-64x64","task":"Image Generation","dataset":"CelebA 64x64","model":"UNCSN++ (RVE) + ST","rank_in_archive_order":38,"of":39,"metrics":{"bits/dimension":"1.97"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-celeba-hq-256x256","task":"Image Generation","dataset":"CelebA-HQ 256x256","model":"UNCSN++ (RVE) + ST","rank_in_archive_order":7,"of":19,"metrics":{"FID":"7.16"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-ffhq-256-x-256","task":"Image Generation","dataset":"FFHQ 256 x 256","model":"UDM (RVE) + ST","rank_in_archive_order":27,"of":51,"metrics":{"FID":"5.54"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-imagenet-32x32","task":"Image Generation","dataset":"ImageNet 32x32","model":"DDPM++ (VP, NLL) + ST","rank_in_archive_order":8,"of":35,"metrics":{"FID":"8.42","Inception score":"11.82","bpd":"3.85"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-lsun-bedroom-256-x-256","task":"Image Generation","dataset":"LSUN Bedroom 256 x 256","model":"UDM (RVE) + ST","rank_in_archive_order":10,"of":32,"metrics":{"FID":"4.57"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-stl-10","task":"Image Generation","dataset":"STL-10","model":"UNCSN++ (RVE) + ST","rank_in_archive_order":2,"of":31,"metrics":{"FID":"7.71","Inception score":"13.43"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.05527","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.05527"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Kim-Dongjun/Soft-Truncation","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"ran":2,"unverified":8},"by_repo_kind":{"official":{"samples":12,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6c419026e778dee2","entry":"get_data_inverse_scaler","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6c419026e778dee2"}},{"code_sha256_prefix":"346d2e6cc8b48a5d","entry":"get_data_scaler","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"346d2e6cc8b48a5d"}},{"code_sha256_prefix":"366bb8c7035a2fc8","entry":"optimization_manager","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"losses.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"366bb8c7035a2fc8"}},{"code_sha256_prefix":"3ac11bfe0ae80195","entry":"variance_scaling","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"models/layers.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/models/layers.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3ac11bfe0ae80195"}},{"code_sha256_prefix":"54a00db9aadf092f","entry":"classifier_fn_from_tfhub","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"evaluation.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/evaluation.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"54a00db9aadf092f"}},{"code_sha256_prefix":"fd14d238ccd5f679","entry":"crop_resize","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fd14d238ccd5f679"}},{"code_sha256_prefix":"db5f935a1d6b4c2c","entry":"get_act","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"models/layers.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/models/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"db5f935a1d6b4c2c"}},{"code_sha256_prefix":"0bfcb6b2d940e393","entry":"get_div_fn","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"likelihood.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/likelihood.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0bfcb6b2d940e393"}},{"code_sha256_prefix":"1b8adb3294e53c6d","entry":"get_normalization","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"models/normalization.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/models/normalization.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1b8adb3294e53c6d"}},{"code_sha256_prefix":"f2382220ad5ac587","entry":"get_optimizer","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"losses.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f2382220ad5ac587"}},{"code_sha256_prefix":"c7b30d41729eeb8e","entry":"load_dataset_stats","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"evaluation.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c7b30d41729eeb8e"}},{"code_sha256_prefix":"85b2235eb7afaa4f","entry":"ncsn_conv1x1","repo":"Kim-Dongjun/Soft-Truncation","repo_kind":"official","path":"models/layers.py","file_url":"https://github.com/Kim-Dongjun/Soft-Truncation/blob/HEAD/models/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"85b2235eb7afaa4f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}