{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploring-diffusion-time-steps-for","title":"Exploring Diffusion Time-steps for Unsupervised Representation Learning","arxiv_id":"2401.11430","date":"2024-01-21","proceeding":null,"authors":["Zhongqi Yue","Jiankun Wang","Qianru Sun","Lei Ji","Eric I-Chao Chang","Hanwang Zhang"],"abstract":"Representation learning is all about discovering the hidden modular attributes that generate the data faithfully. We explore the potential of Denoising Diffusion Probabilistic Model (DM) in unsupervised learning of the modular attributes. We build a theoretical framework that connects the diffusion time-steps and the hidden attributes, which serves as an effective inductive bias for unsupervised learning. Specifically, the forward diffusion process incrementally adds Gaussian noise to samples at each time-step, which essentially collapses different samples into similar ones by losing attributes, e.g., fine-grained attributes such as texture are lost with less noise added (i.e., early time-steps), while coarse-grained ones such as shape are lost by adding more noise (i.e., late time-steps). To disentangle the modular attributes, at each time-step t, we learn a t-specific feature to compensate for the newly lost attribute, and the set of all 1,...,t-specific features, corresponding to the cumulative set of lost attributes, are trained to make up for the reconstruction error of a pre-trained DM at time-step t. On CelebA, FFHQ, and Bedroom datasets, the learned feature significantly improves attribute classification and enables faithful counterfactual generation, e.g., interpolating only one specified attribute between two images, validating the disentanglement quality. Codes are in https://github.com/yue-zhongqi/diti.","url_abs":"https://arxiv.org/abs/2401.11430v1","url_pdf":"https://arxiv.org/pdf/2401.11430v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploring-diffusion-time-steps-for","repo_url":"https://github.com/yue-zhongqi/diti","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"inductive-bias","task_name":"Inductive Bias"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":null,"task_name":"counterfactual"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.11430","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.11430"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/yue-zhongqi/diti","reach":null}],"summary":{"ran_draft_wrong":2,"ran_honours":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"cb50d9dcd12b3ef7","entry":"calculate_theta","repo":"yue-zhongqi/diti","repo_kind":"official","path":"eval/counterfactual_generation.py","file_url":"https://github.com/yue-zhongqi/diti/blob/HEAD/eval/counterfactual_generation.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb50d9dcd12b3ef7"}},{"code_sha256_prefix":"75dd1f00429c0ac5","entry":"lerp","repo":"yue-zhongqi/diti","repo_kind":"official","path":"eval/counterfactual_generation.py","file_url":"https://github.com/yue-zhongqi/diti/blob/HEAD/eval/counterfactual_generation.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"75dd1f00429c0ac5"}},{"code_sha256_prefix":"3f52e7a30971cc76","entry":"slerp","repo":"yue-zhongqi/diti","repo_kind":"official","path":"eval/counterfactual_generation.py","file_url":"https://github.com/yue-zhongqi/diti/blob/HEAD/eval/counterfactual_generation.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3f52e7a30971cc76"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}