{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unsupervised-representation-learning-from-pre","title":"Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models","arxiv_id":"2212.12990","date":"2022-12-26","proceeding":null,"authors":["Zijian Zhang","Zhou Zhao","Zhijie Lin"],"abstract":"Diffusion Probabilistic Models (DPMs) have shown a powerful capacity of generating high-quality image samples. Recently, diffusion autoencoders (Diff-AE) have been proposed to explore DPMs for representation learning via autoencoding. Their key idea is to jointly train an encoder for discovering meaningful representations from images and a conditional DPM as the decoder for reconstructing images. Considering that training DPMs from scratch will take a long time and there have existed numerous pre-trained DPMs, we propose \\textbf{P}re-trained \\textbf{D}PM \\textbf{A}uto\\textbf{E}ncoding (\\textbf{PDAE}), a general method to adapt existing pre-trained DPMs to the decoders for image reconstruction, with better training efficiency and performance than Diff-AE. Specifically, we find that the reason that pre-trained DPMs fail to reconstruct an image from its latent variables is due to the information loss of forward process, which causes a gap between their predicted posterior mean and the true one. From this perspective, the classifier-guided sampling method can be explained as computing an extra mean shift to fill the gap, reconstructing the lost class information in samples. These imply that the gap corresponds to the lost information of the image, and we can reconstruct the image by filling the gap. Drawing inspiration from this, we employ a trainable model to predict a mean shift according to encoded representation and train it to fill as much gap as possible, in this way, the encoder is forced to learn as much information as possible from images to help the filling. By reusing a part of network of pre-trained DPMs and redesigning the weighting scheme of diffusion loss, PDAE can learn meaningful representations from images efficiently. Extensive experiments demonstrate the effectiveness, efficiency and flexibility of PDAE.","url_abs":"https://arxiv.org/abs/2212.12990v3","url_pdf":"https://arxiv.org/pdf/2212.12990v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unsupervised-representation-learning-from-pre","repo_url":"https://github.com/ckczzj/pdae","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"unsupervised-representation-learning-from-pre","repo_url":"https://github.com/GabrieleLozupone/LDAE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"unsupervised-representation-learning-from-pre","repo_url":"https://github.com/yue-zhongqi/diti","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-reconstruction","task_name":"Image Reconstruction"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"fail","method_name":"fail"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2212.12990","atlas_url":"https://app.syntology.ai/?focus=2212.12990","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.12990"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ckczzj/PDAE","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yue-zhongqi/diti","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/GabrieleLozupone/LDAE","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ckczzj/pdae","reach":null}],"summary":{"ran":4,"ran_draft_wrong":1,"unverified":9},"by_repo_kind":{"official":{"samples":10,"ran":4,"repositories":1},"listed":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"580dc76645bca9ff","entry":"DDIM","repo":"yue-zhongqi/diti","repo_kind":"listed","path":"trainer/representation_learning_trainer.py","file_url":"https://github.com/yue-zhongqi/diti/blob/HEAD/trainer/representation_learning_trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"580dc76645bca9ff"}},{"code_sha256_prefix":"afd1425583888760","entry":"QKVAttention","repo":"ckczzj/pdae","repo_kind":"official","path":"model/shift_unet.py","file_url":"https://github.com/ckczzj/pdae/blob/HEAD/model/shift_unet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"afd1425583888760"}},{"code_sha256_prefix":"ec9213c0c8138849","entry":"QKVAttentionLegacy","repo":"ckczzj/pdae","repo_kind":"official","path":"model/shift_unet.py","file_url":"https://github.com/ckczzj/pdae/blob/HEAD/model/shift_unet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ec9213c0c8138849"}},{"code_sha256_prefix":"7778cda2e40a2497","entry":"TimestepBlock","repo":"ckczzj/pdae","repo_kind":"official","path":"model/shift_unet.py","file_url":"https://github.com/ckczzj/pdae/blob/HEAD/model/shift_unet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7778cda2e40a2497"}},{"code_sha256_prefix":"d5eb4721bd7ede86","entry":"normalization","repo":"ckczzj/pdae","repo_kind":"official","path":"model/shift_unet.py","file_url":"https://github.com/ckczzj/pdae/blob/HEAD/model/shift_unet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d5eb4721bd7ede86"}},{"code_sha256_prefix":"b7379a086999f359","entry":"AttentionBlock","repo":"ckczzj/pdae","repo_kind":"official","path":"model/shift_unet.py","file_url":"https://github.com/ckczzj/pdae/blob/HEAD/model/shift_unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b7379a086999f359"}},{"code_sha256_prefix":"4169604fb1fae5cb","entry":"BaseTrainer","repo":"yue-zhongqi/diti","repo_kind":"listed","path":"trainer/representation_learning_trainer.py","file_url":"https://github.com/yue-zhongqi/diti/blob/HEAD/trainer/representation_learning_trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4169604fb1fae5cb"}},{"code_sha256_prefix":"efd4bc02cd02d1cf","entry":"GaussianDiffusion","repo":"yue-zhongqi/diti","repo_kind":"listed","path":"trainer/representation_learning_trainer.py","file_url":"https://github.com/yue-zhongqi/diti/blob/HEAD/trainer/representation_learning_trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"efd4bc02cd02d1cf"}},{"code_sha256_prefix":"3792fabf4fa7e452","entry":"RepresentationLearningTrainer","repo":"yue-zhongqi/diti","repo_kind":"listed","path":"trainer/representation_learning_trainer.py","file_url":"https://github.com/yue-zhongqi/diti/blob/HEAD/trainer/representation_learning_trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3792fabf4fa7e452"}},{"code_sha256_prefix":"6a2fbcc09ba19930","entry":"ResBlock","repo":"ckczzj/pdae","repo_kind":"official","path":"model/shift_unet.py","file_url":"https://github.com/ckczzj/pdae/blob/HEAD/model/shift_unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6a2fbcc09ba19930"}},{"code_sha256_prefix":"cb9834d54ca93700","entry":"ResBlockShift","repo":"ckczzj/pdae","repo_kind":"official","path":"model/shift_unet.py","file_url":"https://github.com/ckczzj/pdae/blob/HEAD/model/shift_unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb9834d54ca93700"}},{"code_sha256_prefix":"629e72d126a0dd83","entry":"ShiftUNet","repo":"ckczzj/pdae","repo_kind":"official","path":"model/shift_unet.py","file_url":"https://github.com/ckczzj/pdae/blob/HEAD/model/shift_unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"629e72d126a0dd83"}},{"code_sha256_prefix":"76666159788ad649","entry":"TimestepContextBlock","repo":"ckczzj/pdae","repo_kind":"official","path":"model/shift_unet.py","file_url":"https://github.com/ckczzj/pdae/blob/HEAD/model/shift_unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"76666159788ad649"}},{"code_sha256_prefix":"546963d82095d48c","entry":"TimestepSequential","repo":"ckczzj/pdae","repo_kind":"official","path":"model/shift_unet.py","file_url":"https://github.com/ckczzj/pdae/blob/HEAD/model/shift_unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"546963d82095d48c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}