{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-conditioned-image-generation-via","title":"Return of Unconditional Generation: A Self-supervised Representation Generation Method","arxiv_id":"2312.03701","date":"2023-12-06","proceeding":null,"authors":["Tianhong Li","Dina Katabi","Kaiming He"],"abstract":"Unconditional generation -- the problem of modeling data distribution without relying on human-annotated labels -- is a long-standing and fundamental challenge in generative models, creating a potential of learning from large-scale unlabeled data. In the literature, the generation quality of an unconditional method has been much worse than that of its conditional counterpart. This gap can be attributed to the lack of semantic information provided by labels. In this work, we show that one can close this gap by generating semantic representations in the representation space produced by a self-supervised encoder. These representations can be used to condition the image generator. This framework, called Representation-Conditioned Generation (RCG), provides an effective solution to the unconditional generation problem without using labels. Through comprehensive experiments, we observe that RCG significantly improves unconditional generation quality: e.g., it achieves a new state-of-the-art FID of 2.15 on ImageNet 256x256, largely reducing the previous best of 5.91 by a relative 64%. Our unconditional results are situated in the same tier as the leading class-conditional ones. We hope these encouraging observations will attract the community's attention to the fundamental problem of unconditional generation. Code is available at https://github.com/LTH14/rcg.","url_abs":"https://arxiv.org/abs/2312.03701v4","url_pdf":"https://arxiv.org/pdf/2312.03701v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-conditioned-image-generation-via","repo_url":"https://github.com/LTH14/rcg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"conditional-image-generation","task_name":"Conditional Image Generation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"unconditional-image-generation","task_name":"Unconditional Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-imagenet-256x256","task":"Image Generation","dataset":"ImageNet 256x256","model":"RCG-L (w/o guidance)","rank_in_archive_order":71,"of":94,"metrics":{"FID":"3.49"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.03701","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.03701"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/LTH14/rcg","reach":null}],"summary":{"ran":9,"ran_draft_wrong":3,"unverified":6},"by_repo_kind":{"official":{"samples":18,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"33aedbcfd18d75e0","entry":"Attention","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"33aedbcfd18d75e0"}},{"code_sha256_prefix":"63c52efe80c94cb3","entry":"BertEmbeddings","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63c52efe80c94cb3"}},{"code_sha256_prefix":"607e78af8a011f8d","entry":"Block","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"607e78af8a011f8d"}},{"code_sha256_prefix":"950818a420758dbe","entry":"DDIMSampler","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"950818a420758dbe"}},{"code_sha256_prefix":"9c54ac910ba566df","entry":"Decoder","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c54ac910ba566df"}},{"code_sha256_prefix":"2deab9d7e8ce0dcd","entry":"Encoder","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2deab9d7e8ce0dcd"}},{"code_sha256_prefix":"72ec0347648af855","entry":"MlmLayer","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"72ec0347648af855"}},{"code_sha256_prefix":"30fa834a4a883839","entry":"ResnetBlock","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"30fa834a4a883839"}},{"code_sha256_prefix":"9d2f5c397afb9f4b","entry":"VectorQuantizer","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9d2f5c397afb9f4b"}},{"code_sha256_prefix":"43b98b8b55d12a3d","entry":"load_pretrained_deit","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43b98b8b55d12a3d"}},{"code_sha256_prefix":"c6c33009466a11bc","entry":"load_pretrained_ibot","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c6c33009466a11bc"}},{"code_sha256_prefix":"606b6d93861e426d","entry":"load_pretrained_moco","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"606b6d93861e426d"}},{"code_sha256_prefix":"928ed9eea191af30","entry":"MaskedGenerativeEncoderViT","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"928ed9eea191af30"}},{"code_sha256_prefix":"5ff5b3c39cac8e4d","entry":"VQModel","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ff5b3c39cac8e4d"}},{"code_sha256_prefix":"43c459d4c7f134b2","entry":"load_model","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43c459d4c7f134b2"}},{"code_sha256_prefix":"bf19fad78c0d01dd","entry":"load_model_from_config","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf19fad78c0d01dd"}},{"code_sha256_prefix":"751c98281389ad11","entry":"load_pretrained_dino","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"751c98281389ad11"}},{"code_sha256_prefix":"92679b16417e54ec","entry":"mask_by_random_topk","repo":"LTH14/rcg","repo_kind":"official","path":"pixel_generator/mage/models_mage.py","file_url":"https://github.com/LTH14/rcg/blob/HEAD/pixel_generator/mage/models_mage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"92679b16417e54ec"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}