{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/soda-bottleneck-diffusion-models-for","title":"SODA: Bottleneck Diffusion Models for Representation Learning","arxiv_id":"2311.17901","date":"2023-11-29","proceeding":"CVPR 2024 1","authors":["Drew A. Hudson","Daniel Zoran","Mateusz Malinowski","Andrew K. Lampinen","Andrew Jaegle","James L. McClelland","Loic Matthey","Felix Hill","Alexander Lerchner"],"abstract":"We introduce SODA, a self-supervised diffusion model, designed for representation learning. The model incorporates an image encoder, which distills a source view into a compact representation, that, in turn, guides the generation of related novel views. We show that by imposing a tight bottleneck between the encoder and a denoising decoder, and leveraging novel view synthesis as a self-supervised objective, we can turn diffusion models into strong representation learners, capable of capturing visual semantics in an unsupervised manner. To the best of our knowledge, SODA is the first diffusion model to succeed at ImageNet linear-probe classification, and, at the same time, it accomplishes reconstruction, editing and synthesis tasks across a wide range of datasets. Further investigation reveals the disentangled nature of its emergent latent space, that serves as an effective interface to control and manipulate the model's produced images. All in all, we aim to shed light on the exciting and promising potential of diffusion models, not only for image generation, but also for learning rich and robust representations.","url_abs":"https://arxiv.org/abs/2311.17901v1","url_pdf":"https://arxiv.org/pdf/2311.17901v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"soda-bottleneck-diffusion-models-for","repo_url":"https://github.com/futurexiang/soda","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"linear-probe-classification","task_name":"Linear-Probe Classification"},{"task_slug":"novel-view-synthesis","task_name":"Novel View Synthesis"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.17901","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.17901"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/futurexiang/soda","reach":null}],"summary":{"ran_honours":3},"by_repo_kind":{"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"6dea3711db7301ac","entry":"linear_beta_schedule","repo":"futurexiang/soda","repo_kind":"listed","path":"model/SODA.py","file_url":"https://github.com/futurexiang/soda/blob/HEAD/model/SODA.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6dea3711db7301ac"}},{"code_sha256_prefix":"a1229a9f4df127e2","entry":"normalize_to_neg_one_to_one","repo":"futurexiang/soda","repo_kind":"listed","path":"model/SODA.py","file_url":"https://github.com/futurexiang/soda/blob/HEAD/model/SODA.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a1229a9f4df127e2"}},{"code_sha256_prefix":"97e1bbd232dfe092","entry":"unnormalize_to_zero_to_one","repo":"futurexiang/soda","repo_kind":"listed","path":"model/SODA.py","file_url":"https://github.com/futurexiang/soda/blob/HEAD/model/SODA.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"97e1bbd232dfe092"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}