{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/discovering-interpretable-directions-in-the","title":"Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models","arxiv_id":"2303.11073","date":"2023-03-20","proceeding":null,"authors":["René Haas","Inbar Huberman-Spiegelglas","Rotem Mulayoff","Stella Graßhof","Sami S. Brandt","Tomer Michaeli"],"abstract":"Denoising Diffusion Models (DDMs) have emerged as a strong competitor to Generative Adversarial Networks (GANs). However, despite their widespread use in image synthesis and editing applications, their latent space is still not as well understood. Recently, a semantic latent space for DDMs, coined `$h$-space', was shown to facilitate semantic image editing in a way reminiscent of GANs. The $h$-space is comprised of the bottleneck activations in the DDM's denoiser across all timesteps of the diffusion process. In this paper, we explore the properties of h-space and propose several novel methods for finding meaningful semantic directions within it. We start by studying unsupervised methods for revealing interpretable semantic directions in pretrained DDMs. Specifically, we show that global latent directions emerge as the principal components in the latent space. Additionally, we provide a novel method for discovering image-specific semantic directions by spectral analysis of the Jacobian of the denoiser w.r.t. the latent code. Next, we extend the analysis by finding directions in a supervised fashion in unconditional DDMs. We demonstrate how such directions can be found by relying on either a labeled data set of real images or by annotating generated samples with a domain-specific attribute classifier. We further show how to semantically disentangle the found direction by simple linear projection. Our approaches are applicable without requiring any architectural modifications, text-based guidance, CLIP-based optimization, or model fine-tuning.","url_abs":"https://arxiv.org/abs/2303.11073v2","url_pdf":"https://arxiv.org/pdf/2303.11073v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"discovering-interpretable-directions-in-the","repo_url":"https://github.com/renhaa/semantic-diffusion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"discovering-interpretable-directions-in-the","repo_url":"https://github.com/JonasLoos/h-space-directions","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.11073","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.11073"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JonasLoos/h-space-directions","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/renhaa/semantic-diffusion","reach":null}],"summary":{"ran_fixture":1,"ran_draft_wrong":1,"ran_violates":1,"unverified":2},"by_repo_kind":{"official":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"42682243422c8c83","entry":"fix_sign_ambiguity","repo":"renhaa/semantic-diffusion","repo_kind":"official","path":"editing/pca.py","file_url":"https://github.com/renhaa/semantic-diffusion/blob/HEAD/editing/pca.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"42682243422c8c83"}},{"code_sha256_prefix":"679959347d2e3c25","entry":"forward_ResnetBlock2D","repo":"renhaa/semantic-diffusion","repo_kind":"official","path":"semanticdiffusion.py","file_url":"https://github.com/renhaa/semantic-diffusion/blob/HEAD/semanticdiffusion.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"679959347d2e3c25"}},{"code_sha256_prefix":"948969a8445d5846","entry":"make_sure_vecs_point_in_same_direction","repo":"renhaa/semantic-diffusion","repo_kind":"official","path":"editing/pca.py","file_url":"https://github.com/renhaa/semantic-diffusion/blob/HEAD/editing/pca.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"948969a8445d5846"}},{"code_sha256_prefix":"e4b95e4cd48e17c5","entry":"make_variations","repo":"renhaa/semantic-diffusion","repo_kind":"official","path":"editing/pca.py","file_url":"https://github.com/renhaa/semantic-diffusion/blob/HEAD/editing/pca.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4b95e4cd48e17c5"}},{"code_sha256_prefix":"ab6d6e56292cdb0e","entry":"mid_block_forward","repo":"renhaa/semantic-diffusion","repo_kind":"official","path":"semanticdiffusion.py","file_url":"https://github.com/renhaa/semantic-diffusion/blob/HEAD/semanticdiffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab6d6e56292cdb0e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}