{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-generative-models-through-the-lens-of","title":"Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections","arxiv_id":"2404.02954","date":"2024-04-03","proceeding":null,"authors":["Gabriel Loaiza-Ganem","Brendan Leigh Ross","Rasa Hosseinzadeh","Anthony L. Caterini","Jesse C. Cresswell"],"abstract":"In recent years there has been increased interest in understanding the interplay between deep generative models (DGMs) and the manifold hypothesis. Research in this area focuses on understanding the reasons why commonly-used DGMs succeed or fail at learning distributions supported on unknown low-dimensional manifolds, as well as developing new models explicitly designed to account for manifold-supported data. This manifold lens provides both clarity as to why some DGMs (e.g. diffusion models and some generative adversarial networks) empirically surpass others (e.g. likelihood-based models such as variational autoencoders, normalizing flows, or energy-based models) at sample generation, and guidance for devising more performant DGMs. We carry out the first survey of DGMs viewed through this lens, making two novel contributions along the way. First, we formally establish that numerical instability of likelihoods in high ambient dimensions is unavoidable when modelling data with low intrinsic dimension. We then show that DGMs on learned representations of autoencoders can be interpreted as approximately minimizing Wasserstein distance: this result, which applies to latent diffusion models, helps justify their outstanding empirical results. The manifold lens provides a rich perspective from which to understand DGMs, and we aim to make this perspective more accessible and widespread.","url_abs":"https://arxiv.org/abs/2404.02954v2","url_pdf":"https://arxiv.org/pdf/2404.02954v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-generative-models-through-the-lens-of","repo_url":"https://github.com/layer6ai-labs/dgm_manifold_survey","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2404.02954","atlas_url":"https://app.syntology.ai/?focus=2404.02954","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.02954"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/layer6ai-labs/dgm_manifold_survey","reach":{"status":"ok"}}],"summary":{"ran":11,"unverified":3},"by_repo_kind":{"official":{"samples":14,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"1faaecb8fc2728aa","entry":"diagonal_gaussian_entropy","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"two_step_zoo/distributions.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/two_step_zoo/distributions.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1faaecb8fc2728aa"}},{"code_sha256_prefix":"cadfe41c1b06b4b5","entry":"diagonal_gaussian_log_prob","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"two_step_zoo/distributions.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/two_step_zoo/distributions.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cadfe41c1b06b4b5"}},{"code_sha256_prefix":"244a06fc14c746be","entry":"diagonal_gaussian_sample","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"two_step_zoo/distributions.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/two_step_zoo/distributions.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"244a06fc14c746be"}},{"code_sha256_prefix":"00dcb5b7f93da666","entry":"get_ae_config","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"config/generalized_autoencoder.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/config/generalized_autoencoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"00dcb5b7f93da666"}},{"code_sha256_prefix":"98585cf6d5df0ba7","entry":"get_arm_config","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"config/density_estimator.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/config/density_estimator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"98585cf6d5df0ba7"}},{"code_sha256_prefix":"12aecb1541f79f36","entry":"get_avb_config","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"config/density_estimator.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/config/density_estimator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"12aecb1541f79f36"}},{"code_sha256_prefix":"3c3e7ed10fe04ce9","entry":"get_avb_config","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"config/generalized_autoencoder.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/config/generalized_autoencoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3c3e7ed10fe04ce9"}},{"code_sha256_prefix":"2152497199b3127d","entry":"get_base_config","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"config/density_estimator.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/config/density_estimator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2152497199b3127d"}},{"code_sha256_prefix":"4daa8c69ffd77817","entry":"get_base_config","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"config/generalized_autoencoder.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/config/generalized_autoencoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4daa8c69ffd77817"}},{"code_sha256_prefix":"9748d473a27987d2","entry":"get_shared_config","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"config/shared_config.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/config/shared_config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9748d473a27987d2"}},{"code_sha256_prefix":"9775fdc8777d63da","entry":"load_configs_from_run_dir","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"config/get_config.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/config/get_config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9775fdc8777d63da"}},{"code_sha256_prefix":"54d15c99cf2d1207","entry":"get_configs","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"config/get_config.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/config/get_config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"54d15c99cf2d1207"}},{"code_sha256_prefix":"d2b6515dae734b5a","entry":"get_single_config","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"config/get_config.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/config/get_config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d2b6515dae734b5a"}},{"code_sha256_prefix":"34f6bc96a60f211e","entry":"get_vae_module","repo":"layer6ai-labs/dgm_manifold_survey","repo_kind":"official","path":"two_step_zoo/factory.py","file_url":"https://github.com/layer6ai-labs/dgm_manifold_survey/blob/HEAD/two_step_zoo/factory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"34f6bc96a60f211e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}