{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/understanding-variational-autoencoders-with","title":"Understanding Variational Autoencoders with Intrinsic Dimension and Information Imbalance","arxiv_id":"2411.01978","date":"2024-11-04","proceeding":null,"authors":["Charles Camboulin","Diego Doimo","Aldo Glielmo"],"abstract":"This work presents an analysis of the hidden representations of Variational Autoencoders (VAEs) using the Intrinsic Dimension (ID) and the Information Imbalance (II). We show that VAEs undergo a transition in behaviour once the bottleneck size is larger than the ID of the data, manifesting in a double hunchback ID profile and a qualitative shift in information processing as captured by the II. Our results also highlight two distinct training phases for architectures with sufficiently large bottleneck sizes, consisting of a rapid fit and a slower generalisation, as assessed by a differentiated behaviour of ID, II, and KL loss. These insights demonstrate that II and ID could be valuable tools for aiding architecture search, for diagnosing underfitting in VAEs, and, more broadly, they contribute to advancing a unified understanding of deep generative models through geometric analysis.","url_abs":"https://arxiv.org/abs/2411.01978v1","url_pdf":"https://arxiv.org/pdf/2411.01978v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"understanding-variational-autoencoders-with","repo_url":"https://github.com/bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"AGPL-3.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.01978","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.01978"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance","reach":{"status":"ok","spdx":"AGPL-3.0"}}],"summary":{"unverified":8},"by_repo_kind":{"official":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"5b90856055b90f54","entry":"calculate_fid","repo":"bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance","repo_kind":"official","path":"vae_framework/utils/analysing_utils.py","file_url":"https://github.com/bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance/blob/HEAD/vae_framework/utils/analysing_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"5b90856055b90f54"}},{"code_sha256_prefix":"fe5b96c6576586d2","entry":"discretize","repo":"bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance","repo_kind":"official","path":"vae_framework/utils/dataset_utils.py","file_url":"https://github.com/bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance/blob/HEAD/vae_framework/utils/dataset_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"fe5b96c6576586d2"}},{"code_sha256_prefix":"caaf112d4d73df3b","entry":"evaluate_vae","repo":"bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance","repo_kind":"official","path":"vae_framework/utils/vae_utils.py","file_url":"https://github.com/bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance/blob/HEAD/vae_framework/utils/vae_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"caaf112d4d73df3b"}},{"code_sha256_prefix":"c7d6a4312e4a147f","entry":"get_data","repo":"bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance","repo_kind":"official","path":"vae_framework/get_all_ids.py","file_url":"https://github.com/bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance/blob/HEAD/vae_framework/get_all_ids.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"c7d6a4312e4a147f"}},{"code_sha256_prefix":"07b9caa85b6f8a22","entry":"load_dataset","repo":"bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance","repo_kind":"official","path":"vae_framework/utils/dataset_utils.py","file_url":"https://github.com/bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance/blob/HEAD/vae_framework/utils/dataset_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"07b9caa85b6f8a22"}},{"code_sha256_prefix":"1fddcd2f74893296","entry":"numpy_to_pil","repo":"bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance","repo_kind":"official","path":"vae_framework/utils/analysing_utils.py","file_url":"https://github.com/bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance/blob/HEAD/vae_framework/utils/analysing_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"1fddcd2f74893296"}},{"code_sha256_prefix":"388c2cd7ce8fa68e","entry":"stack_and_save_layers","repo":"bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance","repo_kind":"official","path":"vae_framework/utils/analysing_utils.py","file_url":"https://github.com/bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance/blob/HEAD/vae_framework/utils/analysing_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"388c2cd7ce8fa68e"}},{"code_sha256_prefix":"da65a6a3f0289a5b","entry":"train_vae","repo":"bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance","repo_kind":"official","path":"vae_framework/utils/vae_utils.py","file_url":"https://github.com/bancaditalia/understanding-variational-autoencoders-with-intrinsic-dimension-and-information-imbalance/blob/HEAD/vae_framework/utils/vae_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AGPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"da65a6a3f0289a5b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}