{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/how-do-variational-autoencoders-learn","title":"How do Variational Autoencoders Learn? Insights from Representational Similarity","arxiv_id":"2205.08399","date":"2022-05-17","proceeding":null,"authors":["Lisa Bonheme","Marek Grzes"],"abstract":"The ability of Variational Autoencoders (VAEs) to learn disentangled representations has made them popular for practical applications. However, their behaviour is not yet fully understood. For example, the questions of when they can provide disentangled representations, or suffer from posterior collapse are still areas of active research. Despite this, there are no layerwise comparisons of the representations learned by VAEs, which would further our understanding of these models. In this paper, we thus look into the internal behaviour of VAEs using representational similarity techniques. Specifically, using the CKA and Procrustes similarities, we found that the encoders' representations are learned long before the decoders', and this behaviour is independent of hyperparameters, learning objectives, and datasets. Moreover, the encoders' representations in all but the mean and variance layers are similar across hyperparameters and learning objectives.","url_abs":"https://arxiv.org/abs/2205.08399v3","url_pdf":"https://arxiv.org/pdf/2205.08399v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"how-do-variational-autoencoders-learn","repo_url":"https://github.com/bonheml/vae_learning_dynamics","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"procrustes","method_name":"Procrustes"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2205.08399","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.08399"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bonheml/vae_learning_dynamics","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"65ae13e771f5f0ae","entry":"get_model_epoch","repo":"bonheml/vae_learning_dynamics","repo_kind":"official","path":"vae_ld/learning_dynamics/utils.py","file_url":"https://github.com/bonheml/vae_learning_dynamics/blob/HEAD/vae_ld/learning_dynamics/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"65ae13e771f5f0ae"}},{"code_sha256_prefix":"4f4e2baaa67c6aca","entry":"get_model_filename","repo":"bonheml/vae_learning_dynamics","repo_kind":"official","path":"vae_ld/learning_dynamics/utils.py","file_url":"https://github.com/bonheml/vae_learning_dynamics/blob/HEAD/vae_ld/learning_dynamics/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4f4e2baaa67c6aca"}},{"code_sha256_prefix":"397e437d1de17eca","entry":"gram_rbf","repo":"bonheml/vae_learning_dynamics","repo_kind":"official","path":"vae_ld/learning_dynamics/information_bottleneck.py","file_url":"https://github.com/bonheml/vae_learning_dynamics/blob/HEAD/vae_ld/learning_dynamics/information_bottleneck.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"397e437d1de17eca"}},{"code_sha256_prefix":"fa89a12fa558ad84","entry":"init_model_with_n_latents","repo":"bonheml/vae_learning_dynamics","repo_kind":"official","path":"vae_ld/learning_dynamics/fondue.py","file_url":"https://github.com/bonheml/vae_learning_dynamics/blob/HEAD/vae_ld/learning_dynamics/fondue.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fa89a12fa558ad84"}},{"code_sha256_prefix":"0fcef8dd3f34d457","entry":"mcc","repo":"bonheml/vae_learning_dynamics","repo_kind":"official","path":"vae_ld/learning_dynamics/mcc.py","file_url":"https://github.com/bonheml/vae_learning_dynamics/blob/HEAD/vae_ld/learning_dynamics/mcc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0fcef8dd3f34d457"}},{"code_sha256_prefix":"82160aff46ed416e","entry":"normalise","repo":"bonheml/vae_learning_dynamics","repo_kind":"official","path":"vae_ld/learning_dynamics/information_bottleneck.py","file_url":"https://github.com/bonheml/vae_learning_dynamics/blob/HEAD/vae_ld/learning_dynamics/information_bottleneck.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"82160aff46ed416e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}