{"url":"/method/beta-vae","slug":"beta-vae","name":"Beta-VAE","full_name":"Beta-VAE","full_name_withheld":false,"description_markdown":"**Beta-VAE** is a type of variational autoencoder that seeks to discover disentangled latent factors. It modifies [VAEs](https://paperswithcode.com/method/vae) with an adjustable hyperparameter $\\beta$ that balances latent channel capacity and independence constraints with reconstruction accuracy. The idea is to maximize the probability of generating the real data while keeping the distance between the real and estimated distributions small, under a threshold $\\epsilon$. We can use the Kuhn-Tucker conditions to write this as a single equation:\r\n\r\n$$ \\mathcal{F}\\left(\\theta, \\phi, \\beta; \\mathbf{x}, \\mathbf{z}\\right) = \\mathbb{E}\\_{q\\_{\\phi}\\left(\\mathbf{z}|\\mathbf{x}\\right)}\\left[\\log{p}\\_{\\theta}\\left(\\mathbf{x}\\mid\\mathbf{z}\\right)\\right] - \\beta\\left[D\\_{KL}\\left(\\log{q}\\_{\\theta}\\left(\\mathbf{z}\\mid\\mathbf{x}\\right)||p\\left(\\mathbf{z}\\right)\\right) - \\epsilon\\right]$$\r\n\r\nwhere the KKT multiplier $\\beta$ is the regularization coefficient that constrains the capacity of the latent channel $\\mathbf{z}$ and puts implicit independence pressure on the learnt posterior due to the isotropic nature of the Gaussian prior $p\\left(\\mathbf{z}\\right)$.\r\n\r\nWe write this again using the complementary slackness assumption to get the Beta-VAE formulation:\r\n\r\n$$ \\mathcal{F}\\left(\\theta, \\phi, \\beta; \\mathbf{x}, \\mathbf{z}\\right) \\geq  \\mathcal{L}\\left(\\theta, \\phi, \\beta; \\mathbf{x}, \\mathbf{z}\\right) = \\mathbb{E}\\_{q\\_{\\phi}\\left(\\mathbf{z}|\\mathbf{x}\\right)}\\left[\\log{p}\\_{\\theta}\\left(\\mathbf{x}\\mid\\mathbf{z}\\right)\\right] - \\beta\\{D}\\_{KL}\\left(\\log{q}\\_{\\theta}\\left(\\mathbf{z}\\mid\\mathbf{x}\\right)||p\\left(\\mathbf{z}\\right)\\right)$$","description_state":"present","introduced_year":null,"introduced_by":{"title":"beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework","paper":"/paper/beta-vae-learning-basic-visual-concepts-with","first_author":"Irina Higgins","n_authors":8,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/beta-vae-learning-basic-visual-concepts-with"},"source":{"url":"https://openreview.net/forum?id=Sy2fzU9gl","title":"beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/AntixK/PyTorch-VAE/blob/master/models/beta_vae.py","code_snippet_url_on_a_code_host":true,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Likelihood-Based Generative Models","url":"/methods/category/likelihood-based-generative-models","pwc_aliases":[]},{"area":"Computer Vision","area_id":"computer-vision","collection":"Generative Models","url":"/methods/category/generative-models","pwc_aliases":[]}],"n_papers_tagged":30,"archive_num_papers":30,"papers_newest_first":[{"paper":null,"title":"Causal Intervention Framework for Variational Auto Encoder Mechanistic Interpretability","date":"2025-05-06","arxiv_id":"2505.03530","n_code_links":0,"syntology":null},{"paper":null,"title":"Alternatives of Unsupervised Representations of Variables on the Latent Space","date":"2024-10-26","arxiv_id":"2410.20172","n_code_links":0,"syntology":null},{"paper":null,"title":"Comparison of Autoencoder Encodings for ECG Representation in Downstream Prediction Tasks","date":"2024-10-03","arxiv_id":"2410.02937","n_code_links":0,"syntology":null},{"paper":"/paper/lewis-s-signaling-game-as-beta-vae-for","title":"Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments","date":"2023-11-08","arxiv_id":"2311.04453","n_code_links":1,"syntology":null},{"paper":null,"title":"From Conceptual Spaces to Quantum Concepts: Formalising and Learning Structured Conceptual Models","date":"2023-11-06","arxiv_id":"2401.08585","n_code_links":0,"syntology":null},{"paper":"/paper/learning-minimal-representations-of","title":"Learning minimal representations of stochastic processes with variational autoencoders","date":"2023-07-21","arxiv_id":"2307.11608","n_code_links":1,"syntology":null},{"paper":null,"title":"Impact of Disentanglement on Pruning Neural Networks","date":"2023-07-19","arxiv_id":"2307.09994","n_code_links":0,"syntology":null},{"paper":"/paper/identification-of-rare-cortical-folding","title":"Identification of Rare Cortical Folding Patterns using Unsupervised Deep Learning","date":"2022-11-29","arxiv_id":"2211.16213","n_code_links":1,"syntology":null},{"paper":null,"title":"Analysis of Master Vein Attacks on Finger Vein Recognition Systems","date":"2022-10-18","arxiv_id":"2210.10667","n_code_links":0,"syntology":null},{"paper":"/paper/topicvae-topic-aware-disentanglement","title":"TopicVAE: Topic-aware Disentanglement Representation Learning for Enhanced Recommendation","date":"2022-10-10","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"The Conceptual VAE","date":"2022-03-21","arxiv_id":"2203.11216","n_code_links":0,"syntology":null},{"paper":null,"title":"Designing Complex Experiments by Applying Unsupervised Machine Learning","date":"2021-09-29","arxiv_id":"2110.01458","n_code_links":0,"syntology":null},{"paper":"/paper/be-more-active-understanding-the-differences","title":"Be More Active! Understanding the Differences between Mean and Sampled Representations of Variational Autoencoders","date":"2021-09-26","arxiv_id":"2109.12679","n_code_links":1,"syntology":null},{"paper":"/paper/finding-simplicity-unsupervised-discovery-of","title":"Finding simplicity: unsupervised discovery of features, patterns, and order parameters via shift-invariant variational autoencoders","date":"2021-06-23","arxiv_id":"2106.12472","n_code_links":1,"syntology":null},{"paper":"/paper/recovering-barabsi-albert-parameters-of","title":"Recovering Barabási-Albert Parameters of Graphs through Disentanglement","date":"2021-05-03","arxiv_id":"2105.00997","n_code_links":1,"syntology":null},{"paper":null,"title":"Learning Stable Representations with Full Encoder","date":"2021-03-25","arxiv_id":"2103.14082","n_code_links":0,"syntology":null},{"paper":"/paper/dynamic-b-vaes-for-quantifying-biodiversity","title":"Dynamic $β$-VAEs for quantifying biodiversity by clustering optically recorded insect signals","date":"2021-02-10","arxiv_id":"2102.05526","n_code_links":1,"syntology":null},{"paper":null,"title":"Autoencoding Slow Representations for Semi-supervised Data Efficient Regression","date":"2020-12-11","arxiv_id":"2012.06279","n_code_links":0,"syntology":null},{"paper":null,"title":"AI Discovering a Coordinate System of Chemical Elements: Dual Representation by Variational Autoencoders","date":"2020-11-24","arxiv_id":"2011.12090","n_code_links":0,"syntology":null},{"paper":null,"title":"Unsupervised anomaly localization using VAE and beta-VAE","date":"2020-05-19","arxiv_id":"2005.10686","n_code_links":0,"syntology":null},{"paper":null,"title":"AI Giving Back to Statistics? Discovery of the Coordinate System of Univariate Distributions by Beta Variational Autoencoder","date":"2020-04-06","arxiv_id":"2004.02687","n_code_links":0,"syntology":null},{"paper":"/paper/variational-learning-with-disentanglement","title":"Variational Learning with Disentanglement-PyTorch","date":"2019-12-11","arxiv_id":"1912.05184","n_code_links":1,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":1}},{"paper":null,"title":"Information bottleneck through variational glasses","date":"2019-12-02","arxiv_id":"1912.00830","n_code_links":0,"syntology":null},{"paper":null,"title":"Flatsomatic: A Method for Compression of Somatic Mutation Profiles in Cancer","date":"2019-11-27","arxiv_id":"1911.13259","n_code_links":0,"syntology":null},{"paper":null,"title":"A Preliminary Study of Disentanglement With Insights on the Inadequacy of Metrics","date":"2019-11-26","arxiv_id":"1911.11791","n_code_links":0,"syntology":null},{"paper":null,"title":"Disentangling Interpretable Generative Parameters of Random and Real-World Graphs","date":"2019-10-12","arxiv_id":"1910.05639","n_code_links":0,"syntology":null},{"paper":"/paper/ib-gan-disentangled-representation-learning","title":"IB-GAN: Disentangled Representation Learning with Information Bottleneck GAN","date":"2019-05-01","arxiv_id":null,"n_code_links":2,"syntology":null},{"paper":null,"title":"ISA-VAE: Independent Subspace Analysis with Variational Autoencoders","date":"2019-05-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"The Information-Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Modeling","date":"2018-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/beta-vae-learning-basic-visual-concepts-with","title":"beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework","date":"2017-04-26","arxiv_id":null,"n_code_links":6,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/disentanglement","name":"Disentanglement","papers":14},{"task":"/task/representation-learning","name":"Representation Learning","papers":5},{"task":"/task/clustering","name":"Clustering","papers":2},{"task":"/task/data-compression","name":"Data Compression","papers":2},{"task":"/task/graph-generation","name":"Graph Generation","papers":2},{"task":"/task/variational-inference","name":"Variational Inference","papers":2},{"task":"/task/anomaly-detection","name":"Anomaly Detection","papers":1},{"task":"/task/anomaly-localization","name":"Anomaly Localization","papers":1},{"task":"/task/machine-learning","name":"BIG-bench Machine Learning","papers":1},{"task":"/task/data-visualization","name":"Data Visualization","papers":1},{"task":"/task/decoder","name":"Decoder","papers":1},{"task":"/task/deep-learning","name":"Deep Learning","papers":1},{"task":"/task/finger-vein-recognition","name":"Finger Vein Recognition","papers":1},{"task":"/task/graph-embedding","name":"Graph Embedding","papers":1},{"task":"/task/graph-neural-network","name":"Graph Neural Network","papers":1},{"task":"/task/graph-representation-learning","name":"Graph Representation Learning","papers":1},{"task":"/task/model-compression","name":"Model Compression","papers":1},{"task":"/task/novelty-detection","name":"Novelty Detection","papers":1},{"task":"/task/recommendation-systems","name":"Recommendation Systems","papers":1},{"task":"/task/scheduling","name":"Scheduling","papers":1}],"tasks_shown":20,"n_tasks":22,"usage_by_year":[{"year":"2017","papers":1},{"year":"2018","papers":1},{"year":"2019","papers":7},{"year":"2020","papers":4},{"year":"2021","papers":6},{"year":"2022","papers":4},{"year":"2023","papers":4},{"year":"2024","papers":2},{"year":"2025","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/beta-vae"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}