{"url":"/method/residual-normal-distribution","slug":"residual-normal-distribution","name":"Residual Normal Distribution","full_name":"Residual Normal Distribution","full_name_withheld":false,"description_markdown":"**Residual Normal Distributions** are used to help the optimization of VAEs, preventing optimization from entering an unstable region. This can happen due to sharp gradients caused in situations where the encoder and decoder produce distributions far away from each other. The residual distribution parameterizes $q\\left(\\mathbf{z}|\\mathbf{x}\\right)$ relative to $p\\left(\\mathbf{z}\\right)$. Let $p\\left(z^{i}\\_{l}|\\mathbf{z}\\_{<l}\\right) := N \\left(\\mu\\_{i}\\left(\\mathbf{z}\\_{<l}\\right), \\sigma\\_{i}\\left(\\mathbf{z}\\_{<l}\\right)\\right)$ be a Normal distribution for the $i$th variable in $\\mathbf{z}\\_{l}$ in prior. Define $q\\left(z^{i}\\_{l}|\\mathbf{z}\\_{<l}, x\\right) := N\\left(\\mu\\_{i}\\left(\\mathbf{z}\\_{<l}\\right) + \\Delta\\mu\\_{i}\\left(\\mathbf{z}\\_{<l}, x\\right), \\sigma\\_{i}\\left(\\mathbf{z}\\_{<l}\\right) \\cdot \\Delta\\sigma\\_{i}\\left(\\mathbf{z}\\_{<l}, x\\right) \\right)$, where $\\Delta\\mu\\_{i}\\left(\\mathbf{z}\\_{<l}, \\mathbf{x}\\right)$ and $\\Delta\\sigma\\_{i}\\left(\\mathbf{z}\\_{<l}, \\mathbf{x}\\right)$ are the relative location and scale of the approximate posterior with respect to the prior. With this parameterization, when the prior moves, the approximate posterior moves accordingly, if not changed.","description_state":"present","introduced_year":null,"introduced_by":{"title":"NVAE: A Deep Hierarchical Variational Autoencoder","paper":"/paper/nvae-a-deep-hierarchical-variational","first_author":"Arash Vahdat","n_authors":2,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/nvae-a-deep-hierarchical-variational"},"source":{"url":"https://arxiv.org/abs/2007.03898v3","title":"NVAE: A Deep Hierarchical Variational Autoencoder","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Variational Optimization","url":"/methods/category/variational-optimization","pwc_aliases":[]}],"n_papers_tagged":5,"archive_num_papers":5,"papers_newest_first":[{"paper":null,"title":"A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck","date":"2022-07-27","arxiv_id":"2207.13529","n_code_links":0,"syntology":null},{"paper":"/paper/defending-variational-autoencoders-from","title":"Alleviating Adversarial Attacks on Variational Autoencoders with MCMC","date":"2022-03-18","arxiv_id":"2203.09940","n_code_links":1,"syntology":null},{"paper":"/paper/polarity-sampling-quality-and-diversity","title":"Polarity Sampling: Quality and Diversity Control of Pre-Trained Generative Networks via Singular Values","date":"2022-03-03","arxiv_id":"2203.01993","n_code_links":1,"syntology":null},{"paper":null,"title":"NVAE-GAN Based Approach for Unsupervised Time Series Anomaly Detection","date":"2021-01-08","arxiv_id":"2101.02908","n_code_links":0,"syntology":null},{"paper":"/paper/nvae-a-deep-hierarchical-variational","title":"NVAE: A Deep Hierarchical Variational Autoencoder","date":"2020-07-08","arxiv_id":"2007.03898","n_code_links":10,"syntology":{"ran":23,"of":41,"unverified":18,"pointer_only":23}}],"papers_shown":5,"tasks":[{"task":"/task/image-generation","name":"Image Generation","papers":2},{"task":"/task/adversarial-attack","name":"Adversarial Attack","papers":1},{"task":"/task/anomaly-detection","name":"Anomaly Detection","papers":1},{"task":"/task/decoder","name":"Decoder","papers":1},{"task":"/task/diversity","name":"Diversity","papers":1},{"task":"/task/time-series-1","name":"Time Series","papers":1},{"task":"/task/time-series","name":"Time Series Analysis","papers":1},{"task":"/task/time-series-anomaly-detection","name":"Time Series Anomaly Detection","papers":1},{"task":"/task/unconditional-image-generation","name":"Unconditional Image Generation","papers":1}],"tasks_shown":9,"n_tasks":9,"usage_by_year":[{"year":"2020","papers":1},{"year":"2021","papers":1},{"year":"2022","papers":3}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/residual-normal-distribution"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}