{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/disentangling-latent-space-for-vae-by-label","title":"Disentangling Latent Space for VAE by Label Relevant/Irrelevant Dimensions","arxiv_id":"1812.09502","date":"2018-12-22","proceeding":"CVPR 2019 6","authors":["Zhilin Zheng","Li Sun"],"abstract":"VAE requires the standard Gaussian distribution as a prior in the latent\nspace. Since all codes tend to follow the same prior, it often suffers the\nso-called \"posterior collapse\". To avoid this, this paper introduces the class\nspecific distribution for the latent code. But different from CVAE, we present\na method for disentangling the latent space into the label relevant and\nirrelevant dimensions, $\\bm{\\mathrm{z}}_s$ and $\\bm{\\mathrm{z}}_u$, for a\nsingle input. We apply two separated encoders to map the input into\n$\\bm{\\mathrm{z}}_s$ and $\\bm{\\mathrm{z}}_u$ respectively, and then give the\nconcatenated code to the decoder to reconstruct the input. The label irrelevant\ncode $\\bm{\\mathrm{z}}_u$ represent the common characteristics of all inputs,\nhence they are constrained by the standard Gaussian, and their encoder is\ntrained in amortized variational inference way, like VAE. While\n$\\bm{\\mathrm{z}}_s$ is assumed to follow the Gaussian mixture distribution in\nwhich each component corresponds to a particular class. The parameters for the\nGaussian components in $\\bm{\\mathrm{z}}_s$ encoder are optimized by the label\nsupervision in a global stochastic way. In theory, we show that our method is\nactually equivalent to adding a KL divergence term on the joint distribution of\n$\\bm{\\mathrm{z}}_s$ and the class label $c$, and it can directly increase the\nmutual information between $\\bm{\\mathrm{z}}_s$ and the label $c$. Our model can\nalso be extended to GAN by adding a discriminator in the pixel domain so that\nit produces high quality and diverse images.","url_abs":"http://arxiv.org/abs/1812.09502v4","url_pdf":"http://arxiv.org/pdf/1812.09502v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"disentangling-latent-space-for-vae-by-label","repo_url":"https://github.com/ZhilZheng/Lr-LiVAE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1812.09502","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}