{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-information-autoencoding-family-a","title":"The Information Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Models","arxiv_id":"1806.06514","date":"2018-06-18","proceeding":null,"authors":["Shengjia Zhao","Jiaming Song","Stefano Ermon"],"abstract":"A large number of objectives have been proposed to train latent variable\ngenerative models. We show that many of them are Lagrangian dual functions of\nthe same primal optimization problem. The primal problem optimizes the mutual\ninformation between latent and visible variables, subject to the constraints of\naccurately modeling the data distribution and performing correct amortized\ninference. By choosing to maximize or minimize mutual information, and choosing\ndifferent Lagrange multipliers, we obtain different objectives including\nInfoGAN, ALI/BiGAN, ALICE, CycleGAN, beta-VAE, adversarial autoencoders, AVB,\nAS-VAE and InfoVAE. Based on this observation, we provide an exhaustive\ncharacterization of the statistical and computational trade-offs made by all\nthe training objectives in this class of Lagrangian duals. Next, we propose a\ndual optimization method where we optimize model parameters as well as the\nLagrange multipliers. This method achieves Pareto optimal solutions in terms of\noptimizing information and satisfying the constraints.","url_abs":"http://arxiv.org/abs/1806.06514v2","url_pdf":"http://arxiv.org/pdf/1806.06514v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-information-autoencoding-family-a","repo_url":"https://github.com/ermongroup/lagvae","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"the-information-autoencoding-family-a","repo_url":"https://github.com/BerenMillidge/LagrangianVAE.jl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"cycle-consistency-loss","method_name":"Cycle Consistency Loss"},{"method_slug":"gan-least-squares-loss","method_name":"GAN Least Squares Loss"},{"method_slug":"instance-normalization","method_name":"Instance Normalization"},{"method_slug":"patchgan","method_name":"PatchGAN"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1806.06514","atlas_url":"https://app.syntology.ai/?focus=1806.06514","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}