{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generated-loss-augmented-training-and","title":"Generated Loss, Augmented Training, and Multiscale VAE","arxiv_id":"1904.10446","date":"2019-04-23","proceeding":null,"authors":["Jason Chou","Gautam Hathi"],"abstract":"The variational autoencoder (VAE) framework remains a popular option for\ntraining unsupervised generative models, especially for discrete data where\ngenerative adversarial networks (GANs) require workaround to create gradient\nfor the generator. In our work modeling US postal addresses, we show that our\ndiscrete VAE with tree recursive architecture demonstrates limited capability\nof capturing field correlations within structured data, even after overcoming\nthe challenge of posterior collapse with scheduled sampling and tuning of the\nKL-divergence weight $\\beta$. Worse, VAE seems to have difficulty mapping its\ngenerated samples to the latent space, as their VAE loss lags behind or even\nincreases during the training process. Motivated by this observation, we show\nthat augmenting training data with generated variants (augmented training) and\ntraining a VAE with multiple values of $\\beta$ simultaneously (multiscale VAE)\nboth improve the generation quality of VAE. Despite their differences in\nmotivation and emphasis, we show that augmented training and multiscale VAE are\nactually connected and have similar effects on the model.","url_abs":"http://arxiv.org/abs/1904.10446v1","url_pdf":"http://arxiv.org/pdf/1904.10446v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generated-loss-augmented-training-and","repo_url":"https://github.com/EIFY/vermont_address","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}