{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/probabilistic-natural-language-generation","title":"Stochastic Wasserstein Autoencoder for Probabilistic Sentence Generation","arxiv_id":"1806.08462","date":"2018-06-22","proceeding":"NAACL 2019 6","authors":["Hareesh Bahuleyan","Lili Mou","Hao Zhou","Olga Vechtomova"],"abstract":"The variational autoencoder (VAE) imposes a probabilistic distribution\n(typically Gaussian) on the latent space and penalizes the Kullback--Leibler\n(KL) divergence between the posterior and prior. In NLP, VAEs are extremely\ndifficult to train due to the problem of KL collapsing to zero. One has to\nimplement various heuristics such as KL weight annealing and word dropout in a\ncarefully engineered manner to successfully train a VAE for text. In this\npaper, we propose to use the Wasserstein autoencoder (WAE) for probabilistic\nsentence generation, where the encoder could be either stochastic or\ndeterministic. We show theoretically and empirically that, in the original WAE,\nthe stochastically encoded Gaussian distribution tends to become a Dirac-delta\nfunction, and we propose a variant of WAE that encourages the stochasticity of\nthe encoder. Experimental results show that the latent space learned by WAE\nexhibits properties of continuity and smoothness as in VAEs, while\nsimultaneously achieving much higher BLEU scores for sentence reconstruction.","url_abs":"http://arxiv.org/abs/1806.08462v2","url_pdf":"http://arxiv.org/pdf/1806.08462v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"probabilistic-natural-language-generation","repo_url":"https://github.com/HareeshBahuleyan/probabilistic_nlg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[{"method_slug":"dropout","method_name":"Dropout"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.08462","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}