{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/maskgan-better-text-generation-via-filling-in-1","title":"MaskGAN: Better Text Generation via Filling in the _______","arxiv_id":null,"date":"2018-01-01","proceeding":"ICLR 2018 1","authors":["William Fedus","Ian Goodfellow","Andrew M. Dai"],"abstract":"Neural text generation models are often autoregressive language models or seq2seq models. Neural autoregressive and seq2seq models that generate text by sampling words sequentially, with each word conditioned on the previous model, are state-of-the-art for several machine translation and summarization benchmarks. These benchmarks are often defined by validation perplexity even though this is not a direct measure of sample quality. Language models are typically trained via maximum likelihood and most often with teacher forcing. Teacher forcing is well-suited to optimizing perplexity but can result in poor sample quality because generating text requires conditioning on sequences of words that were never observed at training time. We propose to improve sample quality using Generative Adversarial Network (GANs), which explicitly train the generator to produce high quality samples and have shown a lot of success in image generation. GANs were originally to designed to output differentiable values, so discrete language generation is challenging for them. We introduce an actor-critic conditional GAN that fills in missing text conditioned on the surrounding context. We show qualitatively and quantitatively, evidence that this produces more realistic text samples compared to a maximum likelihood trained model.","url_abs":"https://openreview.net/forum?id=ByOExmWAb","url_pdf":"https://openreview.net/pdf?id=ByOExmWAb","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":null,"task_name":"Generative Adversarial Network"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"multivariate-time-series-imputation","task_name":"Multivariate Time Series Imputation"},{"task_slug":"text-generation","task_name":"Text Generation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"seq2seq","method_name":"Seq2Seq"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/multivariate-time-series-imputation-on-pems","task":"Multivariate Time Series Imputation","dataset":"PEMS-SF","model":"MaskGAN","rank_in_archive_order":4,"of":5,"metrics":{"L2 Loss (10^-4)":"6.02"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}