{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/maskgan-better-text-generation-via-filling-in","title":"MaskGAN: Better Text Generation via Filling in the______","arxiv_id":"1801.07736","date":"2018-01-23","proceeding":null,"authors":["William Fedus","Ian Goodfellow","Andrew M. Dai"],"abstract":"Neural text generation models are often autoregressive language models or\nseq2seq models. These models generate text by sampling words sequentially, with\neach word conditioned on the previous word, and are state-of-the-art for\nseveral machine translation and summarization benchmarks. These benchmarks are\noften defined by validation perplexity even though this is not a direct measure\nof the quality of the generated text. Additionally, these models are typically\ntrained via maxi- mum likelihood and teacher forcing. These methods are\nwell-suited to optimizing perplexity but can result in poor sample quality\nsince generating text requires conditioning on sequences of words that may have\nnever been observed at training time. We propose to improve sample quality\nusing Generative Adversarial Networks (GANs), which explicitly train the\ngenerator to produce high quality samples and have shown a lot of success in\nimage generation. GANs were originally designed to output differentiable\nvalues, so discrete language generation is challenging for them. We claim that\nvalidation perplexity alone is not indicative of the quality of text generated\nby a model. We introduce an actor-critic conditional GAN that fills in missing\ntext conditioned on the surrounding context. We show qualitatively and\nquantitatively, evidence that this produces more realistic conditional and\nunconditional text samples compared to a maximum likelihood trained model.","url_abs":"http://arxiv.org/abs/1801.07736v3","url_pdf":"http://arxiv.org/pdf/1801.07736v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"multivariate-time-series-imputation","task_name":"Multivariate Time Series Imputation"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/multivariate-time-series-imputation-on-2","task":"Multivariate Time Series Imputation","dataset":"Basketball Players Movement","model":"MaskGAN","rank_in_archive_order":3,"of":5,"metrics":{"OOB Rate (10^−3) ":"4.592","Path Difference":"0.680","Path Length":"0.793","Player Distance ":"0.427","Step Change (10^−3)":"9.622"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1801.07736","atlas_url":"https://app.syntology.ai/?focus=1801.07736","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}