{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/professor-forcing-a-new-algorithm-for","title":"Professor Forcing: A New Algorithm for Training Recurrent Networks","arxiv_id":"1610.09038","date":"2016-10-27","proceeding":"NeurIPS 2016 12","authors":["Alex Lamb","Anirudh Goyal","Ying Zhang","Saizheng Zhang","Aaron Courville","Yoshua Bengio"],"abstract":"The Teacher Forcing algorithm trains recurrent networks by supplying observed\nsequence values as inputs during training and using the network's own\none-step-ahead predictions to do multi-step sampling. We introduce the\nProfessor Forcing algorithm, which uses adversarial domain adaptation to\nencourage the dynamics of the recurrent network to be the same when training\nthe network and when sampling from the network over multiple time steps. We\napply Professor Forcing to language modeling, vocal synthesis on raw waveforms,\nhandwriting generation, and image generation. Empirically we find that\nProfessor Forcing acts as a regularizer, improving test likelihood on character\nlevel Penn Treebank and sequential MNIST. We also find that the model\nqualitatively improves samples, especially when sampling for a large number of\ntime steps. This is supported by human evaluation of sample quality. Trade-offs\nbetween Professor Forcing and Scheduled Sampling are discussed. We produce\nT-SNEs showing that Professor Forcing successfully makes the dynamics of the\nnetwork during training and sampling more similar.","url_abs":"http://arxiv.org/abs/1610.09038v1","url_pdf":"http://arxiv.org/pdf/1610.09038v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"professor-forcing-a-new-algorithm-for","repo_url":"https://github.com/mojesty/professor_forcing","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"domain-adaptation","task_name":"Domain Adaptation"},{"task_slug":"handwriting-generation","task_name":"Handwriting generation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1610.09038","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}