{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sequence-level-training-with-recurrent-neural","title":"Sequence Level Training with Recurrent Neural Networks","arxiv_id":"1511.06732","date":"2015-11-20","proceeding":null,"authors":["Marc'Aurelio Ranzato","Sumit Chopra","Michael Auli","Wojciech Zaremba"],"abstract":"Many natural language processing applications use language models to generate\ntext. These models are typically trained to predict the next word in a\nsequence, given the previous words and some context such as an image. However,\nat test time the model is expected to generate the entire sequence from\nscratch. This discrepancy makes generation brittle, as errors may accumulate\nalong the way. We address this issue by proposing a novel sequence level\ntraining algorithm that directly optimizes the metric used at test time, such\nas BLEU or ROUGE. On three different tasks, our approach outperforms several\nstrong baselines for greedy generation. The method is also competitive when\nthese baselines employ beam search, while being several times faster.","url_abs":"http://arxiv.org/abs/1511.06732v7","url_pdf":"http://arxiv.org/pdf/1511.06732v7.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sequence-level-training-with-recurrent-neural","repo_url":"https://github.com/facebookresearch/MIXER","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"sequence-level-training-with-recurrent-neural","repo_url":"https://github.com/CZWin32768/seqmnist","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"sequence-level-training-with-recurrent-neural","repo_url":"https://github.com/NPCai/Nopie","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"AGPL-3.0"}},{"paper_slug":"sequence-level-training-with-recurrent-neural","repo_url":"https://github.com/eske/seq2seq","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"sequence-level-training-with-recurrent-neural","repo_url":"https://github.com/facebookarchive/mixer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"machine-translation","task_name":"Machine Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/machine-translation-on-iwslt2015-german","task":"Machine Translation","dataset":"IWSLT2015 German-English","model":"Word-level LSTM w/attn","rank_in_archive_order":14,"of":15,"metrics":{"BLEU score":"20.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1511.06732","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}