{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-stable-and-effective-learning-strategy-for","title":"A Stable and Effective Learning Strategy for Trainable Greedy Decoding","arxiv_id":"1804.07915","date":"2018-04-21","proceeding":"EMNLP 2018 10","authors":["Yun Chen","Victor O. K. Li","Kyunghyun Cho","Samuel R. Bowman"],"abstract":"Beam search is a widely used approximate search strategy for neural network\ndecoders, and it generally outperforms simple greedy decoding on tasks like\nmachine translation. However, this improvement comes at substantial\ncomputational cost. In this paper, we propose a flexible new method that allows\nus to reap nearly the full benefits of beam search with nearly no additional\ncomputational cost. The method revolves around a small neural network actor\nthat is trained to observe and manipulate the hidden state of a\npreviously-trained decoder. To train this actor network, we introduce the use\nof a pseudo-parallel corpus built using the output of beam search on a base\nmodel, ranked by a target quality metric like BLEU. Our method is inspired by\nearlier work on this problem, but requires no reinforcement learning, and can\nbe trained reliably on a range of models. Experiments on three parallel corpora\nand three architectures show that the method yields substantial improvements in\ntranslation quality and speed over each base system.","url_abs":"http://arxiv.org/abs/1804.07915v2","url_pdf":"http://arxiv.org/pdf/1804.07915v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-stable-and-effective-learning-strategy-for","repo_url":"https://github.com/vadimkantorov/ctc","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"translation","task_name":"Translation"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1804.07915","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}