{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/competence-based-curriculum-learning-for","title":"Competence-based Curriculum Learning for Neural Machine Translation","arxiv_id":"1903.09848","date":"2019-03-23","proceeding":"NAACL 2019 6","authors":["Emmanouil Antonios Platanios","Otilia Stretcu","Graham Neubig","Barnabas Poczos","Tom M. Mitchell"],"abstract":"Current state-of-the-art NMT systems use large neural networks that are not\nonly slow to train, but also often require many heuristics and optimization\ntricks, such as specialized learning rate schedules and large batch sizes. This\nis undesirable as it requires extensive hyperparameter tuning. In this paper,\nwe propose a curriculum learning framework for NMT that reduces training time,\nreduces the need for specialized heuristics or large batch sizes, and results\nin overall better performance. Our framework consists of a principled way of\ndeciding which training samples are shown to the model at different times\nduring training, based on the estimated difficulty of a sample and the current\ncompetence of the model. Filtering training samples in this manner prevents the\nmodel from getting stuck in bad local optima, making it converge faster and\nreach a better solution than the common approach of uniformly sampling training\nexamples. Furthermore, the proposed method can be easily applied to existing\nNMT models by simply modifying their input data pipelines. We show that our\nframework can help improve the training time and the performance of both\nrecurrent neural network models and Transformers, achieving up to a 70%\ndecrease in training time, while at the same time obtaining accuracy\nimprovements of up to 2.2 BLEU.","url_abs":"http://arxiv.org/abs/1903.09848v2","url_pdf":"http://arxiv.org/pdf/1903.09848v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"competence-based-curriculum-learning-for","repo_url":"https://github.com/josephch405/curriculum-nmt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"nmt","task_name":"NMT"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1903.09848","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}