{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bi-simcut-a-simple-strategy-for-boosting-1","title":"Bi-SimCut: A Simple Strategy for Boosting Neural Machine Translation","arxiv_id":"2206.02368","date":"2022-06-06","proceeding":"NAACL 2022 7","authors":["Pengzhi Gao","Zhongjun He","Hua Wu","Haifeng Wang"],"abstract":"We introduce Bi-SimCut: a simple but effective training strategy to boost neural machine translation (NMT) performance. It consists of two procedures: bidirectional pretraining and unidirectional finetuning. Both procedures utilize SimCut, a simple regularization method that forces the consistency between the output distributions of the original and the cutoff sentence pairs. Without leveraging extra dataset via back-translation or integrating large-scale pretrained model, Bi-SimCut achieves strong translation performance across five translation benchmarks (data sizes range from 160K to 20.2M): BLEU scores of 31.16 for en -> de and 38.37 for de -> en on the IWSLT14 dataset, 30.78 for en -> de and 35.15 for de -> en on the WMT14 dataset, and 27.17 for zh -> en on the WMT17 dataset. SimCut is not a new method, but a version of Cutoff (Shen et al., 2020) simplified and adapted for NMT, and it could be considered as a perturbation-based method. Given the universality and simplicity of SimCut and Bi-SimCut, we believe they can serve as strong baselines for future NMT research.","url_abs":"https://arxiv.org/abs/2206.02368v2","url_pdf":"https://arxiv.org/pdf/2206.02368v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bi-simcut-a-simple-strategy-for-boosting-1","repo_url":"https://github.com/gpengzhi/Bi-SimCut","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"nmt","task_name":"NMT"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":"de-en","task_name":"de-en"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/machine-translation-on-iwslt2014-english","task":"Machine Translation","dataset":"IWSLT2014 English-German","model":"Bi-SimCut","rank_in_archive_order":2,"of":6,"metrics":{"BLEU score":"31.16"},"uses_additional_data":false},{"leaderboard":"/sota/machine-translation-on-iwslt2014-english","task":"Machine Translation","dataset":"IWSLT2014 English-German","model":"SimCut","rank_in_archive_order":3,"of":6,"metrics":{"BLEU score":"30.98"},"uses_additional_data":false},{"leaderboard":"/sota/machine-translation-on-iwslt2014-german","task":"Machine Translation","dataset":"IWSLT2014 German-English","model":"Bi-SimCut","rank_in_archive_order":3,"of":34,"metrics":{"BLEU score":"38.37"},"uses_additional_data":false},{"leaderboard":"/sota/machine-translation-on-iwslt2014-german","task":"Machine Translation","dataset":"IWSLT2014 German-English","model":"SimCut","rank_in_archive_order":7,"of":34,"metrics":{"BLEU score":"37.81"},"uses_additional_data":false},{"leaderboard":"/sota/machine-translation-on-wmt2014-english-german","task":"Machine Translation","dataset":"WMT2014 English-German","model":"Bi-SimCut","rank_in_archive_order":7,"of":91,"metrics":{"BLEU score":"30.78"},"uses_additional_data":false},{"leaderboard":"/sota/machine-translation-on-wmt2014-english-german","task":"Machine Translation","dataset":"WMT2014 English-German","model":"SimCut","rank_in_archive_order":10,"of":91,"metrics":{"BLEU score":"30.56"},"uses_additional_data":false},{"leaderboard":"/sota/machine-translation-on-wmt2014-german-english","task":"Machine Translation","dataset":"WMT2014 German-English","model":"Bi-SimCut","rank_in_archive_order":1,"of":16,"metrics":{"BLEU score":"35.15"},"uses_additional_data":false},{"leaderboard":"/sota/machine-translation-on-wmt2014-german-english","task":"Machine Translation","dataset":"WMT2014 German-English","model":"SimCut","rank_in_archive_order":3,"of":16,"metrics":{"BLEU score":"34.86"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.02368","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}