{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/guiding-teacher-forcing-with-seer-forcing-for","title":"Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation","arxiv_id":"2106.06751","date":"2021-06-12","proceeding":"ACL 2021 5","authors":["Yang Feng","Shuhao Gu","Dengji Guo","Zhengxin Yang","Chenze Shao"],"abstract":"Although teacher forcing has become the main training paradigm for neural machine translation, it usually makes predictions only conditioned on past information, and hence lacks global planning for the future. To address this problem, we introduce another decoder, called seer decoder, into the encoder-decoder framework during training, which involves future information in target predictions. Meanwhile, we force the conventional decoder to simulate the behaviors of the seer decoder via knowledge distillation. In this way, at test the conventional decoder can perform like the seer decoder without the attendance of it. Experiment results on the Chinese-English, English-German and English-Romanian translation tasks show our method can outperform competitive baselines significantly and achieves greater improvements on the bigger data sets. Besides, the experiments also prove knowledge distillation the best way to transfer knowledge from the seer decoder to the conventional decoder compared to adversarial learning and L2 regularization.","url_abs":"https://arxiv.org/abs/2106.06751v1","url_pdf":"https://arxiv.org/pdf/2106.06751v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"l2-regularization","task_name":"L2 Regularization"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2106.06751","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.06751"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ictnlp/SeerForcingNMT","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"07969f7041022050","entry":"distributed_init","repo":"ictnlp/SeerForcingNMT","repo_kind":"found_in_text","path":"fairseq/distributed_utils.py","file_url":"https://github.com/ictnlp/SeerForcingNMT/blob/HEAD/fairseq/distributed_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"07969f7041022050"}},{"code_sha256_prefix":"88ce019380f10478","entry":"get_symbols_to_strip_from_output","repo":"ictnlp/SeerForcingNMT","repo_kind":"found_in_text","path":"generate.py","file_url":"https://github.com/ictnlp/SeerForcingNMT/blob/HEAD/generate.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"88ce019380f10478"}},{"code_sha256_prefix":"90b79fe15491de83","entry":"is_master","repo":"ictnlp/SeerForcingNMT","repo_kind":"found_in_text","path":"fairseq/distributed_utils.py","file_url":"https://github.com/ictnlp/SeerForcingNMT/blob/HEAD/fairseq/distributed_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"90b79fe15491de83"}},{"code_sha256_prefix":"82bdf940665aeeb4","entry":"safe_readline","repo":"ictnlp/SeerForcingNMT","repo_kind":"found_in_text","path":"fairseq/binarizer.py","file_url":"https://github.com/ictnlp/SeerForcingNMT/blob/HEAD/fairseq/binarizer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"82bdf940665aeeb4"}},{"code_sha256_prefix":"199e96db5edd0edd","entry":"all_reduce","repo":"ictnlp/SeerForcingNMT","repo_kind":"found_in_text","path":"fairseq/distributed_utils.py","file_url":"https://github.com/ictnlp/SeerForcingNMT/blob/HEAD/fairseq/distributed_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"199e96db5edd0edd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}