{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-when-to-concentrate-or-divert","title":"Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translation","arxiv_id":"1808.07374","date":"2018-08-22","proceeding":"EMNLP 2018 10","authors":["Junyang Lin","Xu sun","Xuancheng Ren","Muyu Li","Qi Su"],"abstract":"Most of the Neural Machine Translation (NMT) models are based on the\nsequence-to-sequence (Seq2Seq) model with an encoder-decoder framework equipped\nwith the attention mechanism. However, the conventional attention mechanism\ntreats the decoding at each time step equally with the same matrix, which is\nproblematic since the softness of the attention for different types of words\n(e.g. content words and function words) should differ. Therefore, we propose a\nnew model with a mechanism called Self-Adaptive Control of Temperature (SACT)\nto control the softness of attention by means of an attention temperature.\nExperimental results on the Chinese-English translation and English-Vietnamese\ntranslation demonstrate that our model outperforms the baseline models, and the\nanalysis and the case study show that our model can attend to the most relevant\nelements in the source-side contexts and generate the translation of high\nquality.","url_abs":"http://arxiv.org/abs/1808.07374v2","url_pdf":"http://arxiv.org/pdf/1808.07374v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-when-to-concentrate-or-divert","repo_url":"https://github.com/lancopku/SACT","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"nmt","task_name":"NMT"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/machine-translation-on-iwslt2015-english-1","task":"Machine Translation","dataset":"IWSLT2015 English-Vietnamese","model":"Self-Adaptive Control of Temperature","rank_in_archive_order":7,"of":11,"metrics":{"BLEU":"29.12"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1808.07374","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}