{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/what-do-neural-machine-translation-models","title":"What do Neural Machine Translation Models Learn about Morphology?","arxiv_id":"1704.03471","date":"2017-04-11","proceeding":"ACL 2017 7","authors":["Yonatan Belinkov","Nadir Durrani","Fahim Dalvi","Hassan Sajjad","James Glass"],"abstract":"Neural machine translation (MT) models obtain state-of-the-art performance\nwhile maintaining a simple, end-to-end architecture. However, little is known\nabout what these models learn about source and target languages during the\ntraining process. In this work, we analyze the representations learned by\nneural MT models at various levels of granularity and empirically evaluate the\nquality of the representations for learning morphology through extrinsic\npart-of-speech and morphological tagging tasks. We conduct a thorough\ninvestigation along several parameters: word-based vs. character-based\nrepresentations, depth of the encoding layer, the identity of the target\nlanguage, and encoder vs. decoder representations. Our data-driven,\nquantitative evaluation sheds light on important aspects in the neural MT\nsystem and its ability to capture word structure.","url_abs":"http://arxiv.org/abs/1704.03471v3","url_pdf":"http://arxiv.org/pdf/1704.03471v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"what-do-neural-machine-translation-models","repo_url":"https://github.com/boknilev/nmt-repr-analysis","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"morphological-tagging","task_name":"Morphological Tagging"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1704.03471","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}