{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/data-noising-as-smoothing-in-neural-network","title":"Data Noising as Smoothing in Neural Network Language Models","arxiv_id":"1703.02573","date":"2017-03-07","proceeding":null,"authors":["Ziang Xie","Sida I. Wang","Jiwei Li","Daniel Lévy","Aiming Nie","Dan Jurafsky","Andrew Y. Ng"],"abstract":"Data noising is an effective technique for regularizing neural network\nmodels. While noising is widely adopted in application domains such as vision\nand speech, commonly used noising primitives have not been developed for\ndiscrete sequence-level settings such as language modeling. In this paper, we\nderive a connection between input noising in neural network language models and\nsmoothing in $n$-gram models. Using this connection, we draw upon ideas from\nsmoothing to develop effective noising schemes. We demonstrate performance\ngains when applying the proposed schemes to language modeling and machine\ntranslation. Finally, we provide empirical analysis validating the relationship\nbetween noising and smoothing.","url_abs":"http://arxiv.org/abs/1703.02573v1","url_pdf":"http://arxiv.org/pdf/1703.02573v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-noising-as-smoothing-in-neural-network","repo_url":"https://github.com/stanfordmlgroup/nlm-noising","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1703.02573","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}