{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/low-rank-rnn-adaptation-for-context-aware","title":"Low-Rank RNN Adaptation for Context-Aware Language Modeling","arxiv_id":"1710.02603","date":"2017-10-06","proceeding":"TACL 2018 1","authors":["Aaron Jaech","Mari Ostendorf"],"abstract":"A context-aware language model uses location, user and/or domain metadata\n(context) to adapt its predictions. In neural language models, context\ninformation is typically represented as an embedding and it is given to the RNN\nas an additional input, which has been shown to be useful in many applications.\nWe introduce a more powerful mechanism for using context to adapt an RNN by\nletting the context vector control a low-rank transformation of the recurrent\nlayer weight matrix. Experiments show that allowing a greater fraction of the\nmodel parameters to be adjusted has benefits in terms of perplexity and\nclassification for several different types of context.","url_abs":"http://arxiv.org/abs/1710.02603v2","url_pdf":"http://arxiv.org/pdf/1710.02603v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"low-rank-rnn-adaptation-for-context-aware","repo_url":"https://github.com/ajaech/calm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1710.02603","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}