{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/document-context-language-models","title":"Document Context Language Models","arxiv_id":"1511.03962","date":"2015-11-12","proceeding":null,"authors":["Yangfeng Ji","Trevor Cohn","Lingpeng Kong","Chris Dyer","Jacob Eisenstein"],"abstract":"Text documents are structured on multiple levels of detail: individual words\nare related by syntax, but larger units of text are related by discourse\nstructure. Existing language models generally fail to account for discourse\nstructure, but it is crucial if we are to have language models that reward\ncoherence and generate coherent texts. We present and empirically evaluate a\nset of multi-level recurrent neural network language models, called\nDocument-Context Language Models (DCLM), which incorporate contextual\ninformation both within and beyond the sentence. In comparison with word-level\nrecurrent neural network language models, the DCLM models obtain slightly\nbetter predictive likelihoods, and considerably better assessments of document\ncoherence.","url_abs":"http://arxiv.org/abs/1511.03962v4","url_pdf":"http://arxiv.org/pdf/1511.03962v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"document-context-language-models","repo_url":"https://github.com/jiyfeng/dclm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"sentence","task_name":"Sentence"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1511.03962","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}