{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dawt-densely-annotated-wikipedia-texts-across","title":"DAWT: Densely Annotated Wikipedia Texts across multiple languages","arxiv_id":"1703.00948","date":"2017-03-02","proceeding":null,"authors":["Nemanja Spasojevic","Preeti Bhargava","Guoning Hu"],"abstract":"In this work, we open up the DAWT dataset - Densely Annotated Wikipedia Texts\nacross multiple languages. The annotations include labeled text mentions\nmapping to entities (represented by their Freebase machine ids) as well as the\ntype of the entity. The data set contains total of 13.6M articles, 5.0B tokens,\n13.8M mention entity co-occurrences. DAWT contains 4.8 times more anchor text\nto entity links than originally present in the Wikipedia markup. Moreover, it\nspans several languages including English, Spanish, Italian, German, French and\nArabic. We also present the methodology used to generate the dataset which\nenriches Wikipedia markup in order to increase number of links. In addition to\nthe main dataset, we open up several derived datasets including mention entity\nco-occurrence counts and entity embeddings, as well as mappings between\nFreebase ids and Wikidata item ids. We also discuss two applications of these\ndatasets and hope that opening them up would prove useful for the Natural\nLanguage Processing and Information Retrieval communities, as well as\nfacilitate multi-lingual research.","url_abs":"http://arxiv.org/abs/1703.00948v1","url_pdf":"http://arxiv.org/pdf/1703.00948v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"articles","task_name":"Articles"},{"task_slug":"entity-embeddings","task_name":"Entity Embeddings"},{"task_slug":"information-retrieval","task_name":"Information Retrieval"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[{"slug":"dawt","name":"DAWT","full_name":"Densely Annotated Wikipedia Texts"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}