{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/knowledgenet-a-benchmark-dataset-for","title":"KnowledgeNet: A Benchmark Dataset for Knowledge Base Population","arxiv_id":null,"date":"2019-11-01","proceeding":"IJCNLP 2019 11","authors":["Filipe Mesquita","Matteo Cannaviccio","Jordan Schmidek","Paramita Mirza","Denilson Barbosa"],"abstract":"KnowledgeNet is a benchmark dataset for the task of automatically populating a knowledge base (Wikidata) with facts expressed in natural language text on the web. KnowledgeNet provides text exhaustively annotated with facts, thus enabling the holistic end-to-end evaluation of knowledge base population systems as a whole, unlike previous benchmarks that are more suitable for the evaluation of individual subcomponents (e.g., entity linking, relation extraction). We discuss five baseline approaches, where the best approach achieves an F1 score of 0.50, significantly outperforming a traditional approach by 79{\\%} (0.28). However, our best baseline is far from reaching human performance (0.82), indicating our dataset is challenging. The KnowledgeNet dataset and baselines are available at https://github.com/diffbot/knowledge-net","url_abs":"https://aclanthology.org/D19-1069","url_pdf":"https://aclanthology.org/D19-1069.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"knowledgenet-a-benchmark-dataset-for","repo_url":"https://github.com/diffbot/knowledge-net","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"entity-linking","task_name":"Entity Linking"},{"task_slug":"knowledge-base-population","task_name":"Knowledge Base Population"},{"task_slug":"relation-extraction","task_name":"Relation Extraction"}],"methods":[],"datasets_introduced":[{"slug":"knowledgenet","name":"KnowledgeNet","full_name":"KnowledgeNet"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}