</> Code with Papers by Syntology

Papers › QALD-9-plus: A Multilingual Dataset for Question Answering over DBpedia and Wikidata...

QALD-9-plus: A Multilingual Dataset for Question Answering over DBpedia and Wikidata Translated by Native Speakers

31 Jan 2022arXiv:2202.00120archive 2025-07-28

Aleksandr Perevalov, Dennis Diefenbach, Ricardo Usbeck, Andreas Both

The ability to have the same experience for different user groups (i.e., accessibility) is one of the most important characteristics of Web-based systems. The same is true for Knowledge Graph Question Answering (KGQA) systems that provide the access to Semantic Web data via natural language interface. While following our research agenda on the multilingual aspect of accessibility of KGQA systems, we identified several ongoing challenges. One of them is the lack of multilingual KGQA benchmarks. In this work, we extend one of the most popular KGQA benchmarks - QALD-9 by introducing high-quality questions' translations to 8 languages provided by native speakers, and transferring the SPARQL queries of QALD-9 from DBpedia to Wikidata, s.t., the usability and relevance of the dataset is strongly increased. Five of the languages - Armenian, Ukrainian, Lithuanian, Bashkir and Belarusian - to our best knowledge were never considered in KGQA research community before. The latter two of the languages are considered as "endangered" by UNESCO. We call the extended dataset QALD-9-plus and made it available online https://github.com/Perevalov/qald_9_plus.

PaperPDFConference PDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

perevalov/qald_9_plus officialmentioned in papermentioned on GitHubCC-BY-4.0 report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Graph Question AnsweringQuestion Answering

Datasets

Introduced by this paper, per the archive.

QALD-9-Plus

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Knowledge Base Question Answering QALD-9-Plus QAnswer-Wikidata-English Macro F1 0.4459 #1 of 12 Archive leaderboard report
Knowledge Base Question Answering QALD-9-Plus QAnswer-Wikidata-German Macro F1 0.3171 #2 of 12 Archive leaderboard report
Knowledge Base Question Answering QALD-9-Plus QAnswer-DBpedia-English Macro F1 0.3039 #3 of 12 Archive leaderboard report
Knowledge Base Question Answering QALD-9-Plus QAnswer-Wikidata-French Macro F1 0.23 #4 of 12 Archive leaderboard report
Knowledge Base Question Answering QALD-9-Plus QAnswer-Wikidata-Russian Macro F1 0.2143 #5 of 12 Archive leaderboard report
Knowledge Base Question Answering QALD-9-Plus QAnswer-DBpedia-German Macro F1 0.1998 #6 of 12 Archive leaderboard report
Knowledge Base Question Answering QALD-9-Plus QAnswer-DBpedia-French Macro F1 0.1506 #7 of 12 Archive leaderboard report
Knowledge Base Question Answering QALD-9-Plus Platypus-Wikidata-English Macro F1 0.1503 #8 of 12 Archive leaderboard report
Knowledge Base Question Answering QALD-9-Plus DeepPavlov-Wikidata-English Macro F1 0.124 #9 of 12 Archive leaderboard report
Knowledge Base Question Answering QALD-9-Plus QAnswer-DBpedia-Russian Macro F1 0.0957 #10 of 12 Archive leaderboard report
Knowledge Base Question Answering QALD-9-Plus DeepPavlov-Wikidata-Russian Macro F1 0.087 #11 of 12 Archive leaderboard report
Knowledge Base Question Answering QALD-9-Plus Platypus-Wikidata-French Macro F1 0.0417 #12 of 12 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections