Home › Datasets › language › Multilingual
Multilingual datasets
archive 2025-07-2829 datasets carry the language tag "Multilingual", ordered by the archive's paper count. Page 1 of 1: 29 shown of 29. Facet routes are this site's own (the archive records the tag string, not a page).
The archive holds 12,214 dataset rows; 12,172 are listed. 6 are withheld from every listing and count here as vandalised before snapshot (6 with contact-centre spam in the title, 0 with a spam description on a row that has no homepage, no paper and no papers counted; none with more than 1 paper, 0 with a benchmark), listed in withheld.json; 1 listed row carries a vandalised description, withheld on its page. This gate never withholds a row with a homepage or a paper that resolves, and a clean description; the content rules below withhold a row whose name is spam whatever else it carries. The gate is a phrase list: these are the rows it caught, not a claim that the rest is clean. Before that gate, the site's content rules withhold 36 more rows (invite-code, gambling, travel-booking, contact-centre and similar spam in the name or on a row with nothing real behind it); they have no page and are listed in withheld.json.
Filter 50 language tags shown of 367, by dataset count; the full filter by modality, task and language is on /datasets
All 12,172
English 3,998Chinese 460German 199French 190Spanish 156Russian 143Japanese 111Arabic 109Italian 98Portuguese 93Hindi 81Vietnamese 71Korean 68Bengali 66Turkish 63Persian 60Dutch 51Tamil 49Indonesian 44Polish 44Czech 41Danish 35Finnish 35Telugu 35Romanian 34Urdu 34Thai 33Marathi 31Hungarian 29Multilingual 29Swedish 29Greek 26Gujarati 26Hebrew 25Mandarin Chinese 25Estonian 24Malayalam 23Ukrainian 23Bulgarian 22Basque 21Kannada 19Punjabi 19Catalan 18Croatian 18Slovak 18Swahili 18Lithuanian 17Latvian 16Norwegian 16Serbian 16
Multilingual datasets 1–29 of 29
VoxCeleb2 is a large scale speaker recognition dataset obtained automatically from open-source media.
564 papers · 5 benchmarks
XNLI (Cross-lingual Natural Language Inference)
The Cross-lingual Natural Language Inference (XNLI) corpus is the extension of the Multi-Genre NLI (MultiNLI) corpus to 15 languages.
349 papers · 7 benchmarks
XQuAD (Cross-lingual Question Answering Dataset) is a benchmark dataset for evaluating cross-lingual question answering performance.
190 papers · 1 benchmark
VoxPopuli is a large-scale multilingual corpus providing 100K hours of unlabelled speech data in 23 languages.
113 papers · 1 benchmark
The CoNLL-2012 shared task involved predicting coreference in English, Chinese, and Arabic, using the final version, v5.0, of the OntoNotes corpus.
89 papers · 0 benchmarks
FLoRes-101 is an evaluation benchmark for low-resource and multilingual machine translation.
86 papers · 57 benchmarks
WIT (Wikipedia-based Image Text)
Wikipedia-based Image Text (WIT) Dataset is a large multimodal multilingual dataset.
79 papers · 1 benchmark
WikiANN (PAN-X)
WikiANN, also known as PAN-X, is a multilingual named entity recognition dataset.
72 papers · 4 benchmarks
ParaCrawl v.7.1 is a parallel dataset with 41 language pairs primarily aligned with English (39 out of 41) and mined using the parallel-data-crawling tool Bitextor which includes downloading documents, preprocessing and normalization,…
66 papers · 0 benchmarks
OSCAR or Open Super-large Crawled ALMAnaCH coRpus is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture.
64 papers · 0 benchmarks
MLDoc (Multilingual Document Classification Corpus)
Multilingual Document Classification Corpus (MLDoc) is a cross-lingual document classification dataset covering English, German, French, Spanish, Italian, Russian, Japanese and Chinese.
53 papers · 8 benchmarks
AVSpeech is a large-scale audio-visual dataset comprising speech clips with no interfering background signals.
42 papers · 0 benchmarks
We present CulturaX, a substantial multilingual dataset with 6.3 trillion tokens in 167 languages, tailored for large language model (LLM) development.
39 papers · 0 benchmarks
CCAligned consists of parallel or comparable web-document pairs in 137 languages aligned with English.
33 papers · 0 benchmarks
This is the dataset for the 2020 Duolingo shared task on Simultaneous Translation And Paraphrase for Language Education (STAPLE).
10 papers · 0 benchmarks
This repository contains gzipped files containing more than 2 million tokens (words) from answers submitted by more than 6,000 students over the course of their first 30 days of using Duolingo.
3 papers · 0 benchmarks
This is a gzipped CSV file containing the 13 million Duolingo student learning traces used in experiments by Settles & Meeder (2016).
3 papers · 0 benchmarks
QALD-9-Plus Dataset Description QALD-9-Plus is the dataset for Knowledge Graph Question Answering (KGQA) based on well-known QALD-9.
3 papers · 1 benchmark
CC-News (CommonCrawl News dataset)
CommonCrawl News is a dataset containing news articles from news sites all over the world.
2 papers · 0 benchmarks
A multilingual etymological database extracted from the Wiktionary (described in Methodological Aspects of Developing and Managing an Etymological Lexical Resource: Introducing EtymDB-2.0)
2 papers · 0 benchmarks
Clean version of UDHR (Universal Declaration of Human Rights), at the long sentence level.
2 papers · 0 benchmarks
Automatic segmentation, tokenization and morphological and syntactic annotations of raw texts in 45 languages, generated by UDPipe (http://ufal.mff.cuni.cz/udpipe), together with word embeddings of dimension 100 computed from lowercased…
1 paper · 0 benchmarks
GLAMI-1M (A Multilingual Image-Text Fashion Dataset)
We introduce GLAMI-1M: the largest multilingual image-text classification dataset and benchmark.
1 paper · 1 benchmark
Glot500-c (Glot500 Corpus)
A dataset of natural language data collected by putting together more than 150 existing mono-lingual and multilingual datasets together and crawling known multilingual websites.
1 paper · 0 benchmarks
Collection of news websites in low-resource languages.
1 paper · 0 benchmarks
1 paper · 0 benchmarks
WebLI (Web Language Image)
WebLI (Web Language Image) is a web-scale multilingual image-text dataset, designed to support Google’s vision-language research, such as the large-scale pre-training for image understanding, image captioning, visual question answering,…
1 paper · 0 benchmarks
We release the datasets to replicate the results of Coordinated Reply Attacks in Influence Operations: Characterization and Detection'.
1 paper · 0 benchmarks
Paper counts and descriptions are the archive's, frozen 2025-07-28; no citation counts, no stars, no trending. Sorting by "most cited" or "newest" was a live-site feature the archive does not carry.