Datasets › WikiANN

WikiANN (PAN-X)

Introduced by Xiaoman Pan et al. in Cross-lingual Name Tagging and Linking for 282 Languages14 Sep 2023 archive 2025-07-28

WikiANN, also known as PAN-X, is a multilingual named entity recognition dataset. It consists of Wikipedia articles that have been annotated with LOC (location), PER (person), and ORG (organization) tags in the IOB2 format¹². This dataset serves as a valuable resource for training and evaluating named entity recognition models across various languages.

For instance, it includes information about notable individuals, places, and organizations mentioned in Wikipedia articles. Researchers and practitioners can use WikiANN to develop and improve natural language processing systems that identify and classify named entities in text.

(1) wikiann · Datasets at Hugging Face. https://huggingface.co/datasets/wikiann. (2) wikiann | TensorFlow Datasets. https://tensorflow.google.cn/datasets/catalog/wikiann. (3) wikiann · Datasets at Hugging Face. https://huggingface.co/datasets/wikiann/viewer/en. (4) WikiAnn Dataset | Papers With Code. https://paperswithcode.com/dataset/wikiann-1.

Homepage Benchmarks Papers JSON

Benchmarks archive 2025-07-28

All 4 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

		First row (archive order)	Paper	Code
Cross-Lingual NER	WikiAnn NER	ByT5 XXL F1 67.7	ByT5: Towards a token-free future with pre-trained...	huggingface/transformers +4	1	Compare
UIE	WikiANN	KnowCoder-7b-IE F1 score 87.0	KnowCoder: Coding Structured Knowledge into LLMs for...	ICT-GoKnow/KnowCoder	1	Compare
Token Classification	Wikiann	no rows	—	—	0	Compare
Token Classification	wikiann sk	no rows	—	—	0	Compare

Papers archive 2025-07-28

2 shown of 2 papers with a leaderboard row on this dataset's benchmarks, newest first. The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset; the archive's count for this dataset is 72. The Syntology column is from Syntology's graph (read 2026-09-24), stated per sample; it is not part of any archive number.

			Date	Samples run Syntology
KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction	1	1	12 Mar 2024	not harvested
ByT5: Towards a token-free future with pre-trained byte-to-byte models	5	1	28 May 2021	ran 0 of 6 samples (6 unverified)

Dataset loaders archive 2025-07-28

huggingface.co/datasets/unimelb-nlp/wikiann huggingface/datasetstfpytorchjax

huggingface.co/datasets/cfilt/HiNER-original huggingface/datasetstfpytorchjax

huggingface.co/datasets/cfilt/HiNER-collapsed huggingface/datasetstfpytorchjax

huggingface.co/datasets/wikiann huggingface/datasetstfpytorchjax

www.tensorflow.org/datasets/catalog/wikiann tensorflow/datasetstfjax

5 loaders as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

Named Entity Recognition (NER)Cross-Lingual Transfer Cross-Lingual NER UIE Token Classification Named Entity Recognition Word Embeddings

License archive 2025-07-28

No licence recorded in the archive. Absence here is not a statement about the dataset's terms.

Modalities archive 2025-07-28

No modality tagged.

Languages archive 2025-07-28

English French Spanish German Italian Chinese Bengali Multilingual Japanese Russian Portuguese Afrikaans Albanian Amharic Arabic Armenian Bambara Basque Belarusian Breton Bulgarian Catalan Church Slavic Croatian Czech Danish Dutch Erzya Estonian Faroese Finnish Galician Gothic Hebrew Hindi Hungarian Icelandic Indonesian Irish Kazakh Komi-Permyak Korean Latin Latvian Lithuanian Livvi Maltese Manx Marathi Moksha Northern Sami Norwegian Persian Polish Romanian Russia Buriat Sanskrit Scottish Gaelic Serbian Slovak Slovenian Swedish Tagalog Tamil Telugu Thai Turkish Uighur Ukrainian Upper Sorbian Urdu Vietnamese Welsh Wolof Yoruba Afar Abkhazian Achinese Adyghe Akan Tosk Albanian Old English (ca. 450-1100)Official Aramaic (700-300 BCE)Aragonese Egyptian Arabic Assamese Asturian Avaric Aymara South Azerbaijani Azerbaijani Bashkir Bavarian Central Bikol Bislama Banjar Tibetan Bosnian Bishnupriya Buginese Min Dong Chinese Cebuano Chamorro Chechen Choctaw Cherokee Chuvash Cheyenne Central Kurdish Cornish Corsican Cree Crimean Tatar Kashubian Dimli (individual language)Dhivehi Lower Sorbian Dzongkha Modern Greek (1453-)Esperanto Ewe Extremaduran Fijian Arpitan Northern Frisian Western Frisian Fulah Friulian Gagauz Gan Chinese Gilaki Goan Konkani Guarani Gujarati Hakka Chinese Haitian Hausa Hawaiian Serbo-Croatian Herero Fiji Hindi Hiri Motu Igbo Ido Sichuan Yi Inuktitut Interlingue Iloko Interlingua (International Auxiliary Language Association)Inupiaq Jamaican Creole English Javanese Lojban Kara-Kalpak Kabyle Kalaallisut Kannada Kashmiri Georgian Kanuri Kabardian Central Khmer Kikuyu Kinyarwanda Kirghiz Komi Kongo Karachay-Balkar Kölsch Kuanyama Kurdish Ladino Lao Lak Lezghian Ligurian Limburgan Lingala Lombard Northern Luri Latgalian Luxembourgish Ganda Marshallese Maithili Malayalam Eastern Mari Minangkabau Macedonian Malagasy Mongolian Maori Western Mari Malay (macrolanguage)Creek Mirandese Burmese Mazanderani Neapolitan Nauru Navajo Ndonga Low German Nepali (macrolanguage)Newari Norwegian Nynorsk Novial Narom Pedi Nyanja Occitan (post 1500)Oriya (macrolanguage)Oromo Ossetian Pangasinan Pampanga Papiamento Picard Pennsylvania German Pfaelzisch Pitcairn-Norfolk Pali Piemontese Western Panjabi Pontic Pushto Quechua Vlax Romani Romansh Rusyn Rundi Sango Yakut Sicilian Scots Sinhala Samoan Shona Sindhi Somali Southern Sotho Sardinian Sranan Tongo Swati Saterfriesisch Sundanese Swahili (macrolanguage)Silesian Tahitian Tatar Tulu Tetum Tajik Tigrinya Tonga (Tonga Islands)Tok Pisin Tswana Tsonga Turkmen Tumbuka Twi Tuvinian Udmurt Uzbek Venetian Venda Veps Vlaams Volapük Waray (Philippines)Walloon Wu Chinese Kalmyk Xhosa Mingrelian Yiddish Zeeuws Zhuang Zulu

Variants archive 2025-07-28

Wikiann
WikiANN
WikiAnn NER
wikiann sk
HiNER Collapsed
HiNER Original
WikiAnn (Maltese)
WikiAnn (en, ko, es, pt)

8 variant names, as the archive lists them.

At a glance archive 2025-07-28

Introduced: 14 Sep 2023
Papers: 72
Benchmarks: 4
With a result: 2
Languages: English, French, Spanish, German, Italian, Chinese, Bengali, Multilingual, Japanese, Russian, Portuguese, Afrikaans, Albanian, Amharic, Arabic, Armenian, Bambara, Basque, Belarusian, Breton, Bulgarian, Catalan, Church Slavic, Croatian, Czech, Danish, Dutch, Erzya, Estonian, Faroese, Finnish, Galician, Gothic, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Irish, Kazakh, Komi-Permyak, Korean, Latin, Latvian, Lithuanian, Livvi, Maltese, Manx, Marathi, Moksha, Northern Sami, Norwegian, Persian, Polish, Romanian, Russia Buriat, Sanskrit, Scottish Gaelic, Serbian, Slovak, Slovenian, Swedish, Tagalog, Tamil, Telugu, Thai, Turkish, Uighur, Ukrainian, Upper Sorbian, Urdu, Vietnamese, Welsh, Wolof, Yoruba, Afar, Abkhazian, Achinese, Adyghe, Akan, Tosk Albanian, Old English (ca. 450-1100), Official Aramaic (700-300 BCE), Aragonese, Egyptian Arabic, Assamese, Asturian, Avaric, Aymara, South Azerbaijani, Azerbaijani, Bashkir, Bavarian, Central Bikol, Bislama, Banjar, Tibetan, Bosnian, Bishnupriya, Buginese, Min Dong Chinese, Cebuano, Chamorro, Chechen, Choctaw, Cherokee, Chuvash, Cheyenne, Central Kurdish, Cornish, Corsican, Cree, Crimean Tatar, Kashubian, Dimli (individual language), Dhivehi, Lower Sorbian, Dzongkha, Modern Greek (1453-), Esperanto, Ewe, Extremaduran, Fijian, Arpitan, Northern Frisian, Western Frisian, Fulah, Friulian, Gagauz, Gan Chinese, Gilaki, Goan Konkani, Guarani, Gujarati, Hakka Chinese, Haitian, Hausa, Hawaiian, Serbo-Croatian, Herero, Fiji Hindi, Hiri Motu, Igbo, Ido, Sichuan Yi, Inuktitut, Interlingue, Iloko, Interlingua (International Auxiliary Language Association), Inupiaq, Jamaican Creole English, Javanese, Lojban, Kara-Kalpak, Kabyle, Kalaallisut, Kannada, Kashmiri, Georgian, Kanuri, Kabardian, Central Khmer, Kikuyu, Kinyarwanda, Kirghiz, Komi, Kongo, Karachay-Balkar, Kölsch, Kuanyama, Kurdish, Ladino, Lao, Lak, Lezghian, Ligurian, Limburgan, Lingala, Lombard, Northern Luri, Latgalian, Luxembourgish, Ganda, Marshallese, Maithili, Malayalam, Eastern Mari, Minangkabau, Macedonian, Malagasy, Mongolian, Maori, Western Mari, Malay (macrolanguage), Creek, Mirandese, Burmese, Mazanderani, Neapolitan, Nauru, Navajo, Ndonga, Low German, Nepali (macrolanguage), Newari, Norwegian Nynorsk, Novial, Narom, Pedi, Nyanja, Occitan (post 1500), Oriya (macrolanguage), Oromo, Ossetian, Pangasinan, Pampanga, Papiamento, Picard, Pennsylvania German, Pfaelzisch, Pitcairn-Norfolk, Pali, Piemontese, Western Panjabi, Pontic, Pushto, Quechua, Vlax Romani, Romansh, Rusyn, Rundi, Sango, Yakut, Sicilian, Scots, Sinhala, Samoan, Shona, Sindhi, Somali, Southern Sotho, Sardinian, Sranan Tongo, Swati, Saterfriesisch, Sundanese, Swahili (macrolanguage), Silesian, Tahitian, Tatar, Tulu, Tetum, Tajik, Tigrinya, Tonga (Tonga Islands), Tok Pisin, Tswana, Tsonga, Turkmen, Tumbuka, Twi, Tuvinian, Udmurt, Uzbek, Venetian, Venda, Veps, Vlaams, Volapük, Waray (Philippines), Walloon, Wu Chinese, Kalmyk, Xhosa, Mingrelian, Yiddish, Zeeuws, Zhuang, Zulu
Loaders: 5
Homepage: tensorflow.google.cn/datasets/catalog/wikiann

Samples run Syntology

Of the 2 papers with a result here, 1 has harvested code samples in Syntology's graph. Summed over those: Syntology ran 0 of 6 samples; 6 unverified. 1 of the 1 has no sample that ran. Each row in the Papers table states its own "ran N of M samples" with the unverified count; a sample that ran executed on a synthesized fixture, which is not a correctness claim about any result. Read from the graph 2026-09-24.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections