Cross-Lingual Transfer

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

15 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

2 subtasks in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 333 papers with code (782 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Unsupervised Cross-lingual Representation Learning at Scale
  5 Nov 2019 35 repositories listed Syntology ran 25 of 59 samples · 34 unverified · 52 pointer-only (licence)
  We also present a detailed empirical analysis of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and…
• Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
  26 Dec 2018 13 repositories listed Syntology ran 4 of 10 samples · 6 unverified · 4 pointer-only (licence)
  We introduce an architecture to learn joint multilingual sentence representations for 93 languages, belonging to more than 30 different families and written in 28 different scripts.
• Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languages
  20 Apr 2023 6 repositories listed
  This can for instance be observed when finetuning PLMs on one language and evaluating them on data in a closely related language variety with no standardized orthography.
• Unsupervised Dense Information Retrieval with Contrastive Learning
  16 Dec 2021 6 repositories listed Syntology ran 1 of 5 samples · 4 unverified
  In this work, we explore the limits of contrastive learning as a way to train unsupervised dense retrievers and show that it leads to strong performance in various retrieval settings.
• Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings
  23 Oct 2022 4 repositories listed
  Zero-resource cross-lingual transfer approaches aim to apply supervised models from a source language to unlabelled target languages.
• mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models
  15 Oct 2021 4 repositories listed
  We train a multilingual language model with 24 languages with entity representations and show the model consistently outperforms word-based pretrained models in various cross-lingual transfer tasks.
• InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
  15 Jul 2020 4 repositories listed
  In this work, we present an information-theoretic framework that formulates cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts.
• XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
  24 Mar 2020 4 repositories listed Syntology ran 5 of 10 samples · 5 unverified
  However, these broad-coverage benchmarks have been mostly limited to English, and despite an increasing interest in multilingual models, a benchmark that enables the comprehensive evaluation of such methods on a diverse…
• Pushing the Limits of Low-Resource Morphological Inflection
  16 Aug 2019 4 repositories listed
  Recent years have seen exceptional strides in the task of automatic morphological inflection generation.
• Visually Grounded Reasoning across Languages and Cultures
  28 Sep 2021 3 repositories listed Syntology ran 0 of 4 samples · 4 unverified
  The design of widespread vision-and-language datasets and pre-trained encoders directly adopts, or draws inspiration from, the concepts and images of ImageNet.
• Word Alignment by Fine-tuning Embeddings on Parallel Corpora
  20 Jan 2021 3 repositories listed Syntology ran 2 of 2 samples · 0 unverified
  In addition, we demonstrate that we are able to train multilingual word aligners that can obtain robust performance on different language pairs.
• MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer
  30 Apr 2020 3 repositories listed
  The main goal behind state-of-the-art pre-trained multilingual models such as multilingual BERT and XLM-R is enabling and bootstrapping NLP applications in low-resource languages through zero-shot or few-shot…
• End-to-End Slot Alignment and Recognition for Cross-Lingual NLU
  29 Apr 2020 3 repositories listed
  We introduce MultiATIS++, a new multilingual NLU corpus that extends the Multilingual ATIS corpus to nine languages across four language families, and evaluate our method using the corpus.
• Don't Just Scratch the Surface: Enhancing Word Representations for Korean with Hanja
  25 Aug 2019 3 repositories listed
  We propose a simple yet effective approach for improving Korean word representations using additional linguistic annotation (i.
• Cross-lingual transfer of multilingual models on low resource African Languages
  17 Sep 2024 2 repositories listed
  This study benchmarks the cross-lingual transfer capabilities from a high-resource language to a low-resource language for both, monolingual and multilingual models, focusing on Kinyarwanda and Kirundi, two Bantu…
• Soft Language Prompts for Language Transfer
  2 Jul 2024 2 repositories listed
  We further present to our knowledge the first use of soft prompts for language transfer, a technique we call soft language prompts.
• How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?
  17 Jun 2024 2 repositories listed Syntology ran 9 of 12 samples · 3 unverified
  Large language models (LLMs) have shown remarkable capabilities in many languages beyond English.
• UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages
  14 Jun 2024 2 repositories listed
  Our approach tackles two essential elements of a language model: the initialization of embeddings and the optimal vocabulary size.
• MIND Your Language: A Multilingual Dataset for Cross-lingual News Recommendation
  26 Mar 2024 2 repositories listed
  Our findings reveal that (i) current NNRs, even when based on a multilingual language model, suffer from substantial performance losses under ZS-XLT and that (ii) inclusion of target-language data in FS-XLT training has…
• SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects
  14 Sep 2023 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified
  Despite the progress we have recorded in the last few years in multilingual natural language processing, evaluation is typically limited to a small set of languages with available datasets which excludes a large number…
• The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants
  31 Aug 2023 2 repositories listed Syntology ran 0 of 6 samples · 6 unverified · 6 pointer-only (licence)
  We use this dataset to evaluate the capabilities of multilingual masked language models (MLMs) and large language models (LLMs).
• MT4CrossOIE: Multi-stage Tuning for Cross-lingual Open Information Extraction
  12 Aug 2023 2 repositories listed
  Cross-lingual open information extraction aims to extract structured information from raw text across multiple languages.
• ScandEval: A Benchmark for Scandinavian Natural Language Processing
  3 Apr 2023 2 repositories listed
  This paper introduces a Scandinavian benchmarking platform, ScandEval, which can benchmark any pretrained model on four different tasks in the Scandinavian languages.
• Frustratingly Easy Label Projection for Cross-lingual Transfer
  28 Nov 2022 2 repositories listed Syntology ran 0 of 9 samples · 9 unverified
  Translating training data into many languages has emerged as a practical solution for improving cross-lingual transfer.
• Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models
  22 Oct 2022 2 repositories listed Syntology ran 2 of 6 samples · 4 unverified
  Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks.
• An Empirical Study on Cross-X Transfer for Legal Judgment Prediction
  25 Sep 2022 2 repositories listed
  We find that in both settings (legal areas, origin regions), models trained across all groups perform overall better, while they also have improved results in the worst-case scenarios.
• Few-shot Learning with Multilingual Language Models
  20 Dec 2021 2 repositories listed
  Large-scale generative language models such as GPT-3 are competitive few-shot learners.
• When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer
  27 Oct 2021 2 repositories listed Syntology ran 6 of 6 samples · 0 unverified
  While recent work on multilingual language models has demonstrated their capacity for cross-lingual zero-shot transfer on downstream tasks, there is a lack of consensus in the community as to what shared properties…
• Composable Sparse Fine-Tuning for Cross-Lingual Transfer
  14 Oct 2021 2 repositories listed Syntology ran 1 of 2 samples · 1 unverified · 2 pointer-only (licence)
  Both these masks can then be composed with the pretrained model.
• K-Wav2vec 2.0: Automatic Speech Recognition based on Joint Decoding of Graphemes and Syllables
  11 Oct 2021 2 repositories listed
  Wav2vec 2.

Syntology lines on 13 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.