XLM-R

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

1 dataset whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 99 papers with code (221 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Unsupervised Cross-lingual Representation Learning at Scale
  5 Nov 2019 35 repositories listed Syntology ran 25 of 59 samples · 34 unverified · 52 pointer-only (licence)
  We also present a detailed empirical analysis of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and…
• AdapterHub: A Framework for Adapting Transformers
  15 Jul 2020 9 repositories listed Syntology ran 5 of 15 samples · 10 unverified · 12 pointer-only (licence)
  We propose AdapterHub, a framework that allows dynamic "stitching-in" of pre-trained adapters for different tasks and languages.
• MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages
  18 Apr 2022 6 repositories listed Syntology ran 1 of 10 samples · 9 unverified · 1 pointer-only (licence)
  We present the MASSIVE dataset--Multilingual Amazon Slu resource package (SLURP) for Slot-filling, Intent classification, and Virtual assistant Evaluation.
• Emotion Classification in a Resource Constrained Language Using Transformer-based Approach
  17 Apr 2021 4 repositories listed
  A Bengali emotion corpus consists of 6243 texts is developed for the classification task.
• XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models
  25 Jan 2023 3 repositories listed
  Large multilingual language models typically rely on a single vocabulary shared across 100+ languages.
• DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing
  18 Nov 2021 3 repositories listed Syntology ran 0 of 7 samples · 7 unverified
  We thus propose a new gradient-disentangled embedding sharing method that avoids the tug-of-war dynamics, improving both training efficiency and the quality of the pre-trained model.
• BERTweet: A pre-trained language model for English Tweets
  20 May 2020 3 repositories listed
  We present BERTweet, the first public large-scale pre-trained language model for English Tweets.
• MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer
  30 Apr 2020 3 repositories listed
  The main goal behind state-of-the-art pre-trained multilingual models such as multilingual BERT and XLM-R is enabling and bootstrapping NLP applications in low-resource languages through zero-shot or few-shot…
• From N-grams to Pre-trained Multilingual Models For Language Identification
  11 Oct 2024 2 repositories listed
  In this paper, we investigate the use of N-gram models and Large Pre-trained Multilingual models for Language Identification (LID) across 11 South African languages.
• Zero-Shot Tokenizer Transfer
  13 May 2024 2 repositories listed Syntology ran 7 of 12 samples · 5 unverified · 4 pointer-only (licence)
  Finally, we show that a ZeTT hypernetwork trained for a base (L)LM can also be applied to fine-tuned variants without extra training.
• FOCUS: Effective Embedding Initialization for Monolingual Specialization of Multilingual Models
  23 May 2023 2 repositories listed
  However, if we want to use a new tokenizer specialized for the target language, we cannot transfer the source model's embedding matrix.
• DUMB: A Benchmark for Smart Evaluation of Dutch Models
  22 May 2023 2 repositories listed
  The benchmark includes a diverse set of datasets for low-, medium- and high-resource tasks.
• GreekBART: The First Pretrained Greek Sequence-to-Sequence Model
  3 Apr 2023 2 repositories listed
  In addition, we examine its performance on two NLG tasks from GreekSUM, a newly introduced summarization dataset for the Greek language.
• X²-VLM: All-In-One Pre-trained Model For Vision-Language Tasks
  22 Nov 2022 2 repositories listed Syntology ran 2 of 6 samples · 4 unverified · 6 pointer-only (licence)
  Vision language pre-training aims to learn alignments between vision and language from a large amount of data.
• AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities
  12 Nov 2022 2 repositories listed Syntology ran 2 of 11 samples · 9 unverified
  In this work, we present a conceptually simple and effective method to train a strong bilingual/multilingual multimodal representation model.
• XeroAlign: Zero-Shot Cross-lingual Transformer Alignment
  6 May 2021 2 repositories listed
  The introduction of pretrained cross-lingual language models brought decisive improvements to multilingual NLP tasks.
• MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
  31 Dec 2020 2 repositories listed
  We generalize deep self-attention distillation in MiniLM (Wang et al., 2020) by only using self-attention relation distillation for task-agnostic compression of pretrained Transformers.
• ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic
  27 Dec 2020 2 repositories listed
  To evaluate our models, we also introduce ARLUE, a new benchmark for multi-dialectal Arabic language understanding evaluation.
• Applying Occam's Razor to Transformer-Based Dependency Parsing: What Works, What Doesn't, and What is Really Necessary
  23 Oct 2020 2 repositories listed
  We find that the choice of pre-trained embeddings has by far the greatest impact on parser performance and identify XLM-R as a robust choice across the languages in our study.
• A Bayesian Multilingual Document Model for Zero-shot Topic Identification and Discovery
  2 Jul 2020 2 repositories listed
  In this paper, we present a Bayesian multilingual document model for learning language-independent document embeddings.
• XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
  3 Apr 2020 2 repositories listed
  In this paper, we introduce XGLUE, a new benchmark dataset that can be used to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora and evaluate their performance across a diverse…
• Multilingual Encoder Knows more than You Realize: Shared Weights Pretraining for Extremely Low-Resource Languages
  15 Feb 2025 1 repository listed
  While multilingual language models like XLM-R have advanced multilingualism in NLP, they still perform poorly in extremely low-resource languages.
• LangSAMP: Language-Script Aware Multilingual Pretraining
  26 Sep 2024 1 repository listed
  However, this removal increases the burden on token embeddings to encode all language-specific information, which may hinder the model's ability to produce more language-neutral representations.
• GrEmLIn: A Repository of Green Baseline Embeddings for 87 Low-Resource Languages Injected with Multilingual Graph Knowledge
  26 Sep 2024 1 repository listed
  Contextualized embeddings based on large language models (LLMs) are available for various languages, but their coverage is often limited for lower resourced languages.
• Medical Spoken Named Entity Recognition
  19 Jun 2024 1 repository listed
  To our knowledge, our Vietnamese real-world dataset is the largest spoken NER dataset in the world regarding the number of entity types, featuring 18 distinct types.
• Exploring Alignment in Shared Cross-lingual Spaces
  23 May 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Our analysis focuses on quantifying the \textit{alignment} and \textit{overlap} of these concepts across various languages within the latent space.
• Cross-Lingual Transfer Robustness to Lower-Resource Languages on Adversarial Datasets
  29 Mar 2024 1 repository listed
  Multilingual Language Models (MLLMs) exhibit robust cross-lingual transfer capabilities, or the ability to leverage information acquired in a source language and apply it to a target language.
• CICLe: Conformal In-Context Learning for Largescale Multi-Class Food Risk Classification
  18 Mar 2024 1 repository listed
  Contaminated or adulterated food poses a substantial risk to human health.
• KBioXLM: A Knowledge-anchored Biomedical Multilingual Pretrained Language Model
  20 Nov 2023 1 repository listed
  We achieve a biomedical multilingual corpus by incorporating three granularity knowledge alignments (entity, fact, and passage levels) into monolingual corpora.
• MELA: Multilingual Evaluation of Linguistic Acceptability
  15 Nov 2023 1 repository listed
  In this work, we present the largest benchmark to date on linguistic acceptability: Multilingual Evaluation of Linguistic Acceptability -- MELA, with 46K samples covering 10 languages from a diverse set of language…

Syntology lines on 8 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.