Methods › Natural Language Processing › Autoencoding Transformers › XLM
XLM
Introduced by Guillaume Lample et al. in Cross-lingual Language Model Pretraining
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
XLM is a Transformer based architecture that is pre-trained using one of three language modelling objectives:
- Causal Language Modeling - models the probability of a word given the previous words in a sentence.
- Masked Language Modeling - the masked language modeling objective of BERT.
- Translation Language Modeling - a (new) translation language modeling objective for improving cross-lingual pre-training.
The authors find that both the CLM and MLM approaches provide strong cross-lingual features that can be used for pretraining models.
In Syntology about another paper, arXiv:2512.17065: repository known, no verified implementation (declared_code; repositories: 1 listed of 1 in the graph, strongest link: listed only). The lookup matched a paper by its title, arXiv:2512.17065, not a method entry. About another paper: this answer is about arXiv:2512.17065, while the archive credits arXiv:1901.07291.
Origin paper of this answer, arXiv:2512.17065 (its page on this site · its page on Syntology): in Syntology's graph layer with no harvested code sample.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 57 · with a code link: 27 · where Syntology ran a sample: 6 (5 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (6 of 57 tagged: 5 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 57 of 57, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
BioBridge: Unified Bio-Embedding with Bridging Modality in Code-Switched EMR 16 Dec 2024 · 1 repository · arXiv:2412.11671
-
XLM for Autonomous Driving Systems: A Comprehensive Review 16 Sep 2024 · 0 repositories · arXiv:2409.10484
-
Ax-to-Grind Urdu: Benchmark Dataset for Urdu Fake News Detection 20 Mar 2024 · 1 repository · arXiv:2403.14037
-
Mapping Transformer Leveraged Embeddings for Cross-Lingual Document Representation 12 Jan 2024 · 1 repository · arXiv:2401.06583
-
An Empirical study of Unsupervised Neural Machine Translation: analyzing NMT output, model's behavior and sentences' contribution 19 Dec 2023 · 0 repositories · arXiv:2312.12588
-
MedAI Dialog Corpus (MEDIC): Zero-Shot Classification of Doctor and AI Responses in Health Consultations 19 Oct 2023 · 0 repositories · arXiv:2310.12489
-
Benchmarking Procedural Language Understanding for Low-Resource Languages: A Case Study on Turkish 13 Sep 2023 · 1 repository · arXiv:2309.06698
-
PESTS: Persian_English Cross Lingual Corpus for Semantic Textual Similarity 13 May 2023 · 0 repositories · arXiv:2305.07893
-
CLaC at SemEval-2023 Task 2: Comparing Span-Prediction and Sequence-Labeling approaches for NER 5 May 2023 · 0 repositories · arXiv:2305.03845
-
Exploring Methods for Building Dialects-Mandarin Code-Mixing Corpora: A Case Study in Taiwanese Hokkien 21 Jan 2023 · 1 repository · arXiv:2301.08937
-
ALIGN-MLM: Word Embedding Alignment is Crucial for Multilingual Pre-training 15 Nov 2022 · 1 repository · arXiv:2211.08547
-
BERT-Sort: A Zero-shot MLM Semantic Encoder on Ordinal Features for AutoML 1 Jun 2022 · 1 repository
-
GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models 24 May 2022 · 1 repository · arXiv:2205.12247
-
Persian Natural Language Inference: A Meta-learning approach 18 May 2022 · 1 repository · arXiv:2205.08755
-
HiNER: A Large Hindi Named Entity Recognition Dataset 28 Apr 2022 · 1 repository · arXiv:2204.13743
-
Team ÚFAL at CMCL 2022 Shared Task: Figuring out the correct recipe for predicting Eye-Tracking features using Pretrained Language Models 11 Apr 2022 · 0 repositories · arXiv:2204.04998
-
Are You Robert or RoBERTa? Deceiving Online Authorship Attribution Models Using Neural Text Generators 18 Mar 2022 · 0 repositories · arXiv:2203.09813
-
"A Passage to India": Pre-trained Word Embeddings for Indian Languages 27 Dec 2021 · 0 repositories · arXiv:2112.13800
-
Prix-LM: Pretraining for Multilingual Knowledge Base Construction 16 Oct 2021 · 1 repository · arXiv:2110.08443
-
Cross-Language Learning for Entity Matching 7 Oct 2021 · 1 repository · arXiv:2110.03338
-
TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation 27 Sep 2021 · 3 repositories · arXiv:2109.13296Syntology 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)
-
Do Images really do the Talking? Analysing the significance of Images in Tamil Troll meme classification 9 Aug 2021 · 1 repository · arXiv:2108.03886
-
PAW at SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation : Exploring Cross Lingual Transfer, Augmentations and Adversarial Training 1 Aug 2021 · 0 repositories
-
A Primer on Pretrained Multilingual Language Models 1 Jul 2021 · 0 repositories · arXiv:2107.00676
-
LAWDR: Language-Agnostic Weighted Document Representations from Pre-trained Models 7 Jun 2021 · 0 repositories · arXiv:2106.03379
-
Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining 21 May 2021 · 0 repositories · arXiv:2105.10419
-
TeamUNCC@LT-EDI-EACL2021: Hope Speech Detection using Transfer Learning with Transformers 19 Apr 2021 · 1 repository
-
Multilingual Language Models Predict Human Reading Behavior 12 Apr 2021 · 1 repository · arXiv:2104.05433
-
Low-Resource Machine Translation Training Curriculum Fit for Low-Resource Languages 24 Mar 2021 · 0 repositories · arXiv:2103.13272
-
LightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation 11 Mar 2021 · 0 repositories · arXiv:2103.06418
-
Majority Voting with Bidirectional Pre-translation For Bitext Retrieval 10 Mar 2021 · 1 repository · arXiv:2103.06369
-
Analyzing Zero-shot Cross-lingual Transfer in Supervised NLP Tasks 26 Jan 2021 · 0 repositories · arXiv:2101.10649
-
Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval 21 Jan 2021 · 1 repository · arXiv:2101.08370Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Hitachi at SemEval-2020 Task 11: An Empirical Study of Pre-Trained Transformer Family for Propaganda Detection 1 Dec 2020 · 0 repositories
-
Multilingual BERT Post-Pretraining Alignment 23 Oct 2020 · 0 repositories · arXiv:2010.12547
-
Probing for Multilingual Numerical Understanding in Transformer-Based Language Models 13 Oct 2020 · 1 repository · arXiv:2010.06666
-
Chatbot Interaction with Artificial Intelligence: Human Data Augmentation with T5 and Language Transformer Ensemble for Text Classification 12 Oct 2020 · 0 repositories · arXiv:2010.05990
-
SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task 11 Oct 2020 · 0 repositories · arXiv:2010.05122
-
Evaluating Multilingual BERT for Estonian 1 Oct 2020 · 0 repositories · arXiv:2010.00454
-
A Vietnamese Dataset for Evaluating Machine Reading Comprehension 30 Sep 2020 · 0 repositories · arXiv:2009.14725
-
What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties 27 Sep 2020 · 0 repositories · arXiv:2009.12862
-
FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding 10 Sep 2020 · 1 repository · arXiv:2009.05166Syntology 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 1 pointer-only (licence)
-
On Learning Universal Representations Across Languages 31 Jul 2020 · 0 repositories · arXiv:2007.15960
-
Cooking Is All About People: Comment Classification On Cookery Channels Using BERT and Classification Models (Malayalam-English Mix-Code) 15 Jun 2020 · 0 repositories · arXiv:2007.04249
-
``A Passage to India'': Pre-trained Word Embeddings for Indian Languages 1 May 2020 · 0 repositories
-
KLEJ: Comprehensive Benchmark for Polish Language Understanding 1 May 2020 · 1 repository · arXiv:2005.00630Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)
-
Transfer learning applied to text classification in Spanish radiological reports 1 May 2020 · 0 repositories
-
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation 3 Apr 2020 · 2 repositories · arXiv:2004.01401
-
Unsupervised Cross-lingual Representation Learning at Scale 5 Nov 2019 · 35 repositories · arXiv:1911.02116Syntology official (archive's flag): 12 ran · 40 ran (of which 13 constructed an object rather than computing a result; 35 with no instrument failure: 4 honoured, 1 violated, 30 with no contract checked; 5 where Syntology's instrument failed) · 19 unverified (of 59 harvested samples) · 52 pointer-only (licence)
-
Masked Language Model Scoring 31 Oct 2019 · 6 repositories · arXiv:1910.14659Syntology official: no sample here; runs from other or unrecorded repositories · 10 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 0 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 10 harvested samples) · 3 pointer-only (licence)
-
Multilingual Question Answering from Formatted Text applied to Conversational Agents 10 Oct 2019 · 0 repositories · arXiv:1910.04659
-
Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing 15 Sep 2019 · 1 repository · arXiv:1909.06775
-
Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks 3 Sep 2019 · 0 repositories · arXiv:1909.00964
-
Detecting Toxicity in News Articles: Application to Bulgarian 26 Aug 2019 · 1 repository · arXiv:1908.09785
-
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training 16 Aug 2019 · 0 repositories · arXiv:1908.06066
-
Unbabel's Participation in the WMT19 Translation Quality Estimation Shared Task 24 Jul 2019 · 0 repositories · arXiv:1907.10352
-
Cross-lingual Language Model Pretraining 22 Jan 2019 · 17 repositories · arXiv:1901.07291Syntology 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 1 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 1 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 106 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections