Methods › Natural Language Processing › Autoencoding Transformers › XLM
XLM
Introduced by Guillaume Lample et al. in Cross-lingual Language Model Pretraining
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
XLM is a Transformer based architecture that is pre-trained using one of three language modelling objectives:
- Causal Language Modeling - models the probability of a word given the previous words in a sentence.
- Masked Language Modeling - the masked language modeling objective of BERT.
- Translation Language Modeling - a (new) translation language modeling objective for improving cross-lingual pre-training.
The authors find that both the CLM and MLM approaches provide strong cross-lingual features that can be used for pretraining models.
Papers archive 2025-07-28
30 shown of 57, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
BioBridge: Unified Bio-Embedding with Bridging Modality in Code-Switched EMR 16 Dec 2024 · 1 repository · arXiv:2412.11671
-
XLM for Autonomous Driving Systems: A Comprehensive Review 16 Sep 2024 · 0 repositories · arXiv:2409.10484
-
Ax-to-Grind Urdu: Benchmark Dataset for Urdu Fake News Detection 20 Mar 2024 · 1 repository · arXiv:2403.14037
-
Mapping Transformer Leveraged Embeddings for Cross-Lingual Document Representation 12 Jan 2024 · 1 repository · arXiv:2401.06583
-
An Empirical study of Unsupervised Neural Machine Translation: analyzing NMT output, model's behavior and sentences' contribution 19 Dec 2023 · 0 repositories · arXiv:2312.12588
-
MedAI Dialog Corpus (MEDIC): Zero-Shot Classification of Doctor and AI Responses in Health Consultations 19 Oct 2023 · 0 repositories · arXiv:2310.12489
-
Benchmarking Procedural Language Understanding for Low-Resource Languages: A Case Study on Turkish 13 Sep 2023 · 1 repository · arXiv:2309.06698
-
PESTS: Persian_English Cross Lingual Corpus for Semantic Textual Similarity 13 May 2023 · 0 repositories · arXiv:2305.07893
-
CLaC at SemEval-2023 Task 2: Comparing Span-Prediction and Sequence-Labeling approaches for NER 5 May 2023 · 0 repositories · arXiv:2305.03845
-
Exploring Methods for Building Dialects-Mandarin Code-Mixing Corpora: A Case Study in Taiwanese Hokkien 21 Jan 2023 · 1 repository · arXiv:2301.08937
-
ALIGN-MLM: Word Embedding Alignment is Crucial for Multilingual Pre-training 15 Nov 2022 · 1 repository · arXiv:2211.08547
-
BERT-Sort: A Zero-shot MLM Semantic Encoder on Ordinal Features for AutoML 1 Jun 2022 · 1 repository
-
GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models 24 May 2022 · 1 repository · arXiv:2205.12247
-
Persian Natural Language Inference: A Meta-learning approach 18 May 2022 · 1 repository · arXiv:2205.08755
-
HiNER: A Large Hindi Named Entity Recognition Dataset 28 Apr 2022 · 1 repository · arXiv:2204.13743
-
Team ÚFAL at CMCL 2022 Shared Task: Figuring out the correct recipe for predicting Eye-Tracking features using Pretrained Language Models 11 Apr 2022 · 0 repositories · arXiv:2204.04998
-
Are You Robert or RoBERTa? Deceiving Online Authorship Attribution Models Using Neural Text Generators 18 Mar 2022 · 0 repositories · arXiv:2203.09813
-
"A Passage to India": Pre-trained Word Embeddings for Indian Languages 27 Dec 2021 · 0 repositories · arXiv:2112.13800
-
Prix-LM: Pretraining for Multilingual Knowledge Base Construction 16 Oct 2021 · 1 repository · arXiv:2110.08443
-
Cross-Language Learning for Entity Matching 7 Oct 2021 · 1 repository · arXiv:2110.03338
-
TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation 27 Sep 2021 · 3 repositories · arXiv:2109.13296Syntology ran 0 of 7 samples · 7 unverified
-
Do Images really do the Talking? Analysing the significance of Images in Tamil Troll meme classification 9 Aug 2021 · 1 repository · arXiv:2108.03886
-
PAW at SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation : Exploring Cross Lingual Transfer, Augmentations and Adversarial Training 1 Aug 2021 · 0 repositories
-
A Primer on Pretrained Multilingual Language Models 1 Jul 2021 · 0 repositories · arXiv:2107.00676
-
LAWDR: Language-Agnostic Weighted Document Representations from Pre-trained Models 7 Jun 2021 · 0 repositories · arXiv:2106.03379
-
Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining 21 May 2021 · 0 repositories · arXiv:2105.10419
-
TeamUNCC@LT-EDI-EACL2021: Hope Speech Detection using Transfer Learning with Transformers 19 Apr 2021 · 1 repository
-
Multilingual Language Models Predict Human Reading Behavior 12 Apr 2021 · 1 repository · arXiv:2104.05433
-
Low-Resource Machine Translation Training Curriculum Fit for Low-Resource Languages 24 Mar 2021 · 0 repositories · arXiv:2103.13272
-
LightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation 11 Mar 2021 · 0 repositories · arXiv:2103.06418
Tasks archive 2025-07-28
20 shown of 106 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections