Multilingual NLP

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

8 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 42 papers with code (96 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Unsupervised Cross-lingual Representation Learning at Scale
  5 Nov 2019 35 repositories listed Syntology ran 25 of 59 samples · 34 unverified · 52 pointer-only (licence)
  We also present a detailed empirical analysis of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and…
• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
  9 Nov 2022 7 repositories listed Syntology ran 2 of 11 samples · 9 unverified
  Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
• Language-agnostic BERT Sentence Embedding
  3 Jul 2020 6 repositories listed
  While BERT is an effective method for learning monolingual sentence embeddings for semantic similarity and embedding based transfer learning (Reimers and Gurevych, 2019), BERT based cross-lingual sentence embeddings…
• UQA: Corpus for Urdu Question Answering
  2 May 2024 3 repositories listed
  This paper introduces UQA, a novel dataset for question answering and text comprehension in Urdu, a low-resource language with over 70 million native speakers.
• What is "Typological Diversity" in NLP?
  6 Feb 2024 2 repositories listed
  We recommend future work to include an operationalization of 'typological diversity' that empirically justifies the diversity of language samples.
• Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs
  22 May 2023 2 repositories listed
  ColexNet's nodes are concepts and its edges are colexifications.
• XeroAlign: Zero-Shot Cross-lingual Transformer Alignment
  6 May 2021 2 repositories listed
  The introduction of pretrained cross-lingual language models brought decisive improvements to multilingual NLP tasks.
• PMIndia -- A Collection of Parallel Corpora of Languages of India
  27 Jan 2020 2 repositories listed
  Parallel text is required for building high-quality machine translation (MT) systems, as well as for other multilingual NLP applications.
• MMCR4NLP: Multilingual Multiway Corpora Repository for Natural Language Processing
  3 Oct 2017 2 repositories listed
  Multilinguality is gradually becoming ubiquitous in the sense that more and more researchers have successfully shown that using additional languages help improve the results in many Natural Language Processing tasks.
• News Without Borders: Domain Adaptation of Multilingual Sentence Embeddings for Cross-lingual News Recommendation
  18 Jun 2024 1 repository listed
  Rapidly growing numbers of multilingual news consumers pose an increasing challenge to news recommender systems in terms of providing customized recommendations.
• ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models
  13 Jun 2024 1 repository listed Syntology ran 3 of 7 samples · 4 unverified
  Performance prediction is a method to estimate the performance of Language Models (LMs) on various Natural Language Processing (NLP) tasks, mitigating computational costs associated with model capacity and data for…
• Analyzing Language Bias Between French and English in Conventional Multilingual Sentiment Analysis Models
  7 May 2024 1 repository listed
  However, Fairlearn's metrics suggest that the SVM approaches equitable levels with a demographic parity ratio of 0.
• What Drives Performance in Multilingual Language Models?
  29 Apr 2024 1 repository listed
  This study investigates the factors influencing the performance of multilingual large language models (MLLMs) across diverse languages.
• A Measure for Transparent Comparison of Linguistic Diversity in Multilingual NLP Data Sets
  6 Mar 2024 1 repository listed
  Typologically diverse benchmarks are increasingly created to track the progress achieved in multilingual NLP.
• Self-Augmented In-Context Learning for Unsupervised Word Translation
  15 Feb 2024 1 repository listed Syntology ran 7 of 10 samples · 3 unverified
  Recent work has shown that, while large language models (LLMs) demonstrate strong word translation or bilingual lexicon induction (BLI) capabilities in few-shot setups, they still cannot match the performance of…
• On Bilingual Lexicon Induction with Large Language Models
  21 Oct 2023 1 repository listed Syntology ran 3 of 4 samples · 1 unverified
  Bilingual Lexicon Induction (BLI) is a core task in multilingual NLP that still, to a large extent, relies on calculating cross-lingual word representations.
• BaitBuster-Bangla: A Comprehensive Dataset for Clickbait Detection in Bangla with Multi-Feature and Multi-Modal Analysis
  13 Oct 2023 1 repository listed
  This study presents a large multi-modal Bangla YouTube clickbait dataset consisting of 253, 070 data points collected through an automated process using the YouTube API and Python web automation frameworks.
• Evaluating The Effectiveness of Capsule Neural Network in Toxic Comment Classification using Pre-trained BERT Embeddings
  12 Oct 2023 1 repository listed
  By comparing the performance of CapsNet to that of other architectures, such as DistilBERT, Vanilla Neural Networks (VNN), and Convolutional Neural Networks (CNN), we were able to achieve an accuracy of 90.
• Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca
  16 Sep 2023 1 repository listed
  Foundational large language models (LLMs) can be instruction-tuned to perform open-domain question answering, facilitating applications like chat assistants.
• XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages
  19 May 2023 1 repository listed Syntology ran 0 of 11 samples · 11 unverified
  We evaluate commonly used models on the benchmark.
• PMIndiaSum: Multilingual and Cross-lingual Headline Summarization for Languages in India
  15 May 2023 1 repository listed
  This paper introduces PMIndiaSum, a multilingual and massively parallel summarization corpus focused on languages in India.
• A General-Purpose Multilingual Document Encoder
  11 May 2023 1 repository listed
  While a large body of work leveraged MMTs to mine parallel data and induce bilingual document embeddings, much less effort has been devoted to training general-purpose (massively) multilingual document encoder that can…
• PEACH: Pre-Training Sequence-to-Sequence Multilingual Models for Translation with Semi-Supervised Pseudo-Parallel Document Generation
  3 Apr 2023 1 repository listed
  Multilingual pre-training on monolingual data ignores the availability of parallel data in many language pairs.
• Improving Bilingual Lexicon Induction with Cross-Encoder Reranking
  30 Oct 2022 1 repository listed Syntology ran 0 of 4 samples · 4 unverified
  This crucial step is done via 1) creating a word similarity dataset, comprising positive word pairs (i.
• HumSet: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crisis Response
  10 Oct 2022 1 repository listed
  Timely and effective response to humanitarian crises requires quick and accurate analysis of large amounts of text data - a process that can highly benefit from expert-assisted NLP systems trained on validated and…
• DetIE: Multilingual Open Information Extraction Inspired by Object Detection
  24 Jun 2022 1 repository listed
  Our model sets the new state of the art performance of 67.
• TeDDi Sample: Text Data Diversity Sample for Language Comparison and Multilingual NLP
  1 Jun 2022 1 repository listed
  We present the TeDDi sample, a diversity sample of text data for language comparison and multilingual Natural Language Processing.
• Improving Word Translation via Two-Stage Contrastive Learning
  15 Mar 2022 1 repository listed Syntology ran 5 of 6 samples · 1 unverified · 4 pointer-only (licence)
  At Stage C1, we propose to refine standard cross-lingual linear maps between static word embeddings (WEs) via a contrastive learning objective; we also show how to integrate it into the self-learning procedure for even…
• Improving Word Translation via Two-Stage Contrastive Learning
  16 Nov 2021 1 repository listed
  As Stage C1, we propose to refine standard cross-lingual linear maps between static word embeddings (WEs) via a contrastive learning objective; we also show how to integrate it into the self-learning procedure for even…
• WikiNEuRal: Combined Neural and Knowledge-based Silver Data Creation for Multilingual NER
  1 Nov 2021 1 repository listed
  Multilingual Named Entity Recognition (NER) is a key intermediate task which is needed in many areas of NLP.

Syntology lines on 8 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.