Natural Language Understanding

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

6 leaderboard tables shown for this task, 6 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

72 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 72 until expanded.

Subtasks archive 2025-07-28

2 subtasks in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 809 papers with code (1,978 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Attention Is All You Need
  12 Jun 2017 595 repositories listed Syntology ran 600 of 946 samples · 346 unverified · 451 pointer-only (licence)
  The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  11 Oct 2018 534 repositories listed Syntology ran 204 of 659 samples · 455 unverified · 149 pointer-only (licence)
  We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers.
• Learning Transferable Visual Models From Natural Language Supervision
  26 Feb 2021 82 repositories listed Syntology ran 16 of 20 samples · 4 unverified · 16 pointer-only (licence)
  State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories.
• ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
  23 Mar 2020 19 repositories listed Syntology ran 26 of 40 samples · 14 unverified · 10 pointer-only (licence)
  Then, instead of training a model that predicts the original identities of the corrupted tokens, we train a discriminative model that predicts whether each token in the corrupted input was replaced by a generator sample…
• A Neural Conversational Model
  19 Jun 2015 18 repositories listed Syntology ran 2 of 4 samples · 2 unverified · 3 pointer-only (licence)
  We find that this straightforward model can generate simple conversations given a large conversational training dataset.
• Cross-lingual Language Model Pretraining
  22 Jan 2019 17 repositories listed Syntology ran 1 of 7 samples · 6 unverified · 1 pointer-only (licence)
  On unsupervised machine translation, we obtain 34.
• BERT for Joint Intent Classification and Slot Filling
  28 Feb 2019 16 repositories listed Syntology ran 9 of 32 samples · 23 unverified · 3 pointer-only (licence)
  Intent classification and slot filling are two essential tasks for natural language understanding.
• Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces
  25 May 2018 16 repositories listed Syntology ran 1 of 15 samples · 14 unverified
  This paper presents the machine learning architecture of the Snips Voice Platform, a software solution to perform Spoken Language Understanding on microprocessors typical of IoT devices.
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
  5 Jun 2020 14 repositories listed Syntology ran 4 of 13 samples · 9 unverified · 3 pointer-only (licence)
  Recent progress in pre-trained neural language models has significantly improved the performance of many natural language processing (NLP) tasks.
• Improving Language Understanding by Generative Pre-Training
  11 Jun 2018 13 repositories listed
  We demonstrate that large gains on these tasks can be realized by generative pre-training of a language model on a diverse corpus of unlabeled text, followed by discriminative fine-tuning on each specific task.
• data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language
  7 Feb 2022 12 repositories listed Syntology ran 0 of 6 samples · 6 unverified
  While the general idea of self-supervised learning is identical across modalities, the actual algorithms and objectives differ widely because they were developed with a single modality in mind.
• Neural Architecture Search with Reinforcement Learning
  5 Nov 2016 12 repositories listed Syntology ran 5 of 5 samples · 0 unverified · 3 pointer-only (licence)
  Our cell achieves a test set perplexity of 62.
• Know What You Don't Know: Unanswerable Questions for SQuAD
  11 Jun 2018 11 repositories listed
  Extractive reading comprehension systems can often locate the correct answer to a question in a context document, but they also tend to make unreliable guesses on questions for which the correct answer is not stated in…
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
  20 Apr 2018 11 repositories listed Syntology ran 5 of 14 samples · 9 unverified · 1 pointer-only (licence)
  For natural language understanding (NLU) technology to be maximally useful, both practically and as a scientific object of study, it must be general: it must be able to process language in a way that is not exclusively…
• GPT Understands, Too
  18 Mar 2021 10 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 2 pointer-only (licence)
  Prompting a pretrained language model with natural language patterns has been proved effective for natural language understanding (NLU).
• TinyBERT: Distilling BERT for Natural Language Understanding
  23 Sep 2019 10 repositories listed Syntology ran 0 of 4 samples · 4 unverified · 4 pointer-only (licence)
  To accelerate inference and reduce model size while maintaining accuracy, we first propose a novel Transformer distillation method that is specially designed for knowledge distillation (KD) of the Transformer-based…
• NEZHA: Neural Contextualized Representation for Chinese Language Understanding
  31 Aug 2019 10 repositories listed
  The pre-trained language models have achieved great successes in various natural language understanding (NLU) tasks due to its capacity to capture the deep contextualized information in text by pre-training on…
• ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
  27 Apr 2020 9 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  ColBERT introduces a late interaction architecture that independently encodes the query and the document using BERT and then employs a cheap yet powerful interaction step that models their fine-grained similarity.
• Unified Language Model Pre-training for Natural Language Understanding and Generation
  8 May 2019 9 repositories listed
  This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks.
• Benchmarking Natural Language Understanding Services for building Conversational Agents
  13 Mar 2019 9 repositories listed
  We have recently seen the emergence of several publicly available Natural Language Understanding (NLU) toolkits, which map user utterances to structured, but more abstract, Dialogue Act (DA) or Intent specifications,…
• GLM: General Language Model Pretraining with Autoregressive Blank Infilling
  18 Mar 2021 8 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  On a wide range of tasks across NLU, conditional and unconditional generation, GLM outperforms BERT, T5, and GPT given the same model sizes and data, and achieves the best performance from a single pretrained model with…
• ConvBERT: Improving BERT with Span-based Dynamic Convolution
  6 Aug 2020 8 repositories listed
  The novel convolution heads, together with the rest self-attention heads, form a new mixed attention block that is more efficient at both global and local context learning.
• I-BERT: Integer-only BERT Quantization
  5 Jan 2021 7 repositories listed
  Transformer based models, like BERT and RoBERTa, have achieved state-of-the-art results in many Natural Language Processing tasks.
• Leveraging Pre-trained Checkpoints for Sequence Generation Tasks
  29 Jul 2019 7 repositories listed Syntology ran 0 of 6 samples · 6 unverified
  Unsupervised pre-training of large neural models has recently revolutionized Natural Language Processing.
• Multi-Task Deep Neural Networks for Natural Language Understanding
  31 Jan 2019 7 repositories listed Syntology ran 5 of 13 samples · 8 unverified · 2 pointer-only (licence)
  In this paper, we present a Multi-Task Deep Neural Network (MT-DNN) for learning representations across multiple natural language understanding (NLU) tasks.
• MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages
  18 Apr 2022 6 repositories listed Syntology ran 1 of 10 samples · 9 unverified · 1 pointer-only (licence)
  We present the MASSIVE dataset--Multilingual Amazon Slu resource package (SLURP) for Slot-filling, Intent classification, and Virtual assistant Evaluation.
• A Relational Tsetlin Machine with Applications to Natural Language Understanding
  22 Feb 2021 6 repositories listed
  TMs are a pattern recognition approach that uses finite state machines for learning and propositional logic to represent patterns.
• SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
  8 Nov 2019 6 repositories listed Syntology ran 6 of 8 samples · 2 unverified · 1 pointer-only (licence)
  However, due to limited data resources from downstream tasks and the extremely large capacity of pre-trained models, aggressive fine-tuning often causes the adapted model to overfit the data of downstream tasks and…
• Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language
  14 Dec 2022 5 repositories listed Syntology ran 5 of 8 samples · 3 unverified
  Current self-supervised learning algorithms are often modality-specific and require large amounts of computational resources.
• PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
  26 Apr 2021 5 repositories listed
  To enhance the generalization ability of PanGu-α, we collect 1.

Syntology lines on 21 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.