Linguistic Acceptability

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

5 leaderboard tables shown for this task, 5 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

5 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 49 papers with code (72 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  11 Oct 2018 534 repositories listed Syntology ran 204 of 659 samples · 455 unverified · 149 pointer-only (licence)
  We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers.
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
  26 Jul 2019 67 repositories listed Syntology ran 22 of 48 samples · 26 unverified · 23 pointer-only (licence)
  Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
  23 Oct 2019 57 repositories listed Syntology ran 2 of 31 samples · 29 unverified
  Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP).
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
  26 Sep 2019 48 repositories listed Syntology ran 46 of 126 samples · 80 unverified · 22 pointer-only (licence)
  Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks.
• DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
  2 Oct 2019 37 repositories listed Syntology ran 19 of 27 samples · 8 unverified
  As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or…
• Big Bird: Transformers for Longer Sequences
  28 Jul 2020 14 repositories listed Syntology ran 10 of 15 samples · 5 unverified · 11 pointer-only (licence)
  To remedy this, we propose, BigBird, a sparse attention mechanism that reduces this quadratic dependency to linear.
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
  5 Jun 2020 14 repositories listed Syntology ran 4 of 13 samples · 9 unverified · 3 pointer-only (licence)
  Recent progress in pre-trained neural language models has significantly improved the performance of many natural language processing (NLP) tasks.
• data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language
  7 Feb 2022 12 repositories listed Syntology ran 0 of 6 samples · 6 unverified
  While the general idea of self-supervised learning is identical across modalities, the actual algorithms and objectives differ widely because they were developed with a single modality in mind.
• FNet: Mixing Tokens with Fourier Transforms
  9 May 2021 12 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 1 pointer-only (licence)
  At longer input lengths, our FNet model is significantly faster: when compared to the "efficient" Transformers on the Long Range Arena benchmark, FNet matches the accuracy of the most accurate models, while outpacing…
• TinyBERT: Distilling BERT for Natural Language Understanding
  23 Sep 2019 10 repositories listed Syntology ran 0 of 4 samples · 4 unverified · 4 pointer-only (licence)
  To accelerate inference and reduce model size while maintaining accuracy, we first propose a novel Transformer distillation method that is specially designed for knowledge distillation (KD) of the Transformer-based…
• Multi-Task Deep Neural Networks for Natural Language Understanding
  31 Jan 2019 7 repositories listed Syntology ran 5 of 13 samples · 8 unverified · 2 pointer-only (licence)
  In this paper, we present a Multi-Task Deep Neural Network (MT-DNN) for learning representations across multiple natural language understanding (NLU) tasks.
• SqueezeBERT: What can computer vision teach NLP about efficient neural networks?
  19 Jun 2020 6 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  Humans read and write hundreds of billions of messages every day.
• SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
  8 Nov 2019 6 repositories listed Syntology ran 6 of 8 samples · 2 unverified · 1 pointer-only (licence)
  However, due to limited data resources from downstream tasks and the extremely large capacity of pre-trained models, aggressive fine-tuning often causes the adapted model to overfit the data of downstream tasks and…
• Masked Language Model Scoring
  31 Oct 2019 6 repositories listed Syntology ran 5 of 10 samples · 5 unverified · 3 pointer-only (licence)
  Instead, we evaluate MLMs out of the box via their pseudo-log-likelihood scores (PLLs), which are computed by masking tokens one by one.
• SpanBERT: Improving Pre-training by Representing and Predicting Spans
  24 Jul 2019 6 repositories listed Syntology ran 3 of 15 samples · 12 unverified · 4 pointer-only (licence)
  We present SpanBERT, a pre-training method that is designed to better represent and predict spans of text.
• RealFormer: Transformer Likes Residual Attention
  21 Dec 2020 5 repositories listed
  Transformer is the backbone of modern NLP models.
• Q8BERT: Quantized 8Bit BERT
  14 Oct 2019 5 repositories listed Syntology ran 3 of 11 samples · 8 unverified · 3 pointer-only (licence)
  Recently, pre-trained Transformer based language models such as BERT and GPT, have shown great improvement in many Natural Language Processing (NLP) tasks.
• LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
  15 Aug 2022 4 repositories listed Syntology ran 2 of 5 samples · 3 unverified
  We develop a procedure for Int8 matrix multiplication for feed-forward and attention projection layers in transformers, which cut the memory needed for inference by half while retaining full precision performance.
• How to Train BERT with an Academic Budget
  15 Apr 2021 4 repositories listed
  While large language models a la BERT are used ubiquitously in NLP, pretraining them is considered a luxury that only a few well-funded industry labs can afford.
• Entailment as Few-Shot Learner
  29 Apr 2021 3 repositories listed Syntology ran 1 of 3 samples · 2 unverified
  Large pre-trained language models (LMs) have demonstrated remarkable ability as few-shot learners.
• GeDi: Generative Discriminator Guided Sequence Generation
  14 Sep 2020 3 repositories listed Syntology ran 5 of 11 samples · 6 unverified
  While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of the distribution they generate.
• ERNIE 2.0: A Continual Pre-training Framework for Language Understanding
  29 Jul 2019 3 repositories listed Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
  Recently, pre-trained models have achieved state-of-the-art results in various language understanding tasks, which indicates that pre-training on large-scale corpora may play a crucial role in natural language…
• JCoLA: Japanese Corpus of Linguistic Acceptability
  22 Sep 2023 2 repositories listed
  In this paper, we introduce JCoLA (Japanese Corpus of Linguistic Acceptability), which consists of 10, 020 sentences annotated with binary acceptability judgments.
• Can BERT eat RuCoLA? Topological Data Analysis to Explain
  4 Apr 2023 2 repositories listed
  Our results contribute to understanding the behavior of monolingual LMs in the acceptability classification task, provide insights into the functional roles of attention heads, and highlight the advantages of TDA-based…
• ScandEval: A Benchmark for Scandinavian Natural Language Processing
  3 Apr 2023 2 repositories listed
  This paper introduces a Scandinavian benchmarking platform, ScandEval, which can benchmark any pretrained model on four different tasks in the Scandinavian languages.
• Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
  23 Jun 2021 2 repositories listed Syntology ran 7 of 10 samples · 3 unverified
  In this paper, we propose a new model inductive bias that learns a subword tokenization end-to-end as part of the model.
• A Statistical Framework for Low-bitwidth Training of Deep Neural Networks
  27 Oct 2020 2 repositories listed Syntology ran 1 of 4 samples · 3 unverified · 1 pointer-only (licence)
  We show that the FQT gradient is an unbiased estimator of the QAT gradient, and we discuss the impact of gradient quantization on its variance.
• ERNIE: Enhanced Language Representation with Informative Entities
  17 May 2019 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 1 pointer-only (licence)
  Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plain text, and be fine-tuned to consistently improve the performance of various NLP…
• Neural Network Acceptability Judgments
  31 May 2018 2 repositories listed Syntology ran 0 of 2 samples · 2 unverified
  This paper investigates the ability of artificial neural networks to judge the grammatical acceptability of a sentence, with the goal of testing their linguistic competence.
• Fietje: An open, efficient LLM for Dutch
  19 Dec 2024 1 repository listed
  The model is based on Phi 2, an English-centric model of 2.

Syntology lines on 24 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.