Paraphrase Identification

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

11 leaderboard tables shown for this task, 11 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 11 until expanded.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

18 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 76 papers with code (172 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  11 Oct 2018 534 repositories listed Syntology ran 204 of 659 samples · 455 unverified · 149 pointer-only (licence)
  We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers.
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
  19 Jun 2019 27 repositories listed Syntology ran 10 of 24 samples · 14 unverified · 3 pointer-only (licence)
  With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling.
• data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language
  7 Feb 2022 12 repositories listed Syntology ran 0 of 6 samples · 6 unverified
  While the general idea of self-supervised learning is identical across modalities, the actual algorithms and objectives differ widely because they were developed with a single modality in mind.
• FNet: Mixing Tokens with Fourier Transforms
  9 May 2021 12 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 1 pointer-only (licence)
  At longer input lengths, our FNet model is significantly faster: when compared to the "efficient" Transformers on the Long Range Arena benchmark, FNet matches the accuracy of the most accurate models, while outpacing…
• TinyBERT: Distilling BERT for Natural Language Understanding
  23 Sep 2019 10 repositories listed Syntology ran 0 of 4 samples · 4 unverified · 4 pointer-only (licence)
  To accelerate inference and reduce model size while maintaining accuracy, we first propose a novel Transformer distillation method that is specially designed for knowledge distillation (KD) of the Transformer-based…
• Bilateral Multi-Perspective Matching for Natural Language Sentences
  13 Feb 2017 10 repositories listed Syntology ran 1 of 4 samples · 3 unverified · 1 pointer-only (licence)
  Natural language sentence matching is a fundamental technology for a variety of tasks.
• Scaling Instruction-Finetuned Language Models
  20 Oct 2022 9 repositories listed Syntology ran 8 of 17 samples · 9 unverified · 2 pointer-only (licence)
  We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks…
• ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs
  16 Dec 2015 8 repositories listed
  (ii) We propose three attention schemes that integrate mutual influence between sentences into CNN; thus, the representation of each sentence takes into consideration its counterpart.
• Multi-Task Deep Neural Networks for Natural Language Understanding
  31 Jan 2019 7 repositories listed Syntology ran 5 of 13 samples · 8 unverified · 2 pointer-only (licence)
  In this paper, we present a Multi-Task Deep Neural Network (MT-DNN) for learning representations across multiple natural language understanding (NLU) tasks.
• TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning
  14 Apr 2021 6 repositories listed Syntology ran 0 of 4 samples · 4 unverified
  Learning sentence embeddings often requires a large amount of labeled data.
• SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
  8 Nov 2019 6 repositories listed Syntology ran 6 of 8 samples · 2 unverified · 1 pointer-only (licence)
  However, due to limited data resources from downstream tasks and the extremely large capacity of pre-trained models, aggressive fine-tuning often causes the adapted model to overfit the data of downstream tasks and…
• SpanBERT: Improving Pre-training by Representing and Predicting Spans
  24 Jul 2019 6 repositories listed Syntology ran 3 of 15 samples · 12 unverified · 4 pointer-only (licence)
  We present SpanBERT, a pre-training method that is designed to better represent and predict spans of text.
• RealFormer: Transformer Likes Residual Attention
  21 Dec 2020 5 repositories listed
  Transformer is the backbone of modern NLP models.
• Dice Loss for Data-imbalanced NLP Tasks
  7 Nov 2019 4 repositories listed Syntology ran 0 of 3 samples · 3 unverified
  Many NLP tasks such as tagging and machine reading comprehension are faced with the severe data imbalance issue: negative examples significantly outnumber positive examples, and the huge number of background examples…
• Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
  30 Mar 2018 4 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  In this work, we present a simple, effective multi-task learning framework for sentence representations that combines the inductive biases of diverse training objectives in a single model.
• Entailment as Few-Shot Learner
  29 Apr 2021 3 repositories listed Syntology ran 1 of 3 samples · 2 unverified
  Large pre-trained language models (LMs) have demonstrated remarkable ability as few-shot learners.
• PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification
  30 Aug 2019 3 repositories listed
  Most existing work on adversarial data generation focuses on English.
• Simple and Effective Text Matching with Richer Alignment Features
  1 Aug 2019 3 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  In this paper, we present a fast and strong neural approach for general purpose text matching applications.
• A Deep Relevance Matching Model for Ad-hoc Retrieval
  23 Nov 2017 3 repositories listed
  Specifically, our model employs a joint deep architecture at the query term level for relevance matching.
• Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
  23 Jun 2021 2 repositories listed Syntology ran 7 of 10 samples · 3 unverified
  In this paper, we propose a new model inductive bias that learns a subword tokenization end-to-end as part of the model.
• Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning
  22 Dec 2020 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Although pretrained language models can be fine-tuned to produce state-of-the-art results for a very wide range of language understanding tasks, the dynamics of this process are not well understood, especially in the…
• Pay Attention when Required
  9 Sep 2020 2 repositories listed
  Transformer-based models consist of interleaved feed-forward blocks - that capture content meaning, and relatively more expensive self-attention blocks - that capture context meaning.
• Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language
  17 May 2019 2 repositories listed
  This work shows that transfer learning from a multilingual model to monolingual model results in significant growth of performance on such tasks as reading comprehension, paraphrase detection, and sentiment analysis.
• ERNIE: Enhanced Language Representation with Informative Entities
  17 May 2019 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 1 pointer-only (licence)
  Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plain text, and be fine-tuned to consistently improve the performance of various NLP…
• PAWS: Paraphrase Adversaries from Word Scrambling
  1 Apr 2019 2 repositories listed
  Existing paraphrase identification datasets lack sentence pairs that have high lexical overlap without being paraphrases.
• Natural Language Inference over Interaction Space
  13 Sep 2017 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  Natural Language Inference (NLI) task requires an agent to determine the logical relationship between a natural language premise and a natural language hypothesis.
• Enhancing Paraphrase Type Generation: The Impact of DPO and RLHF Evaluated with Human-Ranked Data
  28 May 2025 1 repository listed
  The PTD model surpasses automated metrics and provides a more reliable framework for evaluating paraphrase quality, advancing paraphrase-type research toward richer, user-aligned language generation and establishing a…
• Enhancing Plagiarism Detection in Marathi with a Weighted Ensemble of TF-IDF and BERT Embeddings for Low-Resource Language Processing
  9 Jan 2025 1 repository listed
  Plagiarism involves using another person's work or concepts without proper attribution, presenting them as original creations.
• Tougher Text, Smarter Models: Raising the Bar for Adversarial Defence Benchmarks
  5 Jan 2025 1 repository listed
  Recent advancements in natural language processing have highlighted the vulnerability of deep learning models to adversarial attacks.
• Application Specific Compression of Deep Learning Models
  9 Sep 2024 1 repository listed
  Our method, Application Specific Compression (ASC), identifies and prunes components of the large Deep Learning model that are redundant specifically for the given target application.

Syntology lines on 19 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.