Transfer Learning

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

6 leaderboard tables shown for this task, 6 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

18 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

4 subtasks in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 3,502 papers with code (10,307 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
  28 May 2019 144 repositories listed Syntology ran 171 of 302 samples · 131 unverified · 112 pointer-only (licence)
  Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available.
• Universal Language Model Fine-tuning for Text Classification
  18 Jan 2018 66 repositories listed Syntology ran 2 of 5 samples · 3 unverified · 3 pointer-only (licence)
  Inductive transfer learning has greatly impacted computer vision, but existing approaches in NLP still require task-specific modifications and training from scratch.
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
  27 Aug 2019 64 repositories listed Syntology ran 20 of 58 samples · 38 unverified · 11 pointer-only (licence)
  However, it requires that both sentences are fed into the network, which causes a massive computational overhead: Finding the most similar pair in a collection of 10, 000 sentences requires about 50 million inference…
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
  23 Oct 2019 57 repositories listed Syntology ran 2 of 31 samples · 29 unverified
  Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP).
• High Quality Monocular Depth Estimation via Transfer Learning
  31 Dec 2018 45 repositories listed Syntology ran 5 of 23 samples · 18 unverified · 3 pointer-only (licence)
  Accurate depth estimation from images is a fundamental task in many applications including scene understanding and reconstruction.
• DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
  2 Oct 2019 37 repositories listed Syntology ran 19 of 27 samples · 8 unverified
  As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or…
• ResNeSt: Split-Attention Networks
  19 Apr 2020 36 repositories listed Syntology ran 8 of 48 samples · 40 unverified · 23 pointer-only (licence)
  It is well known that featuremap attention and multi-path representation are important for visual recognition.
• Bag of Tricks for Image Classification with Convolutional Neural Networks
  4 Dec 2018 28 repositories listed Syntology ran 4 of 15 samples · 11 unverified · 5 pointer-only (licence)
  Much of the recent progress made in image classification research can be credited to training procedure refinements, such as changes in data augmentations and optimization methods.
• Universal Sentence Encoder
  29 Mar 2018 24 repositories listed Syntology ran 1 of 22 samples · 21 unverified · 1 pointer-only (licence)
  For both variants, we investigate and report the relationship between model complexity, resource consumption, the availability of transfer task training data, and task performance.
• TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
  23 Jan 2019 23 repositories listed Syntology ran 8 of 26 samples · 18 unverified · 7 pointer-only (licence)
  We introduce a new approach to generative data-driven dialogue systems (e.
• Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
  5 May 2017 23 repositories listed Syntology ran 6 of 7 samples · 1 unverified · 7 pointer-only (licence)
  Many modern NLP systems rely on word embeddings, previously trained in an unsupervised manner on large corpora, as base features.
• Unsupervised Domain Adaptation by Backpropagation
  26 Sep 2014 22 repositories listed Syntology ran 34 of 41 samples · 7 unverified · 14 pointer-only (licence)
  Here, we propose a new approach to domain adaptation in deep architectures that can be trained on large amount of labeled data from the source domain and large amount of unlabeled data from the target domain (no labeled…
• Going deeper with Image Transformers
  31 Mar 2021 21 repositories listed Syntology ran 5 of 11 samples · 6 unverified · 2 pointer-only (licence)
  In particular, we investigate the interplay of architecture and optimization of such dedicated transformers.
• Unsupervised Data Augmentation for Consistency Training
  29 Apr 2019 20 repositories listed Syntology ran 15 of 52 samples · 37 unverified · 9 pointer-only (licence)
  In this work, we present a new perspective on how to effectively noise unlabeled examples and argue that the quality of noising, specifically those produced by advanced data augmentation methods, plays a crucial role in…
• Parameter-Efficient Transfer Learning for NLP
  2 Feb 2019 17 repositories listed Syntology ran 8 of 22 samples · 14 unverified · 3 pointer-only (licence)
  On GLUE, we attain within 0.
• DeiT III: Revenge of the ViT
  14 Apr 2022 12 repositories listed
  Our evaluations on Image classification (ImageNet-1k with and without pre-training on ImageNet-21k), transfer learning and semantic segmentation show that our procedure outperforms by a large margin previous fully…
• FNet: Mixing Tokens with Fourier Transforms
  9 May 2021 12 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 1 pointer-only (licence)
  At longer input lengths, our FNet model is significantly faster: when compared to the "efficient" Transformers on the Long Range Arena benchmark, FNet matches the accuracy of the most accurate models, while outpacing…
• Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
  12 Jun 2018 11 repositories listed
  Clone a voice in 5 seconds to generate arbitrary speech in real-time
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
  20 Apr 2018 11 repositories listed Syntology ran 5 of 14 samples · 9 unverified · 1 pointer-only (licence)
  For natural language understanding (NLU) technology to be maximally useful, both practically and as a scientific object of study, it must be general: it must be able to process language in a way that is not exclusively…
• Rethinking Channel Dimensions for Efficient Model Design
  2 Jul 2020 10 repositories listed Syntology ran 1 of 8 samples · 7 unverified
  We then investigate the channel configuration of a model by searching network architectures concerning the channel configuration under the computational cost restriction.
• WinoGrande: An Adversarial Winograd Schema Challenge at Scale
  24 Jul 2019 10 repositories listed
  The key steps of the dataset construction consist of (1) a carefully designed crowdsourcing procedure, followed by (2) systematic bias reduction using a novel AfLite algorithm that generalizes human-detectable word…
• Towards Compact Single Image Super-Resolution via Contrastive Self-distillation
  25 May 2021 9 repositories listed Syntology ran 0 of 2 samples · 2 unverified · 2 pointer-only (licence)
  Convolutional neural networks (CNNs) are highly successful for super-resolution (SR) but often require sophisticated architectures with heavy memory cost and computational overhead, significantly restricts their…
• GoEmotions: A Dataset of Fine-Grained Emotions
  1 May 2020 9 repositories listed Syntology ran 0 of 10 samples · 10 unverified
  Understanding emotion expressed in language has a wide range of applications, from building empathetic chatbots to detecting harmful online behavior.
• Unveiling COVID-19 from Chest X-ray with deep learning: a hurdles race with small data
  11 Apr 2020 9 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  The possibility to use widespread and simple chest X-ray (CXR) imaging for early screening of COVID-19 patients is attracting much interest from both the clinical and the AI community.
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
  9 Oct 2019 9 repositories listed Syntology ran 3 of 3 samples · 0 unverified
  Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks.
• Continual learning with hypernetworks
  3 Jun 2019 9 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Artificial neural networks suffer from catastrophic forgetting when they are sequentially trained on multiple tasks.
• Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
  30 Mar 2016 9 repositories listed Syntology ran 10 of 13 samples · 3 unverified · 7 pointer-only (licence)
  By following the principles of self-supervision, we build a convolutional neural network (CNN) that can be trained to solve Jigsaw puzzles as a pretext task, which requires no manual labeling, and then later repurposed…
• Fashionpedia: Ontology, Segmentation, and an Attribute Localization Dataset
  26 Apr 2020 8 repositories listed
  In this work we explore the task of instance segmentation with attribute localization, which unifies instance segmentation (detect and segment each object instance) and fine-grained visual attribute categorization…
• TAPAS: Weakly Supervised Table Parsing via Pre-training
  5 Apr 2020 8 repositories listed Syntology ran 0 of 14 samples · 14 unverified
  In this paper, we present TAPAS, an approach to question answering over tables without generating logical forms.
• A Simple Baseline for Bayesian Uncertainty in Deep Learning
  7 Feb 2019 8 repositories listed Syntology ran 6 of 17 samples · 11 unverified
  We propose SWA-Gaussian (SWAG), a simple, scalable, and general purpose approach for uncertainty representation and calibration in deep learning.

Syntology lines on 26 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.