</> Code with Papers by Syntology

Browse State-of-the-Art › Representation Learning

Representation Learning

4,662 papers with code · 5 benchmarks · 9 datasets archive 2025-07-28

Computer VisionMethodologyNatural Language Processing

Representation Learning is a process in machine learning where algorithms extract meaningful patterns from raw data to create representations that are easier to understand and process. These representations can be designed for interpretability, reveal hidden features, or be used for transfer learning. They are valuable across many fundamental machine learning tasks like image classification and retrieval.

Deep neural networks can be considered representation learning models that typically encode information which is projected into a different subspace. These representations are then usually passed on to a linear classifier to, for instance, train a classifier.

Representation learning can be divided into:

  • Supervised representation learning: learning representations on task A using annotated data and used to solve task B
  • Unsupervised representation learning: learning representations on a task in an unsupervised way (label-free data). These are then used to address downstream tasks and reducing the need for annotated data when learning news tasks. Powerful models like GPT and BERT leverage unsupervised representation learning to tackle language tasks.

More recently, self-supervised learning (SSL) is one of the main drivers behind unsupervised representation learning in fields like computer vision and NLP.

Here are some additional readings to go deeper on the task:

( Image credit: Visualizing and Understanding Convolutional Networks )

Description from the archive archive 2025-07-28; Papers-with-Code links inside it are rewritten to this site.

Benchmarks archive 2025-07-28

5 leaderboard tables shown for this task, 5 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

DatasetBest model (first row in archive order)PaperCodeSyntologyCompare
SciDocs (7 rows) SciNCL Neighborhood Contrastive Learning for Scientific Document... code Syntology ran 0 of 1 samples · 1 unverified Compare
Animals-10 (1 row) top_model_weights_with_3d_2 Adjusting for Bias with Procedural Data code — Compare
CIFAR10 (1 row) Resnet 18 AlignMixup: Improving Representations By Interpolating Aligned Features code Syntology ran 3 of 8 samples · 5 unverified Compare
Circle Data (1 row) Morphological Network Morphological Network: How Far Can We Go with Morphological Neurons? — — Compare
Sports10 (1 row) Max Margin Contrastive Contrastive Learning of Generalized Game Representations code — Compare

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

9 datasets whose archive record lists this task, ordered by the archive's paper count.

SciDocsOmniBenchmarkEXTREME CLASSIFICATIONSports10Animals-10SYNTH-PEDESCausal TripletxView3-SARTYC Dataset

Subtasks archive 2025-07-28

14 subtasks in the archive's task tree.

DisentanglementDocument EmbeddingFeature UpsamplingGraph Representation LearningKnowledge Graph EmbeddingsLearning Network RepresentationsLearning Representation Of Multi-View DataLearning Representation On GraphLearning Semantic RepresentationsLearning Word EmbeddingsMultilingual Word EmbeddingsNetwork EmbeddingPart-based Representation LearningSentence Embeddings

Most implemented papers archive 2025-07-28

30 shown of 4,662 papers with code (10,580 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

  • Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
    19 Nov 2015 258 repositories listed Syntology ran 113 of 219 samples · 106 unverified · 111 pointer-only (licence)
    In recent years, supervised learning with convolutional networks (CNNs) has seen huge adoption in computer vision applications.
  • Neural Discrete Representation Learning
    2 Nov 2017 50 repositories listed Syntology ran 53 of 80 samples · 27 unverified · 32 pointer-only (licence)
    Learning useful representations without supervision remains a key challenge in machine learning.
  • Momentum Contrast for Unsupervised Visual Representation Learning
    13 Nov 2019 44 repositories listed Syntology ran 26 of 42 samples · 16 unverified · 16 pointer-only (licence)
    This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning.
  • Deep High-Resolution Representation Learning for Visual Recognition
    20 Aug 2019 42 repositories listed Syntology ran 3 of 34 samples · 31 unverified · 16 pointer-only (licence)
    High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection.
  • High-Resolution Representations for Labeling Pixels and Regions
    9 Apr 2019 39 repositories listed Syntology ran 3 of 18 samples · 15 unverified · 5 pointer-only (licence)
    The proposed approach achieves superior results to existing single-model networks on COCO object detection.
  • Deep High-Resolution Representation Learning for Human Pose Estimation
    25 Feb 2019 39 repositories listed Syntology ran 8 of 25 samples · 17 unverified
    We start from a high-resolution subnetwork as the first stage, gradually add high-to-low resolution subnetworks one by one to form more stages, and connect the mutli-resolution subnetworks in parallel.
  • InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
    12 Jun 2016 38 repositories listed Syntology ran 5 of 6 samples · 1 unverified
    This paper describes InfoGAN, an information-theoretic extension to the Generative Adversarial Network that is able to learn disentangled representations in a completely unsupervised manner.
  • Domain-Adversarial Training of Neural Networks
    28 May 2015 37 repositories listed Syntology ran 33 of 52 samples · 19 unverified · 22 pointer-only (licence)
    Our approach is directly inspired by the theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made based on features that cannot discriminate between the…
  • Improved Baselines with Momentum Contrastive Learning
    9 Mar 2020 36 repositories listed Syntology ran 8 of 43 samples · 35 unverified · 10 pointer-only (licence)
    Contrastive unsupervised learning has recently shown encouraging progress, e.
  • Unsupervised Cross-lingual Representation Learning at Scale
    5 Nov 2019 35 repositories listed Syntology ran 25 of 59 samples · 34 unverified · 52 pointer-only (licence)
    We also present a detailed empirical analysis of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and…
  • Bootstrap your own latent: A new approach to self-supervised Learning
    13 Jun 2020 31 repositories listed Syntology ran 62 of 79 samples · 17 unverified · 46 pointer-only (licence)
    From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view.
  • Representation Learning with Contrastive Predictive Coding
    10 Jul 2018 28 repositories listed Syntology ran 29 of 45 samples · 16 unverified · 22 pointer-only (licence)
    The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models.
  • Exploring Simple Siamese Representation Learning
    20 Nov 2020 26 repositories listed Syntology ran 34 of 39 samples · 5 unverified · 22 pointer-only (licence)
    Our experiments show that collapsing solutions do exist for the loss and structure, but a stop-gradient operation plays an essential role in preventing collapsing.
  • Supervised Contrastive Learning
    23 Apr 2020 26 repositories listed Syntology ran 6 of 23 samples · 17 unverified
    Contrastive learning applied to self-supervised representation learning has seen a resurgence in recent years, leading to state of the art performance in the unsupervised training of deep image models.
  • Unsupervised Representation Learning by Predicting Image Rotations
    21 Mar 2018 20 repositories listed Syntology ran 14 of 22 samples · 8 unverified · 18 pointer-only (licence)
    However, in order to successfully learn those features, they usually require massive amounts of manually labeled data, which is both expensive and impractical to scale.
  • Inductive Representation Learning on Large Graphs
    7 Jun 2017 20 repositories listed Syntology ran 3 of 5 samples · 2 unverified · 5 pointer-only (licence)
    Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions.
  • node2vec: Scalable Feature Learning for Networks
    3 Jul 2016 20 repositories listed Syntology ran 8 of 25 samples · 17 unverified · 3 pointer-only (licence)
    Taken together, our work represents a new way for efficiently learning state-of-the-art task-independent representations in complex networks.
  • HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation
    27 Aug 2019 19 repositories listed Syntology ran 5 of 27 samples · 22 unverified
    HigherHRNet even surpasses all top-down methods on CrowdPose test (67.
  • TabNet: Attentive Interpretable Tabular Learning
    20 Aug 2019 19 repositories listed Syntology ran 1 of 17 samples · 16 unverified · 1 pointer-only (licence)
    We propose a novel high-performance and interpretable canonical deep tabular data learning architecture, TabNet.
  • BioBERT: a pre-trained biomedical language representation model for biomedical text mining
    25 Jan 2019 19 repositories listed Syntology ran 4 of 25 samples · 21 unverified · 1 pointer-only (licence)
    Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows.
  • How Powerful are Graph Neural Networks?
    1 Oct 2018 19 repositories listed Syntology ran 3 of 10 samples · 7 unverified · 5 pointer-only (licence)
    Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures.
  • Learning to Prompt for Vision-Language Models
    2 Sep 2021 18 repositories listed Syntology ran 3 of 12 samples · 9 unverified
    Large pre-trained vision-language models like CLIP have shown great potential in learning representations that are transferable across a wide range of downstream tasks.
  • BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
    30 Jan 2023 17 repositories listed Syntology ran 4 of 8 samples · 4 unverified · 1 pointer-only (licence)
    The cost of vision-and-language pre-training has become increasingly prohibitive due to end-to-end training of large-scale models.
  • ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders
    2 Jan 2023 17 repositories listed Syntology ran 10 of 14 samples · 4 unverified · 9 pointer-only (licence)
    This co-design of self-supervised learning techniques and architectural improvement results in a new model family called ConvNeXt V2, which significantly improves the performance of pure ConvNets on various recognition…
  • Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
    17 Jan 2024 15 repositories listed Syntology ran 2 of 13 samples · 11 unverified · 1 pointer-only (licence)
    The results demonstrate that Vim is capable of overcoming the computation & memory constraints on performing Transformer-style understanding for high-resolution images and it has great potential to be the…
  • Hierarchical Graph Representation Learning with Differentiable Pooling
    22 Jun 2018 14 repositories listed Syntology ran 1 of 20 samples · 19 unverified
    Recently, graph neural networks (GNNs) have revolutionized the field of graph representation learning through effectively learned node embeddings, and achieved state-of-the-art results in tasks such as node…
  • VMamba: Visual State Space Model
    18 Jan 2024 13 repositories listed Syntology ran 20 of 46 samples · 26 unverified
    At the core of VMamba is a stack of Visual State-Space (VSS) blocks with the 2D Selective Scan (SS2D) module.
  • Challenges in Representation Learning: A report on three machine learning contests
    1 Jul 2013 12 repositories listed Syntology ran 0 of 12 samples · 12 unverified
    The ICML 2013 Workshop on Challenges in Representation Learning focused on three challenges: the black box learning challenge, the facial expression recognition challenge, and the multimodal learning challenge.
  • TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?
    21 Jun 2021 11 repositories listed Syntology ran 3 of 3 samples · 0 unverified
    In this paper, we introduce a novel visual representation learning which relies on a handful of adaptively learned tokens, and which is applicable to both image and video understanding tasks.
  • HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
    14 Jun 2021 11 repositories listed Syntology ran 0 of 9 samples · 9 unverified
    Self-supervised approaches for speech representation learning are challenged by three unique problems: (1) there are multiple sound units in each input utterance, (2) there is no lexicon of input sound units during the…

Syntology lines on 30 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections