{"url":"/task/representation-learning","name":"Representation Learning","slug":"representation-learning","description_markdown":"**Representation Learning**  is a process in machine learning where algorithms extract meaningful patterns from raw data to create representations that are easier to understand and process. These representations can be designed for interpretability, reveal hidden features, or be used for transfer learning. They are valuable across many fundamental machine learning tasks like [image classification](/task/image-classification) and [retrieval](/task/image-retrieval).\r\n\r\nDeep neural networks can be considered representation learning models that typically encode information which is projected into a different subspace. These representations are then usually passed on to a linear classifier to, for instance, train a classifier. \r\n\r\nRepresentation learning can be divided into:\r\n\r\n-  **Supervised representation learning**: learning representations on task A using annotated data and used to solve task B\r\n- **Unsupervised representation learning**: learning representations on a task in an unsupervised way (label-free data). These are then used to address downstream tasks and reducing the need for annotated data when learning news tasks. Powerful models like [GPT](/method/gpt) and [BERT](/method/bert) leverage unsupervised representation learning to tackle language tasks.  \r\n\r\nMore recently, [self-supervised learning (SSL)](/task/self-supervised-learning) is one of the main drivers behind unsupervised representation learning in fields like computer vision and NLP. \r\n\r\nHere are some additional readings to go deeper on the task:\r\n\r\n- [Representation Learning: A Review and New Perspectives](/paper/representation-learning-a-review-and-new) - Bengio et al. (2012)\r\n- [A Few Words on Representation Learning](https://sthalles.github.io/a-few-words-on-representation-learning/) - Thalles Silva\r\n\r\n<span style=\"color:grey; opacity: 0.6\">( Image credit: [Visualizing and Understanding Convolutional Networks](https://arxiv.org/pdf/1311.2901.pdf) )</span>","categories":[{"name":"Computer Vision","url":"/area/computer-vision"},{"name":"Methodology","url":"/area/methodology"},{"name":"Natural Language Processing","url":"/area/natural-language-processing"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"archive_url"},"counts":{"papers_tagged":10580,"papers_with_code":4662,"benchmarks":5,"benchmark_tables_in_archive":5,"benchmark_tables_shown":5,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":9,"subtasks":14,"parent_tasks":0},"benchmarks":[{"leaderboard":"/sota/representation-learning-on-scidocs","slug":"representation-learning-on-scidocs","dataset":"SciDocs","dataset_url":"/dataset/scidocs","rows_in_archive":7,"metrics":["Avg."],"first_row_in_archive_order":{"model":"SciNCL","paper_title":"Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings","paper_url":"/paper/neighborhood-contrastive-learning-for-1","paper_date":"2022-02-14","arxiv_id":"2202.06671","code_links":[{"title":"malteos/scincl","url":"https://github.com/malteos/scincl"}],"syntology":{"n":1,"n_ran":0,"n_unverified":1,"n_pointer_only":0}}},{"leaderboard":"/sota/representation-learning-on-animals-10","slug":"representation-learning-on-animals-10","dataset":"Animals-10","dataset_url":"/dataset/animals-10","rows_in_archive":1,"metrics":["1:1 Accuracy"],"first_row_in_archive_order":{"model":"top_model_weights_with_3d_2","paper_title":"Adjusting for Bias with Procedural Data","paper_url":"/paper/adjusting-for-bias-with-procedural-data","paper_date":"2022-04-03","arxiv_id":"2204.01108","code_links":[{"title":"aiskunks/ai_research","url":"https://github.com/aiskunks/ai_research"}],"syntology":null}},{"leaderboard":"/sota/representation-learning-on-cifar10","slug":"representation-learning-on-cifar10","dataset":"CIFAR10","dataset_url":"/dataset/cifar-10","rows_in_archive":1,"metrics":["Accuracy (%)"],"first_row_in_archive_order":{"model":"Resnet 18","paper_title":"AlignMixup: Improving Representations By Interpolating Aligned Features","paper_url":"/paper/alignmix-improving-representation-by","paper_date":"2021-03-29","arxiv_id":"2103.15375","code_links":[{"title":"Westlake-AI/openmixup","url":"https://github.com/Westlake-AI/openmixup"},{"title":"shashankvkt/alignmixup_cvpr22","url":"https://github.com/shashankvkt/alignmixup_cvpr22"}],"syntology":{"n":8,"n_ran":3,"n_unverified":5,"n_pointer_only":0}}},{"leaderboard":"/sota/representation-learning-on-circle-data","slug":"representation-learning-on-circle-data","dataset":"Circle Data","dataset_url":null,"rows_in_archive":1,"metrics":["Accuracy"],"first_row_in_archive_order":{"model":"Morphological Network","paper_title":"Morphological Network: How Far Can We Go with Morphological Neurons?","paper_url":"/paper/dense-morphological-network-an-universal","paper_date":"2019-01-01","arxiv_id":"1901.00109","code_links":[],"syntology":null}},{"leaderboard":"/sota/representation-learning-on-sports10","slug":"representation-learning-on-sports10","dataset":"Sports10","dataset_url":"/dataset/sports10","rows_in_archive":1,"metrics":["Silhouette Score"],"first_row_in_archive_order":{"model":"Max Margin Contrastive","paper_title":"Contrastive Learning of Generalized Game Representations","paper_url":"/paper/contrastive-learning-of-generalized-game","paper_date":"2021-06-18","arxiv_id":"2106.10060","code_links":[{"title":"ChintanTrivedi/contrastive-game-representations","url":"https://github.com/ChintanTrivedi/contrastive-game-representations"}],"syntology":null}}],"datasets":[{"url":"/dataset/scidocs","name":"SciDocs","full_name":"SciDocs","num_papers_in_archive":57},{"url":"/dataset/omnibenchmark","name":"OmniBenchmark","full_name":"","num_papers_in_archive":29},{"url":"/dataset/extreme-classification","name":"EXTREME CLASSIFICATION","full_name":"Extreme Multi-label Classification","num_papers_in_archive":18},{"url":"/dataset/sports10","name":"Sports10","full_name":"","num_papers_in_archive":5},{"url":"/dataset/animals-10","name":"Animals-10","full_name":"","num_papers_in_archive":3},{"url":"/dataset/synth-pedes","name":"SYNTH-PEDES","full_name":"","num_papers_in_archive":3},{"url":"/dataset/causal-triplet","name":"Causal Triplet","full_name":"","num_papers_in_archive":2},{"url":"/dataset/xview3-sar","name":"xView3-SAR","full_name":"","num_papers_in_archive":2},{"url":"/dataset/tyc-dataset","name":"TYC Dataset","full_name":"The TYC Dataset for Understanding Instance-Level Semantics and Motions of Cells in Microstructures","num_papers_in_archive":1}],"subtasks":[{"url":"/task/disentanglement","name":"Disentanglement"},{"url":"/task/document-embedding","name":"Document Embedding"},{"url":"/task/feature-upsampling","name":"Feature Upsampling"},{"url":"/task/graph-representation-learning","name":"Graph Representation Learning"},{"url":"/task/knowledge-graph-embeddings","name":"Knowledge Graph Embeddings"},{"url":"/task/learning-network-representations","name":"Learning Network Representations"},{"url":"/task/learning-representation-of-multi-view-data","name":"Learning Representation Of Multi-View Data"},{"url":"/task/learning-representation-on-graph","name":"Learning Representation On Graph"},{"url":"/task/learning-semantic-representations","name":"Learning Semantic Representations"},{"url":"/task/learning-word-embeddings","name":"Learning Word Embeddings"},{"url":"/task/multilingual-word-embeddings","name":"Multilingual Word Embeddings"},{"url":"/task/network-embedding","name":"Network Embedding"},{"url":"/task/part-based-representation-learning","name":"Part-based Representation Learning"},{"url":"/task/sentence-embeddings","name":"Sentence Embeddings"}],"parent_tasks":[],"papers":{"order":"repositories listed in the archive (desc), then date (desc); the archive holds no stars","population":"papers tagged with this task that list at least one repository in the archive","shown":30,"of":4662,"tagged_in_all":10580,"items":[{"url":"/paper/unsupervised-representation-learning-with-1","title":"Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks","date":"2015-11-19","arxiv_id":"1511.06434","repositories_listed":258,"syntology":{"n":219,"n_ran":113,"n_unverified":106,"n_pointer_only":111}},{"url":"/paper/neural-discrete-representation-learning","title":"Neural Discrete Representation Learning","date":"2017-11-02","arxiv_id":"1711.00937","repositories_listed":50,"syntology":{"n":80,"n_ran":53,"n_unverified":27,"n_pointer_only":32}},{"url":"/paper/momentum-contrast-for-unsupervised-visual","title":"Momentum Contrast for Unsupervised Visual Representation Learning","date":"2019-11-13","arxiv_id":"1911.05722","repositories_listed":44,"syntology":{"n":42,"n_ran":26,"n_unverified":16,"n_pointer_only":16}},{"url":"/paper/190807919","title":"Deep High-Resolution Representation Learning for Visual Recognition","date":"2019-08-20","arxiv_id":"1908.07919","repositories_listed":42,"syntology":{"n":34,"n_ran":3,"n_unverified":31,"n_pointer_only":16}},{"url":"/paper/high-resolution-representations-for-labeling","title":"High-Resolution Representations for Labeling Pixels and Regions","date":"2019-04-09","arxiv_id":"1904.04514","repositories_listed":39,"syntology":{"n":18,"n_ran":3,"n_unverified":15,"n_pointer_only":5}},{"url":"/paper/deep-high-resolution-representation-learning","title":"Deep High-Resolution Representation Learning for Human Pose Estimation","date":"2019-02-25","arxiv_id":"1902.09212","repositories_listed":39,"syntology":{"n":25,"n_ran":8,"n_unverified":17,"n_pointer_only":0}},{"url":"/paper/infogan-interpretable-representation-learning","title":"InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets","date":"2016-06-12","arxiv_id":"1606.03657","repositories_listed":38,"syntology":{"n":6,"n_ran":5,"n_unverified":1,"n_pointer_only":0}},{"url":"/paper/domain-adversarial-training-of-neural","title":"Domain-Adversarial Training of Neural Networks","date":"2015-05-28","arxiv_id":"1505.07818","repositories_listed":37,"syntology":{"n":52,"n_ran":33,"n_unverified":19,"n_pointer_only":22}},{"url":"/paper/improved-baselines-with-momentum-contrastive","title":"Improved Baselines with Momentum Contrastive Learning","date":"2020-03-09","arxiv_id":"2003.04297","repositories_listed":36,"syntology":{"n":43,"n_ran":8,"n_unverified":35,"n_pointer_only":10}},{"url":"/paper/unsupervised-cross-lingual-representation-1","title":"Unsupervised Cross-lingual Representation Learning at Scale","date":"2019-11-05","arxiv_id":"1911.02116","repositories_listed":35,"syntology":{"n":59,"n_ran":25,"n_unverified":34,"n_pointer_only":52}},{"url":"/paper/bootstrap-your-own-latent-a-new-approach-to","title":"Bootstrap your own latent: A new approach to self-supervised Learning","date":"2020-06-13","arxiv_id":"2006.07733","repositories_listed":31,"syntology":{"n":79,"n_ran":62,"n_unverified":17,"n_pointer_only":46}},{"url":"/paper/representation-learning-with-contrastive","title":"Representation Learning with Contrastive Predictive Coding","date":"2018-07-10","arxiv_id":"1807.03748","repositories_listed":28,"syntology":{"n":45,"n_ran":29,"n_unverified":16,"n_pointer_only":22}},{"url":"/paper/exploring-simple-siamese-representation","title":"Exploring Simple Siamese Representation Learning","date":"2020-11-20","arxiv_id":"2011.10566","repositories_listed":26,"syntology":{"n":39,"n_ran":34,"n_unverified":5,"n_pointer_only":22}},{"url":"/paper/supervised-contrastive-learning","title":"Supervised Contrastive Learning","date":"2020-04-23","arxiv_id":"2004.11362","repositories_listed":26,"syntology":{"n":23,"n_ran":6,"n_unverified":17,"n_pointer_only":0}},{"url":"/paper/unsupervised-representation-learning-by-1","title":"Unsupervised Representation Learning by Predicting Image Rotations","date":"2018-03-21","arxiv_id":"1803.07728","repositories_listed":20,"syntology":{"n":22,"n_ran":14,"n_unverified":8,"n_pointer_only":18}},{"url":"/paper/inductive-representation-learning-on-large","title":"Inductive Representation Learning on Large Graphs","date":"2017-06-07","arxiv_id":"1706.02216","repositories_listed":20,"syntology":{"n":5,"n_ran":3,"n_unverified":2,"n_pointer_only":5}},{"url":"/paper/node2vec-scalable-feature-learning-for","title":"node2vec: Scalable Feature Learning for Networks","date":"2016-07-03","arxiv_id":"1607.00653","repositories_listed":20,"syntology":{"n":25,"n_ran":8,"n_unverified":17,"n_pointer_only":3}},{"url":"/paper/bottom-up-higher-resolution-networks-for","title":"HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation","date":"2019-08-27","arxiv_id":"1908.10357","repositories_listed":19,"syntology":{"n":27,"n_ran":5,"n_unverified":22,"n_pointer_only":0}},{"url":"/paper/tabnet-attentive-interpretable-tabular","title":"TabNet: Attentive Interpretable Tabular Learning","date":"2019-08-20","arxiv_id":"1908.07442","repositories_listed":19,"syntology":{"n":17,"n_ran":1,"n_unverified":16,"n_pointer_only":1}},{"url":"/paper/biobert-a-pre-trained-biomedical-language","title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","date":"2019-01-25","arxiv_id":"1901.08746","repositories_listed":19,"syntology":{"n":25,"n_ran":4,"n_unverified":21,"n_pointer_only":1}},{"url":"/paper/how-powerful-are-graph-neural-networks","title":"How Powerful are Graph Neural Networks?","date":"2018-10-01","arxiv_id":"1810.00826","repositories_listed":19,"syntology":{"n":10,"n_ran":3,"n_unverified":7,"n_pointer_only":5}},{"url":"/paper/learning-to-prompt-for-vision-language-models","title":"Learning to Prompt for Vision-Language Models","date":"2021-09-02","arxiv_id":"2109.01134","repositories_listed":18,"syntology":{"n":12,"n_ran":3,"n_unverified":9,"n_pointer_only":0}},{"url":"/paper/blip-2-bootstrapping-language-image-pre","title":"BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models","date":"2023-01-30","arxiv_id":"2301.12597","repositories_listed":17,"syntology":{"n":8,"n_ran":4,"n_unverified":4,"n_pointer_only":1}},{"url":"/paper/convnext-v2-co-designing-and-scaling-convnets","title":"ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders","date":"2023-01-02","arxiv_id":"2301.00808","repositories_listed":17,"syntology":{"n":14,"n_ran":10,"n_unverified":4,"n_pointer_only":9}},{"url":"/paper/vision-mamba-efficient-visual-representation","title":"Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model","date":"2024-01-17","arxiv_id":"2401.09417","repositories_listed":15,"syntology":{"n":13,"n_ran":2,"n_unverified":11,"n_pointer_only":1}},{"url":"/paper/hierarchical-graph-representation-learning","title":"Hierarchical Graph Representation Learning with Differentiable Pooling","date":"2018-06-22","arxiv_id":"1806.08804","repositories_listed":14,"syntology":{"n":20,"n_ran":1,"n_unverified":19,"n_pointer_only":0}},{"url":"/paper/vmamba-visual-state-space-model","title":"VMamba: Visual State Space Model","date":"2024-01-18","arxiv_id":"2401.10166","repositories_listed":13,"syntology":{"n":46,"n_ran":20,"n_unverified":26,"n_pointer_only":0}},{"url":"/paper/challenges-in-representation-learning-a","title":"Challenges in Representation Learning: A report on three machine learning contests","date":"2013-07-01","arxiv_id":"1307.0414","repositories_listed":12,"syntology":{"n":12,"n_ran":0,"n_unverified":12,"n_pointer_only":0}},{"url":"/paper/tokenlearner-what-can-8-learned-tokens-do-for","title":"TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?","date":"2021-06-21","arxiv_id":"2106.11297","repositories_listed":11,"syntology":{"n":3,"n_ran":3,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/hubert-self-supervised-speech-representation","title":"HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units","date":"2021-06-14","arxiv_id":"2106.07447","repositories_listed":11,"syntology":{"n":9,"n_ran":0,"n_unverified":9,"n_pointer_only":0}}],"syntology_records":30,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"description_links":{"kept":6,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-09-24T18:15:14+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; our fixture could not drive it","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"}},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"}}