{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/contrasting-contrastive-self-supervised","title":"Contrasting Contrastive Self-Supervised Representation Learning Pipelines","arxiv_id":"2103.14005","date":"2021-03-25","proceeding":"ICCV 2021 10","authors":["Klemen Kotar","Gabriel Ilharco","Ludwig Schmidt","Kiana Ehsani","Roozbeh Mottaghi"],"abstract":"In the past few years, we have witnessed remarkable breakthroughs in self-supervised representation learning. Despite the success and adoption of representations learned through this paradigm, much is yet to be understood about how different training methods and datasets influence performance on downstream tasks. In this paper, we analyze contrastive approaches as one of the most successful and popular variants of self-supervised representation learning. We perform this analysis from the perspective of the training algorithms, pre-training datasets and end tasks. We examine over 700 training experiments including 30 encoders, 4 pre-training datasets and 20 diverse downstream tasks. Our experiments address various questions regarding the performance of self-supervised models compared to their supervised counterparts, current benchmarks used for evaluation, and the effect of the pre-training data on end task performance. Our Visual Representation Benchmark (ViRB) is available at: https://github.com/allenai/virb.","url_abs":"https://arxiv.org/abs/2103.14005v2","url_pdf":"https://arxiv.org/pdf/2103.14005v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"contrasting-contrastive-self-supervised","repo_url":"https://github.com/allenai/ViRB","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.14005","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.14005"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/allenai/ViRB","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0afffc9c2ecdc153","entry":"get_dataset_class","repo":"allenai/ViRB","repo_kind":"official","path":"ViRBRunner.py","file_url":"https://github.com/allenai/ViRB/blob/HEAD/ViRBRunner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0afffc9c2ecdc153"}},{"code_sha256_prefix":"113beeed61aeb320","entry":"get_optimizer","repo":"allenai/ViRB","repo_kind":"official","path":"ViRBRunner.py","file_url":"https://github.com/allenai/ViRB/blob/HEAD/ViRBRunner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"113beeed61aeb320"}},{"code_sha256_prefix":"cfe25dd353c4c16c","entry":"get_task_head","repo":"allenai/ViRB","repo_kind":"official","path":"ViRBRunner.py","file_url":"https://github.com/allenai/ViRB/blob/HEAD/ViRBRunner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cfe25dd353c4c16c"}},{"code_sha256_prefix":"bc98c103b01322f1","entry":"upshuffle","repo":"allenai/ViRB","repo_kind":"official","path":"models/PixelWisePredictionHead.py","file_url":"https://github.com/allenai/ViRB/blob/HEAD/models/PixelWisePredictionHead.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bc98c103b01322f1"}},{"code_sha256_prefix":"7e6a3a3d8c75f1d9","entry":"upshufflenorelu","repo":"allenai/ViRB","repo_kind":"official","path":"models/PixelWisePredictionHead.py","file_url":"https://github.com/allenai/ViRB/blob/HEAD/models/PixelWisePredictionHead.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7e6a3a3d8c75f1d9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}