{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/benchmarking-representation-learning-for","title":"Benchmarking Representation Learning for Natural World Image Collections","arxiv_id":"2103.16483","date":"2021-03-30","proceeding":"CVPR 2021 1","authors":["Grant van Horn","Elijah Cole","Sara Beery","Kimberly Wilber","Serge Belongie","Oisin Mac Aodha"],"abstract":"Recent progress in self-supervised learning has resulted in models that are capable of extracting rich representations from image collections without requiring any explicit label supervision. However, to date the vast majority of these approaches have restricted themselves to training on standard benchmark datasets such as ImageNet. We argue that fine-grained visual categorization problems, such as plant and animal species classification, provide an informative testbed for self-supervised learning. In order to facilitate progress in this area we present two new natural world visual classification datasets, iNat2021 and NeWT. The former consists of 2.7M images from 10k different species uploaded by users of the citizen science application iNaturalist. We designed the latter, NeWT, in collaboration with domain experts with the aim of benchmarking the performance of representation learning algorithms on a suite of challenging natural world binary classification tasks that go beyond standard species classification. These two new datasets allow us to explore questions related to large-scale representation and transfer learning in the context of fine-grained categories. We provide a comprehensive analysis of feature extractors trained with and without supervision on ImageNet and iNat2021, shedding light on the strengths and weaknesses of different learned features across a diverse set of tasks. We find that features produced by standard supervised methods still outperform those produced by self-supervised approaches such as SimCLR. However, improved self-supervised learning methods are constantly being released and the iNat2021 and NeWT datasets are a valuable resource for tracking their progress.","url_abs":"https://arxiv.org/abs/2103.16483v2","url_pdf":"https://arxiv.org/pdf/2103.16483v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"benchmarking-representation-learning-for","repo_url":"https://github.com/visipedia/newt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"binary-classification","task_name":"Binary Classification"},{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"fine-grained-visual-categorization","task_name":"Fine-Grained Visual Categorization"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"colorjitter","method_name":"ColorJitter"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"feedforward-network","method_name":"Feedforward Network"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"nt-xent","method_name":"NT-Xent"},{"method_slug":"random-gaussian-blur","method_name":"Random Gaussian Blur"},{"method_slug":"random-resized-crop","method_name":"Random Resized Crop"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"simclr","method_name":"SimCLR"}],"datasets_introduced":[{"slug":"inat2021","name":"iNat2021","full_name":"iNaturalist 2021"}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.16483","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.16483"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/visipedia/newt","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":12},"by_repo_kind":{"official":{"samples":12,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cbe50e0989ae1e3f","entry":"conv_block","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/tf_resnet.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/tf_resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cbe50e0989ae1e3f"}},{"code_sha256_prefix":"47c40e00f535da70","entry":"evalute_features","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/evaluate_linear_models.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/evaluate_linear_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"47c40e00f535da70"}},{"code_sha256_prefix":"43006b1579cc0b34","entry":"fixed_padding","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/tf_resnet.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/tf_resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43006b1579cc0b34"}},{"code_sha256_prefix":"2c7bf74214627a61","entry":"identity_block","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/tf_resnet.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/tf_resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c7bf74214627a61"}},{"code_sha256_prefix":"d9c3f2f2c048d6a2","entry":"linearsvc","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/linear_evaluation.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/linear_evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9c3f2f2c048d6a2"}},{"code_sha256_prefix":"26d13a1ab4665887","entry":"load_cub","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/dataset_utils.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/dataset_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26d13a1ab4665887"}},{"code_sha256_prefix":"a4f41438107ab2a7","entry":"load_feature_extractor","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/pt_resnet_feature_extractor.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/pt_resnet_feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a4f41438107ab2a7"}},{"code_sha256_prefix":"11489c1ebee3c555","entry":"load_newt_task","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/dataset_utils.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/dataset_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"11489c1ebee3c555"}},{"code_sha256_prefix":"bfe9138310b695d3","entry":"load_oxford_flowers","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/dataset_utils.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/dataset_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bfe9138310b695d3"}},{"code_sha256_prefix":"65dab949aa3e1b87","entry":"loader","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/pt_resnet_feature_extractor.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/pt_resnet_feature_extractor.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65dab949aa3e1b87"}},{"code_sha256_prefix":"7bde24b55ef593b2","entry":"logreg","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/linear_evaluation.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/linear_evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7bde24b55ef593b2"}},{"code_sha256_prefix":"cb90fce2472de4f8","entry":"sgd","repo":"visipedia/newt","repo_kind":"official","path":"benchmark/linear_evaluation.py","file_url":"https://github.com/visipedia/newt/blob/HEAD/benchmark/linear_evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb90fce2472de4f8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}