{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-visual-task-adaptation-benchmark","title":"A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark","arxiv_id":"1910.04867","date":"2019-10-01","proceeding":"arXiv 2020 2","authors":["Xiaohua Zhai","Joan Puigcerver","Alexander Kolesnikov","Pierre Ruyssen","Carlos Riquelme","Mario Lucic","Josip Djolonga","Andre Susano Pinto","Maxim Neumann","Alexey Dosovitskiy","Lucas Beyer","Olivier Bachem","Michael Tschannen","Marcin Michalski","Olivier Bousquet","Sylvain Gelly","Neil Houlsby"],"abstract":"Representation learning promises to unlock deep learning for the long tail of vision tasks without expensive labelled datasets. Yet, the absence of a unified evaluation for general visual representations hinders progress. Popular protocols are often too constrained (linear classification), limited in diversity (ImageNet, CIFAR, Pascal-VOC), or only weakly related to representation quality (ELBO, reconstruction error). We present the Visual Task Adaptation Benchmark (VTAB), which defines good representations as those that adapt to diverse, unseen tasks with few examples. With VTAB, we conduct a large-scale study of many popular publicly-available representation learning algorithms. We carefully control confounders such as architecture and tuning budget. We address questions like: How effective are ImageNet representations beyond standard natural datasets? How do representations trained via generative and discriminative models compare? To what extent can self-supervision replace labels? And, how close are we to general visual representations?","url_abs":"https://arxiv.org/abs/1910.04867v2","url_pdf":"https://arxiv.org/pdf/1910.04867v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-visual-task-adaptation-benchmark","repo_url":"https://github.com/google-research/task_adaptation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"the-visual-task-adaptation-benchmark","repo_url":"https://github.com/facebookresearch/vissl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"S4L-Exemplar-ResNet50-LargeHyperSweep","rank_in_archive_order":7,"of":34,"metrics":{"Top-1 Accuracy":"72.7"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"S4L-Rotation-ResNet50-LargeHyperSweep","rank_in_archive_order":9,"of":34,"metrics":{"Top-1 Accuracy":"71.5"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"ImageNet-ResNet50-LargeHyperSweep","rank_in_archive_order":10,"of":34,"metrics":{"Top-1 Accuracy":"71.2"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"S4L-Rotation-ResNet50","rank_in_archive_order":13,"of":34,"metrics":{"Top-1 Accuracy":"67.5"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"S4L-Exemplar-ResNet50","rank_in_archive_order":14,"of":34,"metrics":{"Top-1 Accuracy":"67.0"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"ImageNet-ResNet50","rank_in_archive_order":16,"of":34,"metrics":{"Top-1 Accuracy":"65.6"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"S4L-10%-Rotation-ResNet50","rank_in_archive_order":17,"of":34,"metrics":{"Top-1 Accuracy":"64.8"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"S4L-10%-Exemplar-ResNet50","rank_in_archive_order":18,"of":34,"metrics":{"Top-1 Accuracy":"63.9"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"ImageNet-10%-ResNet50","rank_in_archive_order":19,"of":34,"metrics":{"Top-1 Accuracy":"61.6"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"SelfSup-Rotation-ResNet50","rank_in_archive_order":20,"of":34,"metrics":{"Top-1 Accuracy":"59.5"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"ResNet50-LargeHyperSweep","rank_in_archive_order":21,"of":34,"metrics":{"Top-1 Accuracy":"59.2"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"BigBiGAN-ResNet50","rank_in_archive_order":22,"of":34,"metrics":{"Top-1 Accuracy":"59.1"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"SelfSup-Exemplar-ResNet50","rank_in_archive_order":24,"of":34,"metrics":{"Top-1 Accuracy":"57.5"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"SelfSup-Jigsaw-ResNet50","rank_in_archive_order":26,"of":34,"metrics":{"Top-1 Accuracy":"51.1"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"SelfSup-RelativePatchLoc-ResNet50","rank_in_archive_order":27,"of":34,"metrics":{"Top-1 Accuracy":"50.8"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"Unconditional-BigGAN-ResNet50","rank_in_archive_order":28,"of":34,"metrics":{"Top-1 Accuracy":"44.0"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"ResNet50","rank_in_archive_order":29,"of":34,"metrics":{"Top-1 Accuracy":"42.1"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"VAE","rank_in_archive_order":30,"of":34,"metrics":{"Top-1 Accuracy":"37.5"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"WAE-MMD","rank_in_archive_order":31,"of":34,"metrics":{"Top-1 Accuracy":"37.3"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"Conditional-BigGAN","rank_in_archive_order":32,"of":34,"metrics":{"Top-1 Accuracy":"35.3"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"WAE-GAN","rank_in_archive_order":33,"of":34,"metrics":{"Top-1 Accuracy":"32.0"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-vtab-1k-1","task":"Image Classification","dataset":"VTAB-1k","model":"WAE-UKL","rank_in_archive_order":34,"of":34,"metrics":{"Top-1 Accuracy":"31.0"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1910.04867","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1910.04867"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/vissl","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/task_adaptation","reach":null}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"731698a07fab5fc6","entry":"get_data_params_from_flags","repo":"google-research/task_adaptation","repo_kind":"official","path":"task_adaptation/adapt_and_eval.py","file_url":"https://github.com/google-research/task_adaptation/blob/HEAD/task_adaptation/adapt_and_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"731698a07fab5fc6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}