Datasets › VTAB

VTAB (Visual Task Adaptation Benchmark)

Introduced by Xiaohua Zhai et al. in The Visual Task Adaptation Benchmark25 Sep 2019 archive 2025-07-28

The Visual Task Adaptation Benchmark (VTAB) is a benchmark designed to evaluate general visual representations². It consists of a diverse and challenging suite of tasks². The benchmark defines a good general visual representation as one that yields good performance on unseen tasks, when trained on limited task-specific data².

The VTAB benchmark contains the following 19 tasks that are derived from public datasets¹: - Caltech101 - CIFAR-100 - CLEVR distance prediction - CLEVR counting - Diabetic Rethinopathy - Dmlab Frames - dSprites orientation prediction - dSprites location prediction - Describable Textures Dataset (DTD) - EuroSAT - KITTI distance prediction - 102 Category Flower Dataset - Oxford IIIT Pet dataset - PatchCamelyon - Resisc45 - Small NORB azimuth prediction - Small NORB elevation prediction - SUN397 - SVHN

The given model is independently fine-tuned for solving each of the above tasks¹. Average accuracy across all tasks is used to measure the model's performance¹. Detailed description of all tasks, evaluation protocol, and other details can be found in the VTAB paper¹.

(1) Visual Task Adaptation Benchmark. https://google-research.github.io/task_adaptation/. (2) GitHub - google-research/task_adaptation. https://github.com/google-research/task_adaptation. (3) GitHub - KMnP/vpt: ️ Visual Prompt Tuning [ECCV 2022] https://arxiv .... https://github.com/KMnP/vpt.

Benchmarks archive 2025-07-28

All 4 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

First row (archive order)PaperCode
Image Classification VTAB-1k ALIGN (50 hypers/task) Top-1 Accuracy 79.99 Scaling Up Visual and Vision-Language Representation... facebookresearch/metaclip +4 34 Compare
Visual Prompt Tuning VTAB-1k(Natural<7>) SPT-Deep(ViT-B/16_MoCo_v3_pretrained_ImageNet-1K) Mean Accuracy 76.20 Revisiting the Power of Prompt for Visual Tuning — 10 Compare
Visual Prompt Tuning VTAB-1k(Specialized<4>) SPT-Deep(ViT-B/16_MoCo_v3_pretrained_ImageNet-1K) Mean Accuracy 84.95 Revisiting the Power of Prompt for Visual Tuning — 10 Compare
Visual Prompt Tuning VTAB-1k(Structured<8>) SPT-Deep(ViT-B/16_MAE_pretrained_ImageNet-1K) Mean Accuracy 59.23 Revisiting the Power of Prompt for Visual Tuning — 10 Compare

Papers archive 2025-07-28

papers with a benchmark row: 12 · with a code link: 7 · where Syntology ran a sample: 6 (5 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (6 of 12 with a benchmark row: 5 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 12 of 12 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 202. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

PaperCodeResultsDateSamples run Syntology
Revisiting the Power of Prompt for Visual Tuning 0 12 4 Feb 2024 not harvested
Improving Visual Prompt Tuning for Self-supervised Vision Transformers 1 6 8 Jun 2023 official (archive's flag): 4 ran · 4 ran (of which 3 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified
PVP: Pre-trained Visual Parameter-Efficient Tuning 0 1 26 Apr 2023 not harvested
Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification 1 2 20 Jun 2022 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified
Visual Prompt Tuning 6 12 23 Mar 2022 official: no sample here; runs from other or unrecorded repositories · 18 ran (of which 12 constructed an object rather than computing a result; 16 with no instrument failure: 1 honoured, 1 violated, 14 with no contract checked; 2 where Syntology's instrument failed) · 9 unverified (15 pointer-only for licence)
Scaling Vision Transformers 1 1 8 Jun 2021 not harvested
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision 5 1 11 Feb 2021 8 ran (of which 6 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (9 pointer-only for licence)
Deep Ensembles for Low-Data Transfer Learning 0 1 14 Oct 2020 not harvested
Scalable Transfer Learning with Expert Models 0 1 28 Sep 2020 not harvested
Big Transfer (BiT): General Visual Representation Learning 9 4 24 Dec 2019 community repositories only · 9 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 1 violated, 6 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified
Self-Supervised Learning of Video-Induced Visual Invariances 0 1 5 Dec 2019 not harvested
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark 2 22 1 Oct 2019 official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified

Dataset loaders archive 2025-07-28

No loader listed in the archive.

Tasks archive 2025-07-28

License archive 2025-07-28

No licence recorded in the archive. Absence here is not a statement about the dataset's terms.

Modalities archive 2025-07-28

No modality tagged.

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

  • VTAB-1k
  • VTAB-1k(Natural<7>)
  • VTAB-1k(Specialized<4>)
  • VTAB-1k(Structured<8>)
  • VTAB

5 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections