Browse State-of-the-Art › Zero-Shot Image Classification
Zero-Shot Image Classification
64 papers with code · 3 benchmarks · 6 datasets archive 2025-07-28
Zero-shot image classification is a technique in computer vision where a model can classify images into categories that were not present during training. This is achieved by leveraging semantic information about the categories, such as textual descriptions or relationships between classes.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
3 leaderboard tables shown for this task, 3 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
6 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
1 subtask in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 111 · with a code link: 64 · where Syntology ran a sample: 25 (20 with a run with no instrument failure, 5 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (25 of 111 tagged: 20 with a run with no instrument failure, 5 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 111, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
19 Apr 2022 9 repositories listed Syntology official (archive's flag): 5 ran · 12 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 4 where Syntology's instrument failed) · 8 unverified (of 20 harvested samples) · 2 pointer-only (licence)In general, these language-augmented visual models demonstrate strong transferability to a variety of datasets and tasks.
-
14 Dec 2022 5 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)To address these limitations, we investigate scaling laws for contrastive language-image pre-training (CLIP) with the public LAION dataset and the open-source OpenCLIP repository.
-
15 Nov 2021 5 repositories listedThis paper presents contrastive-tuning, a simple method employing contrastive training to align image and text models while still taking advantage of their pre-training.
-
11 Feb 2021 5 repositories listed Syntology 8 ran (of which 6 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 10 harvested samples) · 9 pointer-only (licence)In this paper, we leverage a noisy dataset of over one billion image alt-text pairs, obtained without expensive filtering or post-processing steps in the Conceptual Captions dataset.
-
28 Apr 2021 4 repositories listedOn COCO, ViLD outperforms the previous state-of-the-art by 4.
-
7 Sep 2022 3 repositories listedUnlike traditional classification models, open-vocabulary models classify among any arbitrary set of categories specified with natural language during inference.
-
15 Jul 2024 2 repositories listedThis effectively inverts the CLIP text encoder and allows textual object labels from essentially the entire English language to be generated directly from image-derived embedding vectors, without requiring any a priori…
-
4 Apr 2024 2 repositories listedWe propose a novel architecture and method of explainable classification with Concept Bottleneck Models (CBMs).
-
15 Jun 2023 2 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)However, the compositional reasoning abilities of existing VLMs remains subpar.
-
12 Nov 2022 2 repositories listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 1 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples) · 2 pointer-only (licence)In this work, we present a conceptually simple and effective method to train a strong bilingual/multilingual multimodal representation model.
-
4 Jul 2022 2 repositories listedSpecifically, we (1) developed a cross-modal semantic grounding network to investigate the model's capability of disentangling semantic attributes from the images; (2) applied an attribute-level contrastive learning…
-
29 Dec 2021 2 repositories listedHowever, semantic segmentation and the CLIP model perform on different visual granularity, that semantic segmentation processes on pixels while CLIP performs on images.
-
25 Mar 2025 1 repository listedWe propose a novel vision-language foundation model, LRSCLIP, and a multimodal dataset, LRS2M.
-
6 Feb 2025 1 repository listedIn this paper, we show that the common practice of individually exploiting the text or image encoders of these powerful multi-modal models is highly suboptimal for intra-modal tasks like image-to-image retrieval.
-
20 Jan 2025 1 repository listedTo tackle these challenges, we introduce the Knowledge Proxy Learning (KPL) to mine knowledge from CLIP.
-
8 Dec 2024 1 repository listed Syntology official (archive's flag): 4 ran · 7 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 6 unverified (of 13 harvested samples) · 5 pointer-only (licence)Vision-language models (VLMs), such as CLIP and SigLIP, have found remarkable success in classification, retrieval, and generative tasks.
-
1 Nov 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)TaxaBind is a multimodal embedding space across six modalities: ground-level images of species, geographic location, satellite image, text, audio, and environmental features, useful for solving ecological problems.
-
30 Oct 2024 1 repository listedThis work proposes a novel vision-and-language model for the remote sensing domain, exploring the fine-tuning of a multilingual CLIP model and testing the use of a self-supervised method based on aligning local and…
-
22 Oct 2024 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample) · 1 pointer-only (licence)This paper focuses on creating synthetic data to improve the quality of image captions.
-
20 Oct 2024 1 repository listedOur findings reveal that: i) there is little difference between OVD and COD for object classes with low text-describability under equal conditions in OD pretraining; and ii) although OVD can learn from more diverse data…
-
16 Oct 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)Contrastive Language-Image Pretraining (CLIP) performs zero-shot image classification by mapping images and textual class representation into a shared embedding space, then retrieving the class closest to the image.
-
28 Sep 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)In recent years, Contrastive Language-Image Pre-training (CLIP) has become a cornerstone in multimodal intelligence.
-
16 Aug 2024 1 repository listedFinally, it addresses visual-textual misalignment by aligning textual prototypes with image prototypes to further improve the adaptation performance.
-
13 Aug 2024 1 repository listedHowever, it is unclear if these models are robust to distribution shifts, and how their performance and generalization capabilities vary under changes in data distribution.
-
28 Jun 2024 1 repository listedVision Language Models (VLMs) like CLIP have attracted substantial attention in pathology, serving as backbones for applications such as zero-shot image classification and Whole Slide Image (WSI) analysis.
-
25 Jun 2024 1 repository listed Syntology official (archive's flag): 16 ran · 16 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 3 violated, 6 with no contract checked; 7 where Syntology's instrument failed) · 6 unverified (of 22 harvested samples) · 22 pointer-only (licence)We design AlignCLIP, in order to answer these questions and through extensive experiments, we show that AlignCLIP achieves noticeable enhancements in the cross-modal alignment of the embeddings, and thereby, reduces the…
-
19 Jun 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 3 pointer-only (licence)In response, we present Weight Average Test-Time Adaptation (WATT) of CLIP, a pioneering approach facilitating full test-time adaptation (TTA) of this VLM.
-
5 Jun 2024 1 repository listedDecoding images from non-invasive electroencephalographic (EEG) signals has been a grand challenge in understanding how the human brain process visual information in real-world scenarios.
-
24 May 2024 1 repository listedUsing multimodal LLMs, we generate comprehensive textual representations from input images.
-
13 May 2024 1 repository listedAs training datasets become increasingly drawn from unstructured, uncontrolled environments such as the web, researchers and industry practitioners have increasingly relied upon data filtering techniques to "filter out…
-
24 Apr 2024 1 repository listed
-
29 Mar 2024 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
14 Mar 2024 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
5 Mar 2024 1 repository listed
-
5 Feb 2024 1 repository listed
-
2 Feb 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
27 Nov 2023 1 repository listed
-
2 Oct 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 6 pointer-only (licence)
-
2 Aug 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 8 unverified (of 12 harvested samples) · 5 pointer-only (licence)
-
6 Jul 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 3 pointer-only (licence)
-
19 Jun 2023 1 repository listed
-
14 Jun 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples)
-
27 May 2023 1 repository listed
-
11 Apr 2023 1 repository listed
-
27 Mar 2023 1 repository listed
-
3 Mar 2023 1 repository listed
-
6 Feb 2023 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 1 pointer-only (licence)
-
30 Jan 2023 1 repository listed
-
LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Sparse Retrieval1 Jan 2023 1 repository listed
-
15 Dec 2022 1 repository listed
-
2 Nov 2022 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 3 pointer-only (licence)
-
20 Oct 2022 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
14 Sep 2022 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 1 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples)
-
17 Jul 2022 1 repository listed
-
8 Jun 2022 1 repository listed
-
7 Jun 2022 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 2 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified (of 11 harvested samples) · 4 pointer-only (licence)
-
8 May 2022 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples)
-
25 Apr 2022 1 repository listed
-
2 Mar 2022 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 4 harvested samples) · 4 pointer-only (licence)
-
9 Nov 2021 1 repository listed
-
29 Jun 2021 1 repository listed
-
27 Jan 2021 1 repository listed
-
CIBR: Cross-modal Information Bottleneck Regularization for Robust CLIP Generalization31 Mar 2025 0 repositories listed
-
Beyond the Visible: Multispectral Vision-Language Learning for Earth Observation20 Mar 2025 0 repositories listed
-
Bayesian Test-Time Adaptation for Vision-Language Models12 Mar 2025 0 repositories listed
-
MADS: Multi-Attribute Document Supervision for Zero-Shot Image Classification10 Mar 2025 0 repositories listed
-
MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations2 Mar 2025 0 repositories listed
-
Retaining Knowledge and Enhancing Long-Text Representations in CLIP through Dual-Teacher Distillation1 Jan 2025 0 repositories listed
-
CLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance5 Dec 2024 0 repositories listed
-
TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives4 Nov 2024 0 repositories listed
-
Retrieval-enriched zero-shot image classification in low-resource domains1 Nov 2024 0 repositories listed
-
CSA: Data-efficient Mapping of Unimodal Features to Multimodal Features10 Oct 2024 0 repositories listed
-
LoGra-Med: Long Context Multi-Graph Alignment for Medical Vision-Language Model3 Oct 2024 0 repositories listed
-
CoAPT: Context Attribute words for Prompt Tuning18 Jul 2024 0 repositories listed
-
Semantic Compositions Enhance Vision-Language Contrastive Learning1 Jul 2024 0 repositories listed
-
BaFTA: Backprop-Free Test-Time Adaptation For Zero-Shot Vision-Language Models17 Jun 2024 0 repositories listed
-
Multimodal Adversarial Defense for Vision-Language Models by Leveraging One-To-Many Relationships29 May 2024 0 repositories listed
-
It's Not a Modality Gap: Characterizing and Addressing the Contrastive Gap28 May 2024 0 repositories listed
-
Multi-method Integration with Confidence-based Weighting for Zero-shot Image Classification3 May 2024 0 repositories listed
-
A Progressive Framework of Vision-language Knowledge Distillation and Alignment for Multilingual Scene17 Apr 2024 0 repositories listed
-
Bridge the Modality and Capability Gaps in Vision-Language Model Selection20 Mar 2024 0 repositories listed
-
Exploring Low-Resource Medical Image Classification with Weakly Supervised Prompt Learning6 Feb 2024 0 repositories listed
-
CLAMP: Contrastive LAnguage Model Prompt-tuning4 Dec 2023 0 repositories listed
-
LightCLIP: Learning Multi-Level Interaction for Lightweight Vision-Language Models1 Dec 2023 0 repositories listed
-
Efficient Model-Agnostic Multi-Group Equivariant Networks14 Oct 2023 0 repositories listed
-
Noise-Tolerant Few-Shot Unsupervised Adapter for Vision-Language Models26 Sep 2023 0 repositories listed
-
GrowCLIP: Data-aware Automatic Model Growing for Large-scale Contrastive Language-Image Pre-training22 Aug 2023 0 repositories listed
-
Semantically-Prompted Language Models Improve Visual Descriptions5 Jun 2023 0 repositories listed
-
Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparities26 Jan 2023 0 repositories listed
-
RA-CLIP: Retrieval Augmented Contrastive Language-Image Pre-Training1 Jan 2023 0 repositories listed
-
DiRaC-I: Identifying Diverse and Rare Training Classes for Zero-Shot Learning31 Dec 2022 0 repositories listed
-
When are Lemons Purple? The Concept Association Bias of Vision-Language Models22 Dec 2022 0 repositories listed
-
I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classification5 Dec 2022 0 repositories listed
-
Generative Negative Text Replay for Continual Vision-Language Pretraining31 Oct 2022 0 repositories listed
-
Text2Model: Text-based Model Induction for Zero-shot Image Classification27 Oct 2022 0 repositories listed
-
Efficient Multilingual Multi-modal Pre-training through Triple Contrastive Loss1 Oct 2022 0 repositories listed
-
I2DFormer: Learning Image to Document Attention for Zero-Shot Image Classification21 Sep 2022 0 repositories listed
-
PyramidCLIP: Hierarchical Feature Alignment for Vision-language Model Pretraining29 Apr 2022 0 repositories listed
Syntology lines on 28 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections