Zero-Shot Image Classification

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

3 leaderboard tables shown for this task, 3 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

6 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 64 papers with code (111 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models
  19 Apr 2022 9 repositories listed Syntology ran 4 of 20 samples · 16 unverified
  In general, these language-augmented visual models demonstrate strong transferability to a variety of datasets and tasks.
• Reproducible scaling laws for contrastive language-image learning
  14 Dec 2022 5 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  To address these limitations, we investigate scaling laws for contrastive language-image pre-training (CLIP) with the public LAION dataset and the open-source OpenCLIP repository.
• LiT: Zero-Shot Transfer with Locked-image text Tuning
  15 Nov 2021 5 repositories listed
  This paper presents contrastive-tuning, a simple method employing contrastive training to align image and text models while still taking advantage of their pre-training.
• Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
  11 Feb 2021 5 repositories listed Syntology ran 8 of 10 samples · 2 unverified · 9 pointer-only (licence)
  In this paper, we leverage a noisy dataset of over one billion image alt-text pairs, obtained without expensive filtering or post-processing steps in the Conceptual Captions dataset.
• Open-vocabulary Object Detection via Vision and Language Knowledge Distillation
  28 Apr 2021 4 repositories listed
  On COCO, ViLD outperforms the previous state-of-the-art by 4.
• What does a platypus look like? Generating customized prompts for zero-shot image classification
  7 Sep 2022 3 repositories listed
  Unlike traditional classification models, open-vocabulary models classify among any arbitrary set of categories specified with natural language during inference.
• Unconstrained Open Vocabulary Image Classification: Zero-Shot Transfer from Text to Image via CLIP Inversion
  15 Jul 2024 2 repositories listed
  This effectively inverts the CLIP text encoder and allows textual object labels from essentially the entire English language to be generated directly from image-derived embedding vectors, without requiring any a priori…
• Sparse Concept Bottleneck Models: Gumbel Tricks in Contrastive Learning
  4 Apr 2024 2 repositories listed
  We propose a novel architecture and method of explainable classification with Concept Bottleneck Models (CBMs).
• Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Compositional Understanding
  15 Jun 2023 2 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  However, the compositional reasoning abilities of existing VLMs remains subpar.
• AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities
  12 Nov 2022 2 repositories listed Syntology ran 2 of 11 samples · 9 unverified
  In this work, we present a conceptually simple and effective method to train a strong bilingual/multilingual multimodal representation model.
• DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning
  4 Jul 2022 2 repositories listed
  Specifically, we (1) developed a cross-modal semantic grounding network to investigate the model's capability of disentangling semantic attributes from the images; (2) applied an attribute-level contrastive learning…
• A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model
  29 Dec 2021 2 repositories listed
  However, semantic segmentation and the CLIP model perform on different visual granularity, that semantic segmentation processes on pixels while CLIP performs on images.
• LRSCLIP: A Vision-Language Foundation Model for Aligning Remote Sensing Image with Longer Text
  25 Mar 2025 1 repository listed
  We propose a novel vision-language foundation model, LRSCLIP, and a multimodal dataset, LRS2M.
• Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion
  6 Feb 2025 1 repository listed
  In this paper, we show that the common practice of individually exploiting the text or image encoders of these powerful multi-modal models is highly suboptimal for intra-modal tasks like image-to-image retrieval.
• KPL: Training-Free Medical Knowledge Mining of Vision-Language Models
  20 Jan 2025 1 repository listed
  To tackle these challenges, we introduce the Knowledge Proxy Learning (KPL) to mine knowledge from CLIP.
• Post-hoc Probabilistic Vision-Language Models
  8 Dec 2024 1 repository listed Syntology ran 7 of 13 samples · 6 unverified · 5 pointer-only (licence)
  Vision-language models (VLMs), such as CLIP and SigLIP, have found remarkable success in classification, retrieval, and generative tasks.
• TaxaBind: A Unified Embedding Space for Ecological Applications
  1 Nov 2024 1 repository listed Syntology ran 2 of 3 samples · 1 unverified
  TaxaBind is a multimodal embedding space across six modalities: ground-level images of species, geographic location, satellite image, text, audio, and environmental features, useful for solving ecological problems.
• Multilingual Vision-Language Pre-training for the Remote Sensing Domain
  30 Oct 2024 1 repository listed
  This work proposes a novel vision-and-language model for the remote sensing domain, exploring the fine-tuning of a multilingual CLIP model and testing the use of a self-supervised method based on aligning local and…
• Altogether: Image Captioning via Re-aligning Alt-text
  22 Oct 2024 1 repository listed Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
  This paper focuses on creating synthetic data to improve the quality of image captions.
• Open-vocabulary vs. Closed-set: Best Practice for Few-shot Object Detection Considering Text Describability
  20 Oct 2024 1 repository listed
  Our findings reveal that: i) there is little difference between OVD and COD for object classes with low text-describability under equal conditions in OD pretraining; and ii) although OVD can learn from more diverse data…
• Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledge
  16 Oct 2024 1 repository listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  Contrastive Language-Image Pretraining (CLIP) performs zero-shot image classification by mapping images and textual class representation into a shared embedding space, then retrieving the class closest to the image.
• CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling
  28 Sep 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  In recent years, Contrastive Language-Image Pre-training (CLIP) has become a cornerstone in multimodal intelligence.
• DPA: Dual Prototypes Alignment for Unsupervised Adaptation of Vision-Language Models
  16 Aug 2024 1 repository listed
  Finally, it addresses visual-textual misalignment by aligning textual prototypes with image prototypes to further improve the adaptation performance.
• Do Vision-Language Foundational models show Robust Visual Perception?
  13 Aug 2024 1 repository listed
  However, it is unclear if these models are robust to distribution shifts, and how their performance and generalization capabilities vary under changes in data distribution.
• PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaboration
  28 Jun 2024 1 repository listed
  Vision Language Models (VLMs) like CLIP have attracted substantial attention in pathology, serving as backbones for applications such as zero-shot image classification and Whole Slide Image (WSI) analysis.
• Mitigate the Gap: Investigating Approaches for Improving Cross-Modal Alignment in CLIP
  25 Jun 2024 1 repository listed Syntology ran 14 of 22 samples · 8 unverified · 22 pointer-only (licence)
  We design AlignCLIP, in order to answer these questions and through extensive experiments, we show that AlignCLIP achieves noticeable enhancements in the cross-modal alignment of the embeddings, and thereby, reduces the…
• WATT: Weight Average Test-Time Adaptation of CLIP
  19 Jun 2024 1 repository listed Syntology ran 5 of 7 samples · 2 unverified
  In response, we present Weight Average Test-Time Adaptation (WATT) of CLIP, a pioneering approach facilitating full test-time adaptation (TTA) of this VLM.
• Mind's Eye: Image Recognition by EEG via Multimodal Similarity-Keeping Contrastive Learning
  5 Jun 2024 1 repository listed
  Decoding images from non-invasive electroencephalographic (EEG) signals has been a grand challenge in understanding how the human brain process visual information in real-world scenarios.
• What Do You See? Enhancing Zero-Shot Image Classification with Multimodal Large Language Models
  24 May 2024 1 repository listed
  Using multimodal LLMs, we generate comprehensive textual representations from input images.
• Who's in and who's out? A case study of multimodal CLIP-filtering in DataComp
  13 May 2024 1 repository listed
  As training datasets become increasingly drawn from unstructured, uncontrolled environments such as the web, researchers and industry practitioners have increasingly relied upon data filtering techniques to "filter out…

Syntology lines on 12 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.