Zero-Shot Video Retrieval

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

8 leaderboard tables shown for this task, 8 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

7 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 33 papers with code (40 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
  3 Oct 2023 6 repositories listed Syntology ran 7 of 14 samples · 7 unverified
  We thus propose VIDAL-10M with Video, Infrared, Depth, Audio and their corresponding Language, naming as VIDAL-10M.
• VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
  22 Apr 2021 5 repositories listed Syntology ran 5 of 8 samples · 3 unverified · 8 pointer-only (licence)
  We train VATT end-to-end from scratch using multimodal contrastive losses and evaluate its performance by the downstream tasks of video action recognition, audio event classification, image classification, and…
• CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval
  18 Apr 2021 5 repositories listed Syntology ran 3 of 4 samples · 1 unverified · 3 pointer-only (licence)
  In this paper, we propose a CLIP4Clip model to transfer the knowledge of the CLIP model to video-language retrieval in an end-to-end manner.
• Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
  1 Apr 2021 5 repositories listed Syntology ran 3 of 11 samples · 8 unverified
  Our objective in this work is video-text retrieval - in particular a joint embedding that enables efficient text-to-video retrieval.
• mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
  1 Feb 2023 4 repositories listed Syntology ran 9 of 19 samples · 10 unverified
  In contrast to predominant paradigms of solely relying on sequence-to-sequence generation or encoder-based instance discrimination, mPLUG-2 introduces a multi-module composition network by sharing common universal…
• End-to-End Learning of Visual Representations from Uncurated Instructional Videos
  13 Dec 2019 4 repositories listed Syntology ran 3 of 5 samples · 2 unverified
  Annotating videos is cumbersome, expensive and not scalable.
• ImageBind: One Embedding Space To Bind Them All
  9 May 2023 3 repositories listed Syntology ran 24 of 34 samples · 10 unverified · 32 pointer-only (licence)
  We show that all combinations of paired data are not necessary to train such a joint embedding, and only image-paired data is sufficient to bind the modalities together.
• Gramian Multimodal Representation Learning and Alignment
  16 Dec 2024 2 repositories listed Syntology ran 2 of 12 samples · 10 unverified
  These models typically align each modality to a designated anchor without ensuring the alignment of all modalities with each other, leading to suboptimal performance in tasks requiring a joint understanding of multiple…
• InternVideo2: Scaling Foundation Models for Multimodal Video Understanding
  22 Mar 2024 2 repositories listed
  We introduce InternVideo2, a new family of video foundation models (ViFM) that achieve the state-of-the-art results in video recognition, video-text tasks, and video-centric dialogue.
• InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
  21 Dec 2023 2 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs.
• VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset
  29 May 2023 2 repositories listed Syntology ran 15 of 42 samples · 27 unverified
  Based on the proposed VAST-27M dataset, we train an omni-modality video-text foundational model named VAST, which can perceive and process vision, audio, and subtitle modalities from video, and better support various…
• InternVideo: General Video Foundation Models via Generative and Discriminative Learning
  6 Dec 2022 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified
  Specifically, InternVideo efficiently explores masked video modeling and video-language contrastive learning as the pretraining objectives, and selectively coordinates video representations of these two complementary…
• Revealing Single Frame Bias for Video-and-Language Learning
  7 Jun 2022 2 repositories listed Syntology ran 4 of 12 samples · 8 unverified
  Training an effective video-and-language model intuitively requires multiple frames as model inputs.
• Bridging Video-text Retrieval with Multiple Choice Questions
  13 Jan 2022 2 repositories listed Syntology ran 13 of 24 samples · 11 unverified · 6 pointer-only (licence)
  As an additional benefit, our method achieves competitive results with much shorter pre-training videos on single-modality downstream tasks, e.
• Florence: A New Foundation Model for Computer Vision
  22 Nov 2021 2 repositories listed
  Computer vision foundation models, which are trained on diverse, large-scale dataset and can be adapted to a wide range of downstream tasks, are critical for this mission to solve real-world computer vision applications.
• VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding
  28 Sep 2021 2 repositories listed
  We present VideoCLIP, a contrastive approach to pre-train a unified model for zero-shot video and text understanding, without using any labels on downstream tasks.
• Make Your Training Flexible: Towards Deployment-Efficient Video Models
  18 Mar 2025 1 repository listed Syntology ran 7 of 17 samples · 10 unverified
  By making the sampling grid flexible and leveraging token selection, it is easily adopted in most popular video training frameworks, boosting model robustness with nearly no additional cost.
• vid-TLDR: Training Free Token merging for Light-weight Video Transformer
  20 Mar 2024 1 repository listed Syntology ran 3 of 4 samples · 1 unverified
  To tackle these issues, we propose training free token merging for lightweight video Transformer (vid-TLDR) that aims to enhance the efficiency of video Transformers by merging the background tokens without additional…
• HowToCaption: Prompting LLMs to Transform Video Annotations at Scale
  7 Oct 2023 1 repository listed Syntology ran 4 of 5 samples · 1 unverified · 5 pointer-only (licence)
  Specifically, we prompt an LLM to create plausible video captions based on ASR subtitles of instructional videos.
• BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning
  27 Sep 2023 1 repository listed Syntology ran 1 of 4 samples · 3 unverified
  Without bells and whistles, BT-Adapter achieves (1) state-of-the-art zero-shot results on various video tasks using thousands of fewer GPU hours.
• Unmasked Teacher: Towards Training-Efficient Video Foundation Models
  28 Mar 2023 1 repository listed Syntology ran 3 of 8 samples · 5 unverified
  Previous VFMs rely on Image Foundation Models (IFMs), which face challenges in transferring to the video domain.
• Seeing What You Miss: Vision-Language Pre-training with Semantic Completion Learning
  24 Nov 2022 1 repository listed
  Cross-modal alignment is essential for vision-language pre-training (VLP) models to learn the correct corresponding information across different modalities.
• Clover: Towards A Unified Video-Language Alignment and Fusion Model
  16 Jul 2022 1 repository listed
  We then introduce \textbf{Clover}\textemdash a Correlated Video-Language pre-training method\textemdash towards a universal Video-Language model for solving multiple video understanding tasks with neither performance…
• MILES: Visual BERT Pre-training with Injected Language Semantics for Video-text Retrieval
  26 Apr 2022 1 repository listed
  Dominant pre-training work for video-text retrieval mainly adopt the "dual-encoder" architectures to enable efficient retrieval, where two separate encoders are used to contrast global video and text representations,…
• Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
  1 Apr 2022 1 repository listed
  Large pretrained (e.
• Everything at Once - Multi-Modal Fusion Transformer for Video Retrieval
  1 Jan 2022 1 repository listed
  In this work, we present a multi-modal, modality agnostic fusion transformer that learns to exchange information between multiple modalities, such as video, audio, and text, and integrate them into a fused…
• Align and Prompt: Video-and-Language Pre-training with Entity Prompts
  17 Dec 2021 1 repository listed
  To achieve this, we first introduce an entity prompter module, which is trained with VTC to produce the similarity between a video crop and text prompts instantiated with entity names.
• Everything at Once -- Multi-modal Fusion Transformer for Video Retrieval
  8 Dec 2021 1 repository listed
  Multi-modal learning from video data has seen increased attention recently as it allows to train semantically meaningful embeddings without human annotation enabling tasks like zero-shot retrieval and classification.
• Object-aware Video-language Pre-training for Retrieval
  1 Dec 2021 1 repository listed
  In this work, we present Object-aware Transformers, an object-centric approach that extends video-language transformer to incorporate object representations.
• VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling
  24 Nov 2021 1 repository listed Syntology ran 0 of 2 samples · 2 unverified · 2 pointer-only (licence)
  Further, unlike previous studies that found pre-training tasks on video inputs (e.

Syntology lines on 19 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.