</> Code with Papers by Syntology

Home › Datasets › task › Cross-Modal Retrieval

Cross-Modal Retrieval datasets

archive 2025-07-28

24 datasets carry the task tag "Cross-Modal Retrieval" (the task itself: Cross-Modal Retrieval), ordered by the archive's paper count. Page 1 of 1: 24 shown of 24. Facet routes are this site's own (the archive records the tag string, not a page).

The archive holds 12,214 dataset rows; 12,172 are listed. 6 are withheld from every listing and count here as vandalised before snapshot (6 with contact-centre spam in the title, 0 with a spam description on a row that has no homepage, no paper and no papers counted; none with more than 1 paper, 0 with a benchmark), listed in withheld.json; 1 listed row carries a vandalised description, withheld on its page. This gate never withholds a row with a homepage or a paper that resolves, and a clean description; the content rules below withhold a row whose name is spam whatever else it carries. The gate is a phrase list: these are the rows it caught, not a claim that the rest is clean. Before that gate, the site's content rules withhold 36 more rows (invite-code, gambling, travel-booking, contact-centre and similar spam in the name or on a row with nothing real behind it); they have no page and are listed in withheld.json.

Filter 51 task tags shown of 3,717, by dataset count; the full filter by modality, task and language is on /datasets

All 12,172 Question Answering 413Semantic Segmentation 347Object Detection 332Image Classification 283Classification 222Text Classification 167Language Modelling 166Text Generation 162Visual Question Answering (VQA) 144Named Entity Recognition (NER) 130Pose Estimation 124Anomaly Detection 119Action Recognition 115Instance Segmentation 111Sentiment Analysis 104Text Summarization 98Domain Adaptation 96Speech Recognition 96Reading Comprehension 95Information Retrieval 932D Object Detection 92Image Retrieval 87Machine Translation 82Relation Extraction 822D Semantic Segmentation 81Natural Language Inference 81Image Captioning 79Image Generation 79Depth Estimation 77Node Classification 75Natural Language Understanding 72Autonomous Driving 70Code Generation 70Object Tracking 693D Object Detection 67Face Recognition 67Link Prediction 67Person Re-Identification 65Data Augmentation 63Multi-Task Learning 593D Reconstruction 58Common Sense Reasoning 56Recommendation Systems 56Video Understanding 56Optical Character Recognition (OCR) 55Graph Classification 54Abstractive Text Summarization 52Emotion Recognition 52Medical Image Segmentation 52Word Embeddings 52Cross-Modal Retrieval 24

Cross-Modal Retrieval datasets 1–24 of 24

COCO (Common Objects in Context)
The COCO (Common Objects in Context) dataset is a large-scale object detection, segmentation, and captioning dataset.
ImagesVideosObject DetectionSemantic SegmentationQuestion AnsweringPose EstimationVisual Question Answering (VQA)Instance Segmentation+54 tasks
11,922 papers · 77 benchmarks
Flickr30k
The Flickr30k dataset contains 31,000 images collected from Flickr, together with 5 reference sentences provided by human annotators.
ImagesTextsNode ClassificationImage RetrievalImage CaptioningCross-Modal RetrievalImage-to-Text RetrievalPhrase Grounding+5 tasks
880 papers · 9 benchmarks
NUS-WIDE
The NUS-WIDE dataset contains 269,648 images with a total of 5,018 tags collected from Flickr.
ImagesImage RetrievalMulti-Label ClassificationCross-Modal RetrievalMultiview ClusteringMulti-label zero-shot learning
348 papers · 3 benchmarks
PASCAL VOC 2007
PASCAL VOC 2007 is a dataset for image recognition.
ImagesImage ClassificationObject DetectionSemantic SegmentationMulti-Label ClassificationObject LocalizationCross-Modal Retrieval+10 tasks
126 papers · 13 benchmarks
CUHK-PEDES
The CUHK-PEDES dataset is a caption-annotated pedestrian dataset.
ImagesTextsCross-Modal RetrievalText based Person RetrievalText-based Person Retrieval with Noisy CorrespondencePedestrian Image Caption
93 papers · 3 benchmarks
MSCOCO
Click to add a brief description of the dataset (Markdown and LaTeX enabled).
Object DetectionImage RetrievalImage CaptioningCross-Modal RetrievalWeakly Supervised Object DetectionMulti-Label Image Classification+7 tasks
90 papers · 13 benchmarks
RSICD (Remote Sensing Image Captioning Dataset)
ImagesImage CaptioningCross-Modal RetrievalText RetrievalImage-to-Text RetrievalScene Classification
70 papers · 3 benchmarks
Recipe1M+
Recipe1M+ is a dataset which contains one million structured cooking recipes with 13M associated images.
ImagesTextsImage GenerationCross-Modal RetrievalRecipe GenerationFood Recognition
68 papers · 3 benchmarks
ChEBI-20
Dataset contains 33,010 molecule-description pairs split into 80\%/10\%/10\% train/val/test splits.
TextsGraphsBiomedicalImage CaptioningCross-Modal RetrievalMolecule CaptioningText-based de novo Molecule Generation
43 papers · 4 benchmarks
RSITMD
Click to add a brief description of the dataset (Markdown and LaTeX enabled).
Cross-Modal Retrieval
26 papers · 1 benchmark
SemArt
SemArt is a multi-modal dataset for semantic art understanding.
ImagesTextsCross-Modal RetrievalMulti-Task LearningArt Analysis
16 papers · 0 benchmarks
SoundingEarth
SoundingEarth consists of co-located aerial imagery and audio samples all around the world.
ImagesAudioCross-Modal Retrieval
8 papers · 1 benchmark
ChineseFoodNet
ChineseFoodNet aims to automatically recognizing pictured Chinese dishes.
ImagesImage GenerationCross-Modal RetrievalFood Recognition
6 papers · 0 benchmarks
Flickr-8k
Contains 8k flickr Images with captions.
ImagesImage CaptioningText-to-Image GenerationCross-Modal Retrieval
5 papers · 2 benchmarks
Song Describer Dataset
The Song Describer Dataset (SDD) contains ~1.1k captions for 706 permissively licensed music recordings.
TextsAudioMusicCross-Modal RetrievalText-to-Music GenerationMusic GenerationMusic Captioning
5 papers · 1 benchmark
Twitter100k
Twitter100k is a large-scale dataset for weakly supervised cross-media retrieval.
ImagesTextsOptical Character Recognition (OCR)Cross-Modal RetrievalQuantization
5 papers · 0 benchmarks
CTC (COCO-Text Captioned)
A dataset that allows exploration of cross-modal retrieval where images contain scene-text instances.
ImagesCross-Modal Retrieval
3 papers · 0 benchmarks
PoseScript
PoseScript is a dataset that pairs a few thousand 3D human poses from AMASS with rich human-annotated descriptions of the body parts and their spatial relationships.
3DQuestion AnsweringVisual Question Answering (VQA)Image CaptioningRetrievalCross-Modal Retrieval
2 papers · 0 benchmarks
CiNAT-Birds-2021 (Cross-View iNaturalist Birds 2021)
CiNAT Birds 2021 (Cross-View iNaturalist-2021 Birds) dataset contains ground-level images of bird species along with satellite images associated with the geolocation of the ground-level images.
ImagesFine-Grained Image ClassificationCross-Modal Retrieval
1 paper · 0 benchmarks
Earth on Canvas
A Zero-Shot Sketch-based Inter-Modal Object Retrieval Scheme for Remote Sensing Images WITH the advancement in sensor technology, huge amounts of data are being collected from various satellites.
ImagesCross-Modal RetrievalCross-Domain Few-ShotZero-Shot Cross-Modal Retrieval
1 paper · 0 benchmarks
IAPR TC-12 (IAPR TC-12 Benchmark)
The image collection of the IAPR TC-12 Benchmark consists of 20,000 still natural images taken from locations around the world and comprising an assorted cross-section of still natural images.
ImagesTextsImage RetrievalCross-Modal Retrieval
1 paper · 0 benchmarks
IMPACT Patent (A Large-scale Integrated Multimodal Patent Analysis and Creation Dataset for Design Patents)
It is a large-scale multimodal patent dataset with detailed captions for design patent figures.
ImagesTextsImage ClassificationImage RetrievalCross-Modal RetrievalZero-Shot Cross-Modal RetrievalImage-text RetrievalPatent classification
1 paper · 1 benchmark
MuseChat Dataset (MuseChat: A Conversational Music Recommendation System for Videos (CVPR 2024 Highlight Paper))
Music recommendation for videos attracts growing interest in multi-modal research.
VideosTextsAudioRecommendation SystemsVideo UnderstandingCross-Modal RetrievalMusic Recommendation
1 paper · 0 benchmarks
TaxaBench-8k
A truly multimodal dataset for benchmarking deep learning models on ecological tasks.
Cross-Modal RetrievalZero-Shot Image Classification
1 paper · 0 benchmarks

Paper counts and descriptions are the archive's, frozen 2025-07-28; no citation counts, no stars, no trending. Sorting by "most cited" or "newest" was a live-site feature the archive does not carry.