ILIAS (ILIAS: Instance-Level Image retrieval At Scale)

ILIAS is a large-scale test dataset for evaluation on Instance-Level Image retrieval At Scale. It is designed to support future research in image-to-image and text-to-image retrieval for particular objects and serves as a benchmark for evaluating representations of foundation or customized vision and vision-language models, as well as specialized retrieval techniques.

website | dataset | arxiv | huggingface

Composition

The dataset includes 1,000 object instances across diverse domains, with: * 5,947 images in total: * 1,232 image queries, depicting query objects on clean or uniform background * 4,715 positive images, featuring the query objects in real-world conditions with clutter, occlusions, scale variations, and partial views * 1,000 text queries, providing fine-grained textual descriptions of the query objects * 100M distractors from YFCC100M to evaluate retrieval performance under large-scale settings, while asserting noise-free ground truth