Papers › Zero-Shot Composed Image Retrieval with Textual Inversion

Zero-Shot Composed Image Retrieval with Textual Inversion

27 Mar 2023ICCV 2023 1arXiv:2303.15247archive 2025-07-28

Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, Alberto del Bimbo

Composed Image Retrieval (CIR) aims to retrieve a target image based on a query composed of a reference image and a relative caption that describes the difference between the two images. The high effort and cost required for labeling datasets for CIR hamper the widespread usage of existing methods, as they rely on supervised learning. In this work, we propose a new task, Zero-Shot CIR (ZS-CIR), that aims to address CIR without requiring a labeled training dataset. Our approach, named zero-Shot composEd imAge Retrieval with textuaL invErsion (SEARLE), maps the visual features of the reference image into a pseudo-word token in CLIP token embedding space and integrates it with the relative caption. To support research on ZS-CIR, we introduce an open-domain benchmarking dataset named Composed Image Retrieval on Common Objects in context (CIRCO), which is the first dataset for CIR containing multiple ground truths for each query. The experiments show that SEARLE exhibits better performance than the baselines on the two main datasets for CIR tasks, FashionIQ and CIRR, and on the proposed CIRCO. The dataset, the code and the model are publicly available at https://github.com/miccunifi/SEARLE.

PaperPDFConference PDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

miccunifi/circo officialmentioned in papermentioned on GitHubpytorchNOASSERTION report
miccunifi/searle officialmentioned in papermentioned on GitHubpytorchNOASSERTION report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Composed Image Retrieval (CoIR)Image RetrievalRetrievalZero-Shot Composed Image Retrieval (ZS-CIR)

Datasets

Introduced by this paper, per the archive.

CIRCO

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Zero-Shot Composed Image Retrieval (ZS-CIR) CIRCO SEARLE-XL (CLIP L/14) mAP@10 12.73 #35 of 43 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) CIRCO SEARLE (CLIP B/32) mAP@10 9.94 #40 of 43 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) CIRR SEARLE R@5 53.42 #43 of 47 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) CIRR SEARLE-XL R@5 52.48 #45 of 47 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) COCO (Common Objects in Context) SEARLE-XL-OTI (CLIP L/14) Actions Recall@5 31.43 #2 of 10 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) COCO (Common Objects in Context) SEARLE-XL (CLIP L/14) Actions Recall@5 29.02 #4 of 10 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) COCO (Common Objects in Context) SEARLE-OTI (CLIP B/32) Actions Recall@5 26.00 #8 of 10 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) COCO (Common Objects in Context) SEARLE (CLIP B/32) Actions Recall@5 24.58 #10 of 10 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) Fashion IQ SEARLE-XL-OTI (CLIP L/14) (Recall@10+Recall@50)/2 37.76 #31 of 41 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) Fashion IQ SEARLE-XL (CLIP L/14) (Recall@10+Recall@50)/2 35.90 #34 of 41 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) Fashion IQ SEARLE (CLIP B/32) (Recall@10+Recall@50)/2 32.71 #38 of 41 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) Fashion IQ SEARLE-OTI (CLIP B/32) (Recall@10+Recall@50)/2 32.39 #39 of 41 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) FashionIQ SEARLE-XL-OTI R@10 27.61 #1 of 1 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) GeneCIS SEARLE (CLIP B/32) A-R@1 14.4 #4 of 11 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) GeneCIS SEARLE (CLIP L/14) A-R@1 14.4 #5 of 11 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) ImageNet SEARLE-XL (CLIP L/14) Average Recall 21.54 #4 of 11 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) ImageNet SEARLE-XL-OTI (CLIP B/32) Average Recall 20.42 #5 of 11 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) ImageNet SEARLE-OTI (CLIP B/32) Average Recall 12.77 #10 of 11 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) ImageNet SEARLE (CLIP B/32) Average Recall 11.94 #11 of 11 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) ImageNet-R SEARLE-XL (CLIP L/14) (Recall@10+Recall@50)/2 21.54 #7 of 20 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) ImageNet-R SEARLE-XL-OTI (CLIP B/32) (Recall@10+Recall@50)/2 20.42 #8 of 20 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) ImageNet-R SEARLE-OTI (CLIP B/32) (Recall@10+Recall@50)/2 12.77 #13 of 20 Archive leaderboard report
Zero-Shot Composed Image Retrieval (ZS-CIR) ImageNet-R SEARLE (CLIP B/32) (Recall@10+Recall@50)/2 11.94 #14 of 20 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

CLIP

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections