Papers › An Efficient Post-hoc Framework for Reducing Task Discrepancy of Text Encoders for...
An Efficient Post-hoc Framework for Reducing Task Discrepancy of Text Encoders for Composed Image Retrieval
Jaeseok Byun, Seokhyeon Jeong, Wonjae Kim, Sanghyuk Chun, Taesup Moon
Composed Image Retrieval (CIR) aims to retrieve a target image based on a reference image and conditioning text, enabling controllable image searches. The mainstream Zero-Shot (ZS) CIR methods bypass the need for expensive training CIR triplets by projecting image embeddings into the text token embedding space, forming a composed query for retrieval. However, we highlight an inherent limitation in these projection-based CIR: a task discrepancy of text encoders between the original pre-training task of the encoders (text ↔ image) and the target CIR task (image + text ↔ image), which potentially negatively impacts CIR performance. To reduce such a discrepancy, a naive solution would be to train both image and text encoders with CIR triplets in a supervised manner. Instead, we introduce Reducing Task Discrepancy of Text Encoders (RTD), an efficient text-only post-hoc framework that complements projection-based CIR methods. We devise a novel target-anchored text contrastive learning designed to enhance the capability of the text encoder for CIR. We also propose two key enhancements: (1) a hard negative-based refined batch sampling strategy and (2) a refined concatenation scheme to further mitigate training-inference discrepancy. Integrating RTD into state-of-the-art projection-based methods achieves performance comparable to, or even surpassing, resource-intensive state-of-the-art synthetic CIR triplet-based approaches only with 23 minutes of additional training on 4 A100 GPUs (up to 100× faster in training). Our code will be available upon acceptance.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
1 archive task tag without a task page not shown.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Zero-Shot Composed Image Retrieval (ZS-CIR) | CIRCO | RTD + LinCIR (CLIP G/14) | mAP@10 | 22.29 | #21 of 43 | Archive leaderboard | report |
| Zero-Shot Composed Image Retrieval (ZS-CIR) | CIRCO | RTD + LinCIR (CLIP L/14) | mAP@10 | 18.11 | #28 of 43 | Archive leaderboard | report |
| Zero-Shot Composed Image Retrieval (ZS-CIR) | CIRR | RTD + LinCIR (CLIP G/14) | R@5 | 67.47 | #23 of 47 | Archive leaderboard | report |
| Zero-Shot Composed Image Retrieval (ZS-CIR) | CIRR | RTD + LinCIR (CLIP L/14) | R@5 | 56.17 | #33 of 47 | Archive leaderboard | report |
| Zero-Shot Composed Image Retrieval (ZS-CIR) | Fashion IQ | RTD + LinCIR (CLIP G/14) | (Recall@10+Recall@50)/2 | 56.74 | #1 of 41 | Archive leaderboard | report |
| Zero-Shot Composed Image Retrieval (ZS-CIR) | Fashion IQ | RTD + LinCIR (CLIP L/14) | (Recall@10+Recall@50)/2 | 40.66 | #23 of 41 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections