{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/data-roaming-and-early-fusion-for-composed","title":"Data Roaming and Quality Assessment for Composed Image Retrieval","arxiv_id":"2303.09429","date":"2023-03-16","proceeding":null,"authors":["Matan Levy","Rami Ben-Ari","Nir Darshan","Dani Lischinski"],"abstract":"The task of Composed Image Retrieval (CoIR) involves queries that combine image and text modalities, allowing users to express their intent more effectively. However, current CoIR datasets are orders of magnitude smaller compared to other vision and language (V&L) datasets. Additionally, some of these datasets have noticeable issues, such as queries containing redundant modalities. To address these shortcomings, we introduce the Large Scale Composed Image Retrieval (LaSCo) dataset, a new CoIR dataset which is ten times larger than existing ones. Pre-training on our LaSCo, shows a noteworthy improvement in performance, even in zero-shot. Furthermore, we propose a new approach for analyzing CoIR datasets and methods, which detects modality redundancy or necessity, in queries. We also introduce a new CoIR baseline, the Cross-Attention driven Shift Encoder (CASE). This baseline allows for early fusion of modalities using a cross-attention module and employs an additional auxiliary task during training. Our experiments demonstrate that this new baseline outperforms the current state-of-the-art methods on established benchmarks like FashionIQ and CIRR.","url_abs":"https://arxiv.org/abs/2303.09429v2","url_pdf":"https://arxiv.org/pdf/2303.09429v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-roaming-and-early-fusion-for-composed","repo_url":"https://github.com/levymsn/LaSCo","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"composed-image-retrieval","task_name":"Composed Image Retrieval (CoIR)"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[{"slug":"lasco","name":"LaSCo","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-retrieval-on-cirr","task":"Image Retrieval","dataset":"CIRR","model":"CASE (Pre-trained on LaSCo.Ca)","rank_in_archive_order":7,"of":17,"metrics":{"(Recall@5+Recall_subset@1)/2":"78.25","Recall@10":"88.75"},"uses_additional_data":true},{"leaderboard":"/sota/image-retrieval-on-cirr","task":"Image Retrieval","dataset":"CIRR","model":"CASE","rank_in_archive_order":8,"of":17,"metrics":{"(Recall@5+Recall_subset@1)/2":"77.5","Recall@10":"87.25"},"uses_additional_data":false},{"leaderboard":"/sota/image-retrieval-on-fashion-iq","task":"Image Retrieval","dataset":"Fashion IQ","model":"CASE","rank_in_archive_order":7,"of":22,"metrics":{"(Recall@10+Recall@50)/2":"59.73","Recall@10":"48.79"},"uses_additional_data":false},{"leaderboard":"/sota/image-retrieval-on-lasco","task":"Image Retrieval","dataset":"LaSCo","model":"CASE","rank_in_archive_order":1,"of":3,"metrics":{"Recall@1 (%)":"7.08"},"uses_additional_data":false},{"leaderboard":"/sota/image-retrieval-on-lasco","task":"Image Retrieval","dataset":"LaSCo","model":"BLIP4CIR","rank_in_archive_order":2,"of":3,"metrics":{"Recall@1 (%)":"4.26"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2303.09429","atlas_url":"https://app.syntology.ai/?focus=2303.09429","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}