{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/composed-image-retrieval-with-text-feedback","title":"Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization","arxiv_id":"2211.07394","date":"2022-11-14","proceeding":null,"authors":["Yiyang Chen","Zhedong Zheng","Wei Ji","Leigang Qu","Tat-Seng Chua"],"abstract":"We investigate composed image retrieval with text feedback. Users gradually look for the target of interest by moving from coarse to fine-grained feedback. However, existing methods merely focus on the latter, i.e., fine-grained search, by harnessing positive and negative pairs during training. This pair-based paradigm only considers the one-to-one distance between a pair of specific points, which is not aligned with the one-to-many coarse-grained retrieval process and compromises the recall rate. In an attempt to fill this gap, we introduce a unified learning approach to simultaneously modeling the coarse- and fine-grained retrieval by considering the multi-grained uncertainty. The key idea underpinning the proposed method is to integrate fine- and coarse-grained retrieval as matching data points with small and large fluctuations, respectively. Specifically, our method contains two modules: uncertainty modeling and uncertainty regularization. (1) The uncertainty modeling simulates the multi-grained queries by introducing identically distributed fluctuations in the feature space. (2) Based on the uncertainty modeling, we further introduce uncertainty regularization to adapt the matching objective according to the fluctuation range. Compared with existing methods, the proposed strategy explicitly prevents the model from pushing away potential candidates in the early stage, and thus improves the recall rate. On the three public datasets, i.e., FashionIQ, Fashion200k, and Shoes, the proposed method has achieved +4.03%, +3.38%, and +2.40% Recall@50 accuracy over a strong baseline, respectively.","url_abs":"https://arxiv.org/abs/2211.07394v6","url_pdf":"https://arxiv.org/pdf/2211.07394v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"composed-image-retrieval-with-text-feedback","repo_url":"https://github.com/Monoxide-Chen/uncertainty_retrieval","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"composed-image-retrieval","task_name":"Composed Image Retrieval (CoIR)"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"multi-modal","task_name":"Image Retrieval with Multi-Modal Query"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-retrieval-on-fashion-iq","task":"Image Retrieval","dataset":"Fashion IQ","model":"MUR (4*ResNet50)","rank_in_archive_order":13,"of":22,"metrics":{"(Recall@10+Recall@50)/2":"50.61"},"uses_additional_data":false},{"leaderboard":"/sota/image-retrieval-on-fashion-iq","task":"Image Retrieval","dataset":"Fashion IQ","model":"MUR","rank_in_archive_order":15,"of":22,"metrics":{"(Recall@10+Recall@50)/2":"47.28"},"uses_additional_data":false},{"leaderboard":"/sota/image-retrieval-with-multi-modal-query-on","task":"Image Retrieval with Multi-Modal Query","dataset":"Fashion200k","model":"Multi-grained Uncertainty Regularization(MUR)","rank_in_archive_order":3,"of":8,"metrics":{"Recall@1":"21.8","Recall@10":"52.1","Recall@50":"70.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.07394","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.07394"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Monoxide-Chen/uncertainty_retrieval","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"official":{"samples":10,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"77dc755eec4b339c","entry":"calculate_mean_std","repo":"Monoxide-Chen/uncertainty_retrieval","repo_kind":"official","path":"models/utils.py","file_url":"https://github.com/Monoxide-Chen/uncertainty_retrieval/blob/HEAD/models/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77dc755eec4b339c"}},{"code_sha256_prefix":"82c40579a3041ca0","entry":"equal_lr","repo":"Monoxide-Chen/uncertainty_retrieval","repo_kind":"official","path":"models/utils.py","file_url":"https://github.com/Monoxide-Chen/uncertainty_retrieval/blob/HEAD/models/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82c40579a3041ca0"}},{"code_sha256_prefix":"68994a7ec705cdf0","entry":"reshape_text_features_to_concat","repo":"Monoxide-Chen/uncertainty_retrieval","repo_kind":"official","path":"models/utils.py","file_url":"https://github.com/Monoxide-Chen/uncertainty_retrieval/blob/HEAD/models/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68994a7ec705cdf0"}},{"code_sha256_prefix":"7657330ca2d9594a","entry":"create_experiment_export_folder","repo":"Monoxide-Chen/uncertainty_retrieval","repo_kind":"official","path":"set_up.py","file_url":"https://github.com/Monoxide-Chen/uncertainty_retrieval/blob/HEAD/set_up.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7657330ca2d9594a"}},{"code_sha256_prefix":"6cad2a4ad3a87d25","entry":"create_lr_schedulers","repo":"Monoxide-Chen/uncertainty_retrieval","repo_kind":"official","path":"optimizers.py","file_url":"https://github.com/Monoxide-Chen/uncertainty_retrieval/blob/HEAD/optimizers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6cad2a4ad3a87d25"}},{"code_sha256_prefix":"fffafb2f1a4e975d","entry":"create_optimizers","repo":"Monoxide-Chen/uncertainty_retrieval","repo_kind":"official","path":"optimizers.py","file_url":"https://github.com/Monoxide-Chen/uncertainty_retrieval/blob/HEAD/optimizers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fffafb2f1a4e975d"}},{"code_sha256_prefix":"6e93e17cdd292238","entry":"create_read_func","repo":"Monoxide-Chen/uncertainty_retrieval","repo_kind":"official","path":"language/utils.py","file_url":"https://github.com/Monoxide-Chen/uncertainty_retrieval/blob/HEAD/language/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e93e17cdd292238"}},{"code_sha256_prefix":"aac3a343da033f5d","entry":"create_write_func","repo":"Monoxide-Chen/uncertainty_retrieval","repo_kind":"official","path":"language/utils.py","file_url":"https://github.com/Monoxide-Chen/uncertainty_retrieval/blob/HEAD/language/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aac3a343da033f5d"}},{"code_sha256_prefix":"15df572f14aa3ffb","entry":"fix_random_seed_as","repo":"Monoxide-Chen/uncertainty_retrieval","repo_kind":"official","path":"set_up.py","file_url":"https://github.com/Monoxide-Chen/uncertainty_retrieval/blob/HEAD/set_up.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"15df572f14aa3ffb"}},{"code_sha256_prefix":"465d0cb5ececdb3e","entry":"get_name_of_experiment_path","repo":"Monoxide-Chen/uncertainty_retrieval","repo_kind":"official","path":"set_up.py","file_url":"https://github.com/Monoxide-Chen/uncertainty_retrieval/blob/HEAD/set_up.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"465d0cb5ececdb3e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}