{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-enhancement-improves-text-image","title":"Self-Enhancement Improves Text-Image Retrieval in Foundation Visual-Language Models","arxiv_id":"2306.06691","date":"2023-06-11","proceeding":null,"authors":["Yuguang Yang","Yiming Wang","Shupeng Geng","Runqi Wang","Yimi Wang","Sheng Wu","Baochang Zhang"],"abstract":"The emergence of cross-modal foundation models has introduced numerous approaches grounded in text-image retrieval. However, on some domain-specific retrieval tasks, these models fail to focus on the key attributes required. To address this issue, we propose a self-enhancement framework, A^{3}R, based on the CLIP-ViT/G-14, one of the largest cross-modal models. First, we perform an Attribute Augmentation strategy to enrich the textual description for fine-grained representation before model learning. Then, we propose an Adaption Re-ranking method to unify the representation space of textual query and candidate images and re-rank candidate images relying on the adapted query after model learning. The proposed framework is validated to achieve a salient improvement over the baseline and other teams' solutions in the cross-modal image retrieval track of the 1st foundation model challenge without introducing any additional samples. The code is available at \\url{https://github.com/CapricornGuang/A3R}.","url_abs":"https://arxiv.org/abs/2306.06691v1","url_pdf":"https://arxiv.org/pdf/2306.06691v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-enhancement-improves-text-image","repo_url":"https://github.com/capricornguang/a3r","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"re-ranking","task_name":"Re-Ranking"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[{"method_slug":"focus","method_name":"Focus"},{"method_slug":"fail","method_name":"fail"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}