{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/word-for-person-zero-shot-composed-person","title":"Automatic Synthetic Data and Fine-grained Adaptive Feature Alignment for Composed Person Retrieval","arxiv_id":"2311.16515","date":"2023-11-25","proceeding":null,"authors":["Delong Liu","Haiwen Li","Zhaohui Hou","Zhicheng Zhao","Fei Su","Yuan Dong"],"abstract":"Person retrieval has attracted rising attention. Existing methods are mainly divided into two retrieval modes, namely image-only and text-only. However, they are unable to make full use of the available information and are difficult to meet diverse application requirements. To address the above limitations, we propose a new Composed Person Retrieval (CPR) task, which combines visual and textual queries to identify individuals of interest from large-scale person image databases. Nevertheless, the foremost difficulty of the CPR task is the lack of available annotated datasets. Therefore, we first introduce a scalable automatic data synthesis pipeline, which decomposes complex multimodal data generation into the creation of textual quadruples followed by identity-consistent image synthesis using fine-tuned generative models. Meanwhile, a multimodal filtering method is designed to ensure the resulting SynCPR dataset retains 1.15 million high-quality and fully synthetic triplets. Additionally, to improve the representation of composed person queries, we propose a novel Fine-grained Adaptive Feature Alignment (FAFA) framework through fine-grained dynamic alignment and masked feature reasoning. Moreover, for objective evaluation, we manually annotate the Image-Text Composed Person Retrieval (ITCPR) test set. The extensive experiments demonstrate the effectiveness of the SynCPR dataset and the superiority of the proposed FAFA framework when compared with the state-of-the-art methods. All code and data will be provided at https://github.com/Delong-liu-bupt/Composed_Person_Retrieval.","url_abs":"https://arxiv.org/abs/2311.16515v4","url_pdf":"https://arxiv.org/pdf/2311.16515v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"word-for-person-zero-shot-composed-person","repo_url":"https://github.com/Delong-liu-bupt/Word4Per","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"person-retrieval","task_name":"Person Retrieval"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"nlp-based-person-retrival","task_name":"Text based Person Retrieval"},{"task_slug":"text-based-person-retrieval","task_name":"Text-based Person Retrieval"},{"task_slug":"zero-shot-composed-person-retrieval","task_name":"Zero-shot Composed Person Retrieval"}],"methods":[],"datasets_introduced":[{"slug":"itcpr-dataset","name":"ITCPR dataset","full_name":"Image-Text Composed Person Retrieval dataset"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/zero-shot-composed-person-retrieval-on-itcpr","task":"Zero-shot Composed Person Retrieval","dataset":"ITCPR dataset","model":"FAFA","rank_in_archive_order":1,"of":2,"metrics":{" Rank-1":"46.54","mAP":"55.60"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-composed-person-retrieval-on-itcpr","task":"Zero-shot Composed Person Retrieval","dataset":"ITCPR dataset","model":"Word4Per（FAFA old version）","rank_in_archive_order":2,"of":2,"metrics":{" Rank-1":"45.55","mAP":"55.26"},"uses_additional_data":true}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}