{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-unified-text-based-person-retrieval-a","title":"Towards Unified Text-based Person Retrieval: A Large-scale Multi-Attribute and Language Search Benchmark","arxiv_id":"2306.02898","date":"2023-06-05","proceeding":null,"authors":["Shuyu Yang","Yinan Zhou","Yaxiong Wang","Yujiao Wu","Li Zhu","Zhedong Zheng"],"abstract":"In this paper, we introduce a large Multi-Attribute and Language Search dataset for text-based person retrieval, called MALS, and explore the feasibility of performing pre-training on both attribute recognition and image-text matching tasks in one stone. In particular, MALS contains 1,510,330 image-text pairs, which is about 37.5 times larger than prevailing CUHK-PEDES, and all images are annotated with 27 attributes. Considering the privacy concerns and annotation costs, we leverage the off-the-shelf diffusion models to generate the dataset. To verify the feasibility of learning from the generated data, we develop a new joint Attribute Prompt Learning and Text Matching Learning (APTM) framework, considering the shared knowledge between attribute and text. As the name implies, APTM contains an attribute prompt learning stream and a text matching learning stream. (1) The attribute prompt learning leverages the attribute prompts for image-attribute alignment, which enhances the text matching learning. (2) The text matching learning facilitates the representation learning on fine-grained details, and in turn, boosts the attribute prompt learning. Extensive experiments validate the effectiveness of the pre-training on MALS, achieving state-of-the-art retrieval performance via APTM on three challenging real-world benchmarks. In particular, APTM achieves a consistent improvement of +6.96%, +7.68%, and +16.95% Recall@1 accuracy on CUHK-PEDES, ICFG-PEDES, and RSTPReid datasets by a clear margin, respectively.","url_abs":"https://arxiv.org/abs/2306.02898v4","url_pdf":"https://arxiv.org/pdf/2306.02898v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-unified-text-based-person-retrieval-a","repo_url":"https://github.com/Shuyu-XJTU/APTM","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"image-text-matching","task_name":"Image-text matching"},{"task_slug":"pedestrian-attribute-recognition","task_name":"Pedestrian Attribute Recognition"},{"task_slug":"person-retrieval","task_name":"Person Retrieval"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"text-matching","task_name":"Text Matching"},{"task_slug":"nlp-based-person-retrival","task_name":"Text based Person Retrieval"},{"task_slug":"text-based-person-retrieval","task_name":"Text-based Person Retrieval"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/pedestrian-attribute-recognition-on-pa-100k","task":"Pedestrian Attribute Recognition","dataset":"PA-100K","model":"APTM","rank_in_archive_order":7,"of":13,"metrics":{"Accuracy":"80.17"},"uses_additional_data":true},{"leaderboard":"/sota/nlp-based-person-retrival-on-cuhk-pedes","task":"Text based Person Retrieval","dataset":"CUHK-PEDES","model":"APTM","rank_in_archive_order":2,"of":21,"metrics":{"R@1":"76.53","R@10":"94.15","R@5":"90.04","mAP":"66.91"},"uses_additional_data":true},{"leaderboard":"/sota/text-based-person-retrieval-on-icfg-pedes","task":"Text based Person Retrieval","dataset":"ICFG-PEDES","model":"APTM","rank_in_archive_order":1,"of":12,"metrics":{"R@1":"68.51","mAP":"41.22"},"uses_additional_data":true},{"leaderboard":"/sota/text-based-person-retrieval-on-rstpreid-1","task":"Text based Person Retrieval","dataset":"RSTPReid","model":"APTM","rank_in_archive_order":2,"of":9,"metrics":{"R@1":"67.50","R@10":"91.45","R@5":"85.70"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2306.02898","atlas_url":"https://app.syntology.ai/?focus=2306.02898","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.02898"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Shuyu-XJTU/APTM","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":4,"ran_fixture":1,"ran":1,"unverified":6},"by_repo_kind":{"official":{"samples":12,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cf9ffa02a42184af","entry":"whitespace_tokenize","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"models/tokenization_bert.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/models/tokenization_bert.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf9ffa02a42184af"}},{"code_sha256_prefix":"144d10b49baeb8a6","entry":"window_partition","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"models/swin_transformer.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/models/swin_transformer.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":2,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"144d10b49baeb8a6"}},{"code_sha256_prefix":"51e7514fd736cca3","entry":"build_mlp","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"models/aptm.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/models/aptm.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"51e7514fd736cca3"}},{"code_sha256_prefix":"e7fbc7a74a3457c7","entry":"load_vocab","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"models/tokenization_bert.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/models/tokenization_bert.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e7fbc7a74a3457c7"}},{"code_sha256_prefix":"9e8f81565e828ebf","entry":"top_k_top_p_filtering","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"models/bert.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/models/bert.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9e8f81565e828ebf"}},{"code_sha256_prefix":"61bf152e6a42a184","entry":"window_reverse","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"models/swin_transformer.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/models/swin_transformer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61bf152e6a42a184"}},{"code_sha256_prefix":"d7bf59874fc1bb8f","entry":"accs","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"reTools.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/reTools.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d7bf59874fc1bb8f"}},{"code_sha256_prefix":"0aa5e4369a69b540","entry":"create_optimizer","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"optim.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/optim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0aa5e4369a69b540"}},{"code_sha256_prefix":"37c7bac7616d388a","entry":"create_scheduler","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"scheduler.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/scheduler.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"37c7bac7616d388a"}},{"code_sha256_prefix":"199b2766250bd139","entry":"interpolate_relative_pos_embed","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"models/swin_transformer.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/models/swin_transformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"199b2766250bd139"}},{"code_sha256_prefix":"26be70dca3249c0b","entry":"load_tf_weights_in_bert","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"models/bert.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/models/bert.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26be70dca3249c0b"}},{"code_sha256_prefix":"c28a1bc789b78770","entry":"mlm","repo":"Shuyu-XJTU/APTM","repo_kind":"official","path":"train_tools.py","file_url":"https://github.com/Shuyu-XJTU/APTM/blob/HEAD/train_tools.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c28a1bc789b78770"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}