{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-prior-instruction-representation-framework","title":"A Prior Instruction Representation Framework for Remote Sensing Image-text Retrieval","arxiv_id":null,"date":"2023-10-27","proceeding":"ACMMM 2023 10","authors":["Jiancheng Pan","Qing Ma","Cong Bai"],"abstract":"This paper presents a prior instruction representation framework (PIR) for remote sensing image-text retrieval, aimed at remote sensing vision-language understanding tasks to solve the semantic noise problem. Our highlight is the proposal of a paradigm that draws on prior knowledge to instruct adaptive learning of vision and text representations. Concretely, two progressive attention encoder (PAE) structures, Spatial-PAE and Temporal-PAE, are proposed to perform long-range dependency modeling to enhance key feature representation. In vision representation, Vision Instruction Representation (VIR) based on Spatial-PAE exploits the prior-guided knowledge of the remote sensing scene recognition by building a belief matrix to select key features for reducing the impact of semantic noise. In text representation, Language Cycle Attention (LCA) based on Temporal-PAE uses the previous time step to cyclically activate the current time step to enhance text representation capability. A cluster-wise affiliation loss is proposed to constrain the inter-classes and to reduce the semantic confusion zones in the common subspace. Comprehensive experiments demonstrate that using prior knowledge instruction could enhance vision and text representations and could outperform the state-of-the-art methods on two benchmark datasets, RSICD and RSITMD.","url_abs":"https://dl.acm.org/doi/10.1145/3581783.3612374","url_pdf":"https://dl.acm.org/doi/10.1145/3581783.3612374","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-prior-instruction-representation-framework","repo_url":"https://github.com/jaychempan/PIR","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"cross-modal-retrieval","task_name":"Cross-Modal Retrieval"},{"task_slug":"image-text-retrieval","task_name":"Image-text Retrieval"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"scene-recognition","task_name":"Scene Recognition"},{"task_slug":"text-retrieval","task_name":"Text Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/cross-modal-retrieval-on-rsicd","task":"Cross-Modal Retrieval","dataset":"RSICD","model":"PIR","rank_in_archive_order":6,"of":10,"metrics":{"Image-to-text R@1":"9.88%","Mean Recall":"24.46%","text-to-image R@1":"6.97%"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-on-rsitmd","task":"Cross-Modal Retrieval","dataset":"RSITMD","model":"PIR","rank_in_archive_order":6,"of":10,"metrics":{"Image-to-text R@1":"18.14%","Mean Recall":"38.24%","text-to-imageR@1":"12.17%"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}