{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/imram-iterative-matching-with-recurrent","title":"IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrieval","arxiv_id":"2003.03772","date":"2020-03-08","proceeding":"CVPR 2020 6","authors":["Hui Chen","Guiguang Ding","Xudong Liu","Zijia Lin","Ji Liu","Jungong Han"],"abstract":"Enabling bi-directional retrieval of images and texts is important for understanding the correspondence between vision and language. Existing methods leverage the attention mechanism to explore such correspondence in a fine-grained manner. However, most of them consider all semantics equally and thus align them uniformly, regardless of their diverse complexities. In fact, semantics are diverse (i.e. involving different kinds of semantic concepts), and humans usually follow a latent structure to combine them into understandable languages. It may be difficult to optimally capture such sophisticated correspondences in existing methods. In this paper, to address such a deficiency, we propose an Iterative Matching with Recurrent Attention Memory (IMRAM) method, in which correspondences between images and texts are captured with multiple steps of alignments. Specifically, we introduce an iterative matching scheme to explore such fine-grained correspondence progressively. A memory distillation unit is used to refine alignment knowledge from early steps to later ones. Experiment results on three benchmark datasets, i.e. Flickr8K, Flickr30K, and MS COCO, show that our IMRAM achieves state-of-the-art performance, well demonstrating its effectiveness. Experiments on a practical business advertisement dataset, named \\Ads{}, further validates the applicability of our method in practical scenarios.","url_abs":"https://arxiv.org/abs/2003.03772v1","url_pdf":"https://arxiv.org/pdf/2003.03772v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"imram-iterative-matching-with-recurrent","repo_url":"https://github.com/HuiChen24/IMRAM","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"cross-modal-retrieval","task_name":"Cross-Modal Retrieval"},{"task_slug":"image-text-retrieval","task_name":"Image-text Retrieval"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"text-retrieval","task_name":"Text Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/cross-modal-retrieval-on-coco-2014","task":"Cross-Modal Retrieval","dataset":"COCO 2014","model":"IMRAM","rank_in_archive_order":31,"of":36,"metrics":{"Image-to-text R@1":"53.7","Image-to-text R@10":"91.0","Image-to-text R@5":"83.2","Text-to-image R@1":"39.7","Text-to-image R@10":"79.8","Text-to-image R@5":"69.1"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-on-flickr30k","task":"Cross-Modal Retrieval","dataset":"Flickr30k","model":"IMRAM","rank_in_archive_order":20,"of":27,"metrics":{"Image-to-text R@1":"74.1","Image-to-text R@10":"96.6","Image-to-text R@5":"93.0","Text-to-image R@1":"53.9","Text-to-image R@10":"87.2","Text-to-image R@5":"79.4"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2003.03772","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.03772"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HuiChen24/IMRAM","reach":null}],"summary":{"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"7ff5ad918339d214","entry":"encode_data","repo":"HuiChen24/IMRAM","repo_kind":"official","path":"evaluation.py","file_url":"https://github.com/HuiChen24/IMRAM/blob/HEAD/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7ff5ad918339d214"}},{"code_sha256_prefix":"23f423f02c60ac8d","entry":"shard_xattn","repo":"HuiChen24/IMRAM","repo_kind":"official","path":"evaluation.py","file_url":"https://github.com/HuiChen24/IMRAM/blob/HEAD/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"23f423f02c60ac8d"}},{"code_sha256_prefix":"ecdf3a2569c46dbb","entry":"evalrank","repo":"HuiChen24/IMRAM","repo_kind":"official","path":"evaluation.py","file_url":"https://github.com/HuiChen24/IMRAM/blob/HEAD/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ecdf3a2569c46dbb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}