{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/natural-language-object-retrieval","title":"Natural Language Object Retrieval","arxiv_id":"1511.04164","date":"2015-11-13","proceeding":"CVPR 2016 6","authors":["Ronghang Hu","Huazhe Xu","Marcus Rohrbach","Jiashi Feng","Kate Saenko","Trevor Darrell"],"abstract":"In this paper, we address the task of natural language object retrieval, to\nlocalize a target object within a given image based on a natural language query\nof the object. Natural language object retrieval differs from text-based image\nretrieval task as it involves spatial information about objects within the\nscene and global scene context. To address this issue, we propose a novel\nSpatial Context Recurrent ConvNet (SCRC) model as scoring function on candidate\nboxes for object retrieval, integrating spatial configurations and global\nscene-level contextual information into the network. Our model processes query\ntext, local image descriptors, spatial configurations and global context\nfeatures through a recurrent network, outputs the probability of the query text\nconditioned on each candidate box as a score for the box, and can transfer\nvisual-linguistic knowledge from image captioning domain to our task.\nExperimental results demonstrate that our method effectively utilizes both\nlocal and global information, outperforming previous baseline methods\nsignificantly on different datasets and scenarios, and can exploit large scale\nvision and language datasets for knowledge transfer.","url_abs":"http://arxiv.org/abs/1511.04164v3","url_pdf":"http://arxiv.org/pdf/1511.04164v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"natural-language-object-retrieval","repo_url":"https://github.com/ronghanghu/natural-language-object-retrieval","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"caffe2","reach":null}],"tasks":[{"task_slug":"image-captioning","task_name":"Image Captioning"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"object","task_name":"Object"},{"task_slug":"phrase-grounding","task_name":"Phrase Grounding"},{"task_slug":"referring-expression-comprehension","task_name":"Referring Expression Comprehension"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/phrase-grounding-on-flickr30k-entities-test","task":"Phrase Grounding","dataset":"Flickr30k Entities Test","model":"SCRC","rank_in_archive_order":17,"of":18,"metrics":{"R@1":"27.8","R@10":"62.9"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1511.04164","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1511.04164"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ronghanghu/natural-language-object-retrieval","reach":null}],"summary":{"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"listed":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"19e214c28fe2f67d","entry":"build_vocab_dict_from_captioner","repo":"ronghanghu/natural-language-object-retrieval","repo_kind":"listed","path":"retriever.py","file_url":"https://github.com/ronghanghu/natural-language-object-retrieval/blob/HEAD/retriever.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"19e214c28fe2f67d"}},{"code_sha256_prefix":"c8b05034710bc9bb","entry":"build_vocab_dict_from_file","repo":"ronghanghu/natural-language-object-retrieval","repo_kind":"listed","path":"retriever.py","file_url":"https://github.com/ronghanghu/natural-language-object-retrieval/blob/HEAD/retriever.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c8b05034710bc9bb"}},{"code_sha256_prefix":"4f7c8255650ece96","entry":"sentence2vocab_indices","repo":"ronghanghu/natural-language-object-retrieval","repo_kind":"listed","path":"retriever.py","file_url":"https://github.com/ronghanghu/natural-language-object-retrieval/blob/HEAD/retriever.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4f7c8255650ece96"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}