{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/flickr30k-entities-collecting-region-to","title":"Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models","arxiv_id":"1505.04870","date":"2015-05-19","proceeding":"ICCV 2015 12","authors":["Bryan A. Plummer","Li-Wei Wang","Chris M. Cervantes","Juan C. Caicedo","Julia Hockenmaier","Svetlana Lazebnik"],"abstract":"The Flickr30k dataset has become a standard benchmark for sentence-based\nimage description. This paper presents Flickr30k Entities, which augments the\n158k captions from Flickr30k with 244k coreference chains, linking mentions of\nthe same entities across different captions for the same image, and associating\nthem with 276k manually annotated bounding boxes. Such annotations are\nessential for continued progress in automatic image description and grounded\nlanguage understanding. They enable us to define a new benchmark for\nlocalization of textual entity mentions in an image. We present a strong\nbaseline for this task that combines an image-text embedding, detectors for\ncommon objects, a color classifier, and a bias towards selecting larger\nobjects. While our baseline rivals in accuracy more complex state-of-the-art\nmodels, we show that its gains cannot be easily parlayed into improvements on\nsuch tasks as image-sentence retrieval, thus underlining the limitations of\ncurrent methods and the need for further research.","url_abs":"http://arxiv.org/abs/1505.04870v4","url_pdf":"http://arxiv.org/pdf/1505.04870v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"flickr30k-entities-collecting-region-to","repo_url":"https://github.com/bryanplummer/flickr30k_entities","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"flickr30k-entities-collecting-region-to","repo_url":"https://github.com/zhanyang-nwpu/rsvg-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":null,"task_name":"Image Description"},{"task_slug":"phrase-grounding","task_name":"Phrase Grounding"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-retrieval","task_name":"Sentence Retrieval"}],"methods":[],"datasets_introduced":[{"slug":"flickr30k-entities","name":"Flickr30K Entities","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-retrieval-on-flickr30k-1k-test","task":"Image Retrieval","dataset":"Flickr30K 1K test","model":"HGLMM FV","rank_in_archive_order":17,"of":18,"metrics":{"R@1":"24.7","R@10":"66.8","R@5":"53.4"},"uses_additional_data":false},{"leaderboard":"/sota/phrase-grounding-on-flickr30k-entities-test","task":"Phrase Grounding","dataset":"Flickr30k Entities Test","model":"CCA - Fast RCNN","rank_in_archive_order":15,"of":18,"metrics":{"R@1":"41.77","R@10":"70.77","R@5":"64.52"},"uses_additional_data":false},{"leaderboard":"/sota/phrase-grounding-on-flickr30k-entities-test","task":"Phrase Grounding","dataset":"Flickr30k Entities Test","model":"CCA - VGG19","rank_in_archive_order":16,"of":18,"metrics":{"R@1":"30.83","R@10":"67.15","R@5":"58.01"},"uses_additional_data":false},{"leaderboard":"/sota/phrase-grounding-on-flickr30k-entities-test","task":"Phrase Grounding","dataset":"Flickr30k Entities Test","model":"CCA","rank_in_archive_order":18,"of":18,"metrics":{"R@1":"25.30","R@10":"59.66"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1505.04870","atlas_url":"https://app.syntology.ai/?focus=1505.04870","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1505.04870"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bryanplummer/flickr30k_entities","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhanyang-nwpu/rsvg-pytorch","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":2},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"66a196e3cfa1ebe6","entry":"get_annotations","repo":"bryanplummer/flickr30k_entities","repo_kind":"listed","path":"flickr30k_entities_utils.py","file_url":"https://github.com/bryanplummer/flickr30k_entities/blob/HEAD/flickr30k_entities_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"66a196e3cfa1ebe6"}},{"code_sha256_prefix":"c1b24fe52cde4f1c","entry":"get_sentence_data","repo":"bryanplummer/flickr30k_entities","repo_kind":"listed","path":"flickr30k_entities_utils.py","file_url":"https://github.com/bryanplummer/flickr30k_entities/blob/HEAD/flickr30k_entities_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c1b24fe52cde4f1c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}