{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spice-semantic-propositional-image-caption","title":"SPICE: Semantic Propositional Image Caption Evaluation","arxiv_id":"1607.08822","date":"2016-07-29","proceeding":null,"authors":["Peter Anderson","Basura Fernando","Mark Johnson","Stephen Gould"],"abstract":"There is considerable interest in the task of automatically generating image\ncaptions. However, evaluation is challenging. Existing automatic evaluation\nmetrics are primarily sensitive to n-gram overlap, which is neither necessary\nnor sufficient for the task of simulating human judgment. We hypothesize that\nsemantic propositional content is an important component of human caption\nevaluation, and propose a new automated caption evaluation metric defined over\nscene graphs coined SPICE. Extensive evaluations across a range of models and\ndatasets indicate that SPICE captures human judgments over model-generated\ncaptions better than other automatic metrics (e.g., system-level correlation of\n0.88 with human judgments on the MS COCO dataset, versus 0.43 for CIDEr and\n0.53 for METEOR). Furthermore, SPICE can answer questions such as `which\ncaption-generator best understands colors?' and `can caption-generators count?'","url_abs":"http://arxiv.org/abs/1607.08822v1","url_pdf":"http://arxiv.org/pdf/1607.08822v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spice-semantic-propositional-image-caption","repo_url":"https://github.com/peteanderson80/SPICE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"AGPL-3.0"}},{"paper_slug":"spice-semantic-propositional-image-caption","repo_url":"https://github.com/QQQiwi/coursepaper-2year","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"spice-semantic-propositional-image-caption","repo_url":"https://github.com/gabrielsantosrv/coco-caption","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"spice-semantic-propositional-image-caption","repo_url":"https://github.com/jmhessel/pycocoevalcap","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"spice-semantic-propositional-image-caption","repo_url":"https://github.com/luoweizhou/coco-caption","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"spice-semantic-propositional-image-caption","repo_url":"https://github.com/lvapeab/coco-caption","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"spice-semantic-propositional-image-caption","repo_url":"https://github.com/mtanti/coco-caption","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"spice-semantic-propositional-image-caption","repo_url":"https://github.com/ruotianluo/coco-caption","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"spice-semantic-propositional-image-caption","repo_url":"https://github.com/salaniz/pycocoevalcap","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"spice-semantic-propositional-image-caption","repo_url":"https://github.com/shao-chi/ImageCaption","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"spice-semantic-propositional-image-caption","repo_url":"https://github.com/tylin/coco-caption","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"image-captioning","task_name":"Image Captioning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1607.08822","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}