{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multimodal-pivots-for-image-caption","title":"Multimodal Pivots for Image Caption Translation","arxiv_id":"1601.03916","date":"2016-01-15","proceeding":"ACL 2016 8","authors":["Julian Hitschler","Shigehiko Schamoni","Stefan Riezler"],"abstract":"We present an approach to improve statistical machine translation of image\ndescriptions by multimodal pivots defined in visual space. The key idea is to\nperform image retrieval over a database of images that are captioned in the\ntarget language, and use the captions of the most similar images for\ncrosslingual reranking of translation outputs. Our approach does not depend on\nthe availability of large amounts of in-domain parallel data, but only relies\non available large datasets of monolingually captioned images, and on\nstate-of-the-art convolutional neural networks to compute image similarities.\nOur experimental evaluation shows improvements of 1 BLEU point over strong\nbaselines.","url_abs":"http://arxiv.org/abs/1601.03916v3","url_pdf":"http://arxiv.org/pdf/1601.03916v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"reranking","task_name":"Reranking"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[{"slug":"decoco","name":"DeCOCO","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1601.03916","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}