{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/image-captioning-as-neural-machine","title":"Image Captioning as Neural Machine Translation Task in SOCKEYE","arxiv_id":"1810.04101","date":"2018-10-09","proceeding":null,"authors":["Loris Bazzani","Tobias Domhan","Felix Hieber"],"abstract":"Image captioning is an interdisciplinary research problem that stands between\ncomputer vision and natural language processing. The task is to generate a\ntextual description of the content of an image. The typical model used for\nimage captioning is an encoder-decoder deep network, where the encoder captures\nthe essence of an image while the decoder is responsible for generating a\nsentence describing the image. Attention mechanisms can be used to\nautomatically focus the decoder on parts of the image which are relevant to\npredict the next word. In this paper, we explore different decoders and\nattentional models popular in neural machine translation, namely attentional\nrecurrent neural networks, self-attentional transformers, and\nfully-convolutional networks, which represent the current state of the art of\nneural machine translation. The image captioning module is available as part of\nSOCKEYE at https://github.com/awslabs/sockeye which tutorial can be found at\nhttps://awslabs.github.io/sockeye/image_captioning.html .","url_abs":"http://arxiv.org/abs/1810.04101v3","url_pdf":"http://arxiv.org/pdf/1810.04101v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"image-captioning-as-neural-machine","repo_url":"https://github.com/awslabs/sockeye","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"mxnet","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"image-captioning","task_name":"Image Captioning"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}