{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/aiding-intra-text-representations-with-visual","title":"Aiding Intra-Text Representations with Visual Context for Multimodal Named Entity Recognition","arxiv_id":"1904.01356","date":"2019-04-02","proceeding":null,"authors":["Omer Arshad","Ignazio Gallo","Shah Nawaz","Alessandro Calefati"],"abstract":"With massive explosion of social media such as Twitter and Instagram, people\ndaily share billions of multimedia posts, containing images and text.\nTypically, text in these posts is short, informal and noisy, leading to\nambiguities which can be resolved using images. In this paper we explore\ntext-centric Named Entity Recognition task on these multimedia posts. We\npropose an end to end model which learns a joint representation of a text and\nan image. Our model extends multi-dimensional self attention technique, where\nnow image helps to enhance relationship between words. Experiments show that\nour model is capable of capturing both textual and visual contexts with greater\naccuracy, achieving state-of-the-art results on Twitter multimodal Named Entity\nRecognition dataset.","url_abs":"http://arxiv.org/abs/1904.01356v1","url_pdf":"http://arxiv.org/pdf/1904.01356v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"aiding-intra-text-representations-with-visual","repo_url":"https://github.com/omerarshad/MultiModalNER","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"named-entity-recognition-1","task_name":"Named Entity Recognition"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"named-entity-recognition","task_name":"named-entity-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1904.01356","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}