{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-attention-a-better-building-block-for","title":"Self-Attention: A Better Building Block for Sentiment Analysis Neural Network Classifiers","arxiv_id":"1812.07860","date":"2018-12-19","proceeding":"WS 2018 10","authors":["Artaches Ambartsoumian","Fred Popowich"],"abstract":"Sentiment Analysis has seen much progress in the past two decades. For the\npast few years, neural network approaches, primarily RNNs and CNNs, have been\nthe most successful for this task. Recently, a new category of neural networks,\nself-attention networks (SANs), have been created which utilizes the attention\nmechanism as the basic building block. Self-attention networks have been shown\nto be effective for sequence modeling tasks, while having no recurrence or\nconvolutions. In this work we explore the effectiveness of the SANs for\nsentiment analysis. We demonstrate that SANs are superior in performance to\ntheir RNN and CNN counterparts by comparing their classification accuracy on\nsix datasets as well as their model characteristics such as training speed and\nmemory consumption. Finally, we explore the effects of various SAN\nmodifications such as multi-head attention as well as two methods of\nincorporating sequence position information into SANs.","url_abs":"http://arxiv.org/abs/1812.07860v1","url_pdf":"http://arxiv.org/pdf/1812.07860v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-attention-a-better-building-block-for","repo_url":"https://github.com/Artaches/SSAN-self-attention-sentiment-analysis-classification","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"speed","method_name":"SPEED"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}