{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improved-sentence-modeling-using-suffix","title":"Improved Sentence Modeling using Suffix Bidirectional LSTM","arxiv_id":"1805.07340","date":"2018-05-18","proceeding":null,"authors":["Siddhartha Brahma"],"abstract":"Recurrent neural networks have become ubiquitous in computing representations\nof sequential data, especially textual data in natural language processing. In\nparticular, Bidirectional LSTMs are at the heart of several neural models\nachieving state-of-the-art performance in a wide variety of tasks in NLP.\nHowever, BiLSTMs are known to suffer from sequential bias - the contextual\nrepresentation of a token is heavily influenced by tokens close to it in a\nsentence. We propose a general and effective improvement to the BiLSTM model\nwhich encodes each suffix and prefix of a sequence of tokens in both forward\nand reverse directions. We call our model Suffix Bidirectional LSTM or\nSuBiLSTM. This introduces an alternate bias that favors long range\ndependencies. We apply SuBiLSTMs to several tasks that require sentence\nmodeling. We demonstrate that using SuBiLSTM instead of a BiLSTM in existing\nmodels leads to improvements in performance in learning general sentence\nrepresentations, text classification, textual entailment and paraphrase\ndetection. Using SuBiLSTM we achieve new state-of-the-art results for\nfine-grained sentiment classification and question classification.","url_abs":"http://arxiv.org/abs/1805.07340v2","url_pdf":"http://arxiv.org/pdf/1805.07340v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"sentiment-classification","task_name":"Sentiment Classification"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[{"method_slug":"bilstm","method_name":"BiLSTM"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sentiment-analysis-on-cr","task":"Sentiment Analysis","dataset":"CR","model":"SuBiLSTM-Tied","rank_in_archive_order":7,"of":9,"metrics":{"Accuracy":"86.5"},"uses_additional_data":false},{"leaderboard":"/sota/sentiment-analysis-on-mr","task":"Sentiment Analysis","dataset":"MR","model":"SuBiLSTM-Tied","rank_in_archive_order":8,"of":19,"metrics":{"Accuracy":"81.6"},"uses_additional_data":false},{"leaderboard":"/sota/sentiment-analysis-on-sst-2-binary","task":"Sentiment Analysis","dataset":"SST-2 Binary classification","model":"Suffix BiLSTM","rank_in_archive_order":57,"of":87,"metrics":{"Accuracy":"91.2"},"uses_additional_data":false},{"leaderboard":"/sota/sentiment-analysis-on-sst-5-fine-grained","task":"Sentiment Analysis","dataset":"SST-5 Fine-grained classification","model":"BCN+Suffix BiLSTM-Tied+CoVe","rank_in_archive_order":5,"of":31,"metrics":{"Accuracy":"56.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1805.07340","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}