{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/supervised-and-semi-supervised-text","title":"Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings","arxiv_id":"1602.02373","date":"2016-02-07","proceeding":null,"authors":["Rie Johnson","Tong Zhang"],"abstract":"One-hot CNN (convolutional neural network) has been shown to be effective for\ntext categorization (Johnson & Zhang, 2015). We view it as a special case of a\ngeneral framework which jointly trains a linear model with a non-linear feature\ngenerator consisting of `text region embedding + pooling'. Under this\nframework, we explore a more sophisticated region embedding method using Long\nShort-Term Memory (LSTM). LSTM can embed text regions of variable (and possibly\nlarge) sizes, whereas the region size needs to be fixed in a CNN. We seek\neffective and efficient use of LSTM for this purpose in the supervised and\nsemi-supervised settings. The best results were obtained by combining region\nembeddings in the form of LSTM and convolution layers trained on unlabeled\ndata. The results indicate that on this task, embeddings of text regions, which\ncan convey complex concepts, are more useful than embeddings of single words in\nisolation. We report performances exceeding the previous best results on four\nbenchmark datasets.","url_abs":"http://arxiv.org/abs/1602.02373v2","url_pdf":"http://arxiv.org/pdf/1602.02373v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"text-categorization","task_name":"Text Categorization"},{"task_slug":"text-classification","task_name":"Text Classification"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sentiment-analysis-on-imdb","task":"Sentiment Analysis","dataset":"IMDb","model":"oh-LSTM","rank_in_archive_order":23,"of":49,"metrics":{"Accuracy":"94.1"},"uses_additional_data":true},{"leaderboard":"/sota/sentiment-analysis-on-yelp-binary","task":"Sentiment Analysis","dataset":"Yelp Binary classification","model":"CNN","rank_in_archive_order":10,"of":20,"metrics":{"Error":"2.9"},"uses_additional_data":false},{"leaderboard":"/sota/sentiment-analysis-on-yelp-fine-grained","task":"Sentiment Analysis","dataset":"Yelp Fine-grained classification","model":"CNN","rank_in_archive_order":9,"of":17,"metrics":{"Error":"32.39"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-ag-news","task":"Text Classification","dataset":"AG News","model":"CNN","rank_in_archive_order":8,"of":24,"metrics":{"Error":"6.57"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-dbpedia","task":"Text Classification","dataset":"DBpedia","model":"CNN","rank_in_archive_order":9,"of":21,"metrics":{"Error":"0.84"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-rcv1","task":"Text Classification","dataset":"RCV1","model":"oh-CNN + two LSTM tv-embed.","rank_in_archive_order":1,"of":4,"metrics":{"Accuracy":"92.85"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1602.02373","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}