Papers › Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings
Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings
Rie Johnson, Tong Zhang
One-hot CNN (convolutional neural network) has been shown to be effective for text categorization (Johnson & Zhang, 2015). We view it as a special case of a general framework which jointly trains a linear model with a non-linear feature generator consisting of `text region embedding + pooling'. Under this framework, we explore a more sophisticated region embedding method using Long Short-Term Memory (LSTM). LSTM can embed text regions of variable (and possibly large) sizes, whereas the region size needs to be fixed in a CNN. We seek effective and efficient use of LSTM for this purpose in the supervised and semi-supervised settings. The best results were obtained by combining region embeddings in the form of LSTM and convolution layers trained on unlabeled data. The results indicate that on this task, embeddings of text regions, which can convey complex concepts, are more useful than embeddings of single words in isolation. We report performances exceeding the previous best results on four benchmark datasets.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Sentiment Analysis | IMDb | oh-LSTM | Accuracy | 94.1 | #23 of 49 | Archive leaderboard | report |
| Sentiment Analysis | Yelp Binary classification | CNN | Error | 2.9 | #10 of 20 | Archive leaderboard | report |
| Sentiment Analysis | Yelp Fine-grained classification | CNN | Error | 32.39 | #9 of 17 | Archive leaderboard | report |
| Text Classification | AG News | CNN | Error | 6.57 | #8 of 24 | Archive leaderboard | report |
| Text Classification | DBpedia | CNN | Error | 0.84 | #9 of 21 | Archive leaderboard | report |
| Text Classification | RCV1 | oh-CNN + two LSTM tv-embed. | Accuracy | 92.85 | #1 of 4 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections