Papers › Cell-aware Stacked LSTMs for Modeling Sentences

Cell-aware Stacked LSTMs for Modeling Sentences

7 Sep 2018arXiv:1809.02279archive 2025-07-28

Jihun Choi, Taeuk Kim, Sang-goo Lee

We propose a method of stacking multiple long short-term memory (LSTM) layers for modeling sentences. In contrast to the conventional stacked LSTMs where only hidden states are fed as input to the next layer, the suggested architecture accepts both hidden and memory cell states of the preceding layer and fuses information from the left and the lower context using the soft gating mechanism of LSTMs. Thus the architecture modulates the amount of information to be delivered not only in horizontal recurrence but also in vertical connections, from which useful features extracted from lower layers are effectively conveyed to upper layers. We dub this architecture Cell-aware Stacked LSTM (CAS-LSTM) and show from experiments that our models bring significant performance gain over the standard LSTMs on benchmark datasets for natural language inference, paraphrase detection, sentiment classification, and machine translation. We also conduct extensive qualitative analysis to understand the internal behavior of the suggested approach.

Paper PDF

Code

No code repository is listed for this paper in the archive or in Syntology's graph.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Machine Translation Natural Language Inference Paraphrase Identification Sentiment Analysis Sentiment Classification Translation

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Natural Language Inference	SNLI	300D 2-layer Bi-CAS-LSTM	% Test Accuracy	87	#47 of 98	Archive leaderboard	report
Paraphrase Identification	Quora Question Pairs	Bi-CAS-LSTM	Accuracy	88.6	#22 of 31	Archive leaderboard	report
Sentiment Analysis	SST-2 Binary classification	Bi-CAS-LSTM	Accuracy	91.3	#54 of 87	Archive leaderboard	report
Sentiment Analysis	SST-5 Fine-grained classification	Bi-CAS-LSTM	Accuracy	53.6	#11 of 31	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

LSTM Sigmoid Activation Tanh Activation

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections