Papers › Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling...

Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms

24 May 2018ACL 2018 7arXiv:1805.09843archive 2025-07-28

Dinghan Shen, Guoyin Wang, Wenlin Wang, Martin Renqiang Min, Qinliang Su, Yizhe Zhang, Chunyuan Li, Ricardo Henao, Lawrence Carin

Many deep learning architectures have been proposed to model the compositionality in text sequences, requiring a substantial number of parameters and expensive computations. However, there has not been a rigorous evaluation regarding the added value of sophisticated compositional functions. In this paper, we conduct a point-by-point comparative study between Simple Word-Embedding-based Models (SWEMs), consisting of parameter-free pooling operations, relative to word-embedding-based RNN/CNN models. Surprisingly, SWEMs exhibit comparable or even superior performance in the majority of cases considered. Based upon this understanding, we propose two additional pooling strategies over learned word embeddings: (i) a max-pooling operation for improved interpretability; and (ii) a hierarchical pooling operation, which preserves spatial (n-gram) information within text sequences. We present experiments on 17 datasets encompassing three tasks: (i) (long) document classification; (ii) text sequence matching; and (iii) short text tasks, including classification and tagging. The source code and datasets can be obtained from https:// github.com/dinghanshen/SWEM.

PaperPDFConference PDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

dinghanshen/SWEM officialmentioned in papermentioned on GitHubtf report
nyk510/scdv-python mentioned on GitHub report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Document ClassificationGeneral ClassificationNamed Entity Recognition (NER)Sentiment AnalysisSubjectivity AnalysisText ClassificationWord Embeddings

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Named Entity Recognition (NER) CoNLL 2000 SWEM-CRF F1 90.34 #1 of 1 Archive leaderboard report
Named Entity Recognition (NER) CoNLL 2003 (English) SWEM-CRF F1 86.28 #72 of 73 Archive leaderboard report
Natural Language Inference MultiNLI SWEM-max Matched 68.2 #54 of 67 Archive leaderboard report
Natural Language Inference MultiNLI SWEM-max Mismatched 67.7 #54 of 67 Archive leaderboard report
Natural Language Inference SNLI SWEM-max % Test Accuracy 83.8 #82 of 98 Archive leaderboard report
Paraphrase Identification MSRP SWEM-concat Accuracy 71.5 #3 of 3 Archive leaderboard report
Paraphrase Identification MSRP SWEM-concat F1 81.3 #3 of 3 Archive leaderboard report
Question Answering Quora Question Pairs SWEM-concat Accuracy 83.03% #17 of 19 Archive leaderboard report
Question Answering WikiQA SWEM-concat MAP 0.6788 #19 of 25 Archive leaderboard report
Question Answering WikiQA SWEM-concat MRR 0.6908 #19 of 25 Archive leaderboard report
Sentiment Analysis MR SWEM-concat Accuracy 78.2 #12 of 19 Archive leaderboard report
Sentiment Analysis SST-2 Binary classification SWEM-concat Accuracy 84.3 #78 of 87 Archive leaderboard report
Sentiment Analysis SST-5 Fine-grained classification SWEM-concat Accuracy 46.1 #24 of 31 Archive leaderboard report
Sentiment Analysis Yelp Binary classification SWEM-hier Error 4.19 #16 of 20 Archive leaderboard report
Sentiment Analysis Yelp Fine-grained classification SWEM-hier Error 36.21 #15 of 17 Archive leaderboard report
Subjectivity Analysis SUBJ SWEM-concat Accuracy 93 #13 of 19 Archive leaderboard report
Text Classification AG News SWEM-concat Error 7.34 #11 of 24 Archive leaderboard report
Text Classification DBpedia SWEM-concat Error 1.43 #19 of 21 Archive leaderboard report
Text Classification TREC-6 SWEM-aver Error 7.8 #16 of 19 Archive leaderboard report
Text Classification Yahoo! Answers SWEM-concat Accuracy 73.53 #8 of 10 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections