Papers › Enhancing Sentence Embedding with Generalized Pooling

Enhancing Sentence Embedding with Generalized Pooling

26 Jun 2018COLING 2018 8arXiv:1806.09828archive 2025-07-28

Qian Chen, Zhen-Hua Ling, Xiaodan Zhu

Pooling is an essential component of a wide variety of sentence representation and embedding models. This paper explores generalized pooling methods to enhance sentence embedding. We propose vector-based multi-head attention that includes the widely used max pooling, mean pooling, and scalar self-attention as special cases. The model benefits from properly designed penalization terms to reduce redundancy in multi-head attention. We evaluate the proposed model on three different tasks: natural language inference (NLI), author profiling, and sentiment classification. The experiments show that the proposed model achieves significant improvement over strong sentence-encoding-based methods, resulting in state-of-the-art performances on four datasets. The proposed approach can be easily implemented for more problems than we discuss in this paper.

Paper PDF Conference PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

lukecq1231/generalized-pooling officialmentioned in paper report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Author Profiling General Classification Natural Language Inference Sentence Sentence Embedding Sentence-Embedding Sentiment Analysis Sentiment Classification

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Natural Language Inference	SNLI	600D BiLSTM with generalized pooling	% Test Accuracy	86.6	#53 of 98	Archive leaderboard	report
Natural Language Inference	SNLI	600D BiLSTM with generalized pooling	% Train Accuracy	94.9	#53 of 98	Archive leaderboard	report
Natural Language Inference	SNLI	600D BiLSTM with generalized pooling	Parameters	65m	#53 of 98	Archive leaderboard	report
Sentiment Analysis	Yelp Fine-grained classification	BiLSTM generalized pooling	Error	33.45	#10 of 17	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections