Papers › Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers...

Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets

30 Nov 2022arXiv:2211.16878archive 2025-07-28

Fabian Karl, Ansgar Scherp

Short text classification is a crucial and challenging aspect of Natural Language Processing. For this reason, there are numerous highly specialized short text classifiers. However, in recent short text research, State of the Art (SOTA) methods for traditional text classification, particularly the pure use of Transformers, have been unexploited. In this work, we examine the performance of a variety of short text classifiers as well as the top performing traditional text classifier. We further investigate the effects on two new real-world short text datasets in an effort to address the issue of becoming overly dependent on benchmark datasets with a limited number of characteristics. Our experiments unambiguously demonstrate that Transformers achieve SOTA accuracy on short text classification tasks, raising the question of whether specialized short text techniques are necessary.

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

FKarl/short-text-classification officialmentioned in paperpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

ClassificationText Classificationtext-classification

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Text Classification MR DeBERTa Accuracy 90.21 #2 of 10 Archive leaderboard report
Text Classification MR ERNIE 2.0 (optimized) Accuracy 89.53 #4 of 10 Archive leaderboard report
Text Classification MR RoBERTa Accuracy 89.42 #5 of 10 Archive leaderboard report
Text Classification MR ERNIE 2.0 Accuracy 88.97 #6 of 10 Archive leaderboard report
Text Classification MR BERT Accuracy 86.94 #7 of 10 Archive leaderboard report
Text Classification MR ALBERTv2 Accuracy 86.02 #8 of 10 Archive leaderboard report
Text Classification MR DistilBERT Accuracy 85.31 #9 of 10 Archive leaderboard report
Text Classification NICE-2 RoBERTa Accuracy 99.76 #1 of 1 Archive leaderboard report
Text Classification NICE-45 BERT Accuracy 72.79 #1 of 1 Archive leaderboard report
Text Classification R8 DeBERTa Accuracy 98.451 #1 of 21 Archive leaderboard report
Text Classification R8 C-BERT (ESGNN + BERT) Accuracy 98.28 #2 of 21 Archive leaderboard report
Text Classification R8 ESGNN Accuracy 98.23 #3 of 21 Archive leaderboard report
Text Classification R8 BERT Accuracy 98.171 #5 of 21 Archive leaderboard report
Text Classification R8 SGNN Accuracy 98.09 #6 of 21 Archive leaderboard report
Text Classification R8 ERNIE 2.0 Accuracy 98.041 #7 of 21 Archive leaderboard report
Text Classification R8 DistilBERT Accuracy 97.981 #8 of 21 Archive leaderboard report
Text Classification R8 ALBERTv2 Accuracy 97.62 #10 of 21 Archive leaderboard report
Text Classification R8 WideMLP Accuracy 96.98 #18 of 21 Archive leaderboard report
Text Classification R8 fastText Accuracy 96.13 #21 of 21 Archive leaderboard report
Text Classification SST-2 DeBERTa Accuracy 94.78 #1 of 2 Archive leaderboard report
Text Classification SST-2 BERT Accuracy 91.37 #2 of 2 Archive leaderboard report
Text Classification STOPS-2 ERNIE 2.0 STOPS-2 99.88 #1 of 1 Archive leaderboard report
Text Classification STOPS-41 DeBERTa Accuracy 89.73 #1 of 1 Archive leaderboard report
Text Classification Searchsnippets DistilBERT Accuracy 89.69 #1 of 2 Archive leaderboard report
Text Classification Searchsnippets BERT Accuracy 88.2 #2 of 2 Archive leaderboard report
Text Classification TREC-10 BERT Accuracy 99.40 #1 of 1 Archive leaderboard report
Text Classification Twitter ERNIE 2.0 Accuracy 99.97 #1 of 3 Archive leaderboard report
Text Classification Twitter BERT Accuracy 99.96 #2 of 3 Archive leaderboard report
Text Classification Twitter DistilBERT Accuracy 99.96 #3 of 3 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections