Papers › BERT-Based Sentiment Analysis: A Software Engineering Perspective

BERT-Based Sentiment Analysis: A Software Engineering Perspective

4 Jun 2021arXiv:2106.02581archive 2025-07-28

Himanshu Batra, Narinder Singh Punn, Sanjay Kumar Sonbhadra, Sonali Agarwal

Sentiment analysis can provide a suitable lead for the tools used in software engineering along with the API recommendation systems and relevant libraries to be used. In this context, the existing tools like SentiCR, SentiStrength-SE, etc. exhibited low f1-scores that completely defeats the purpose of deployment of such strategies, thereby there is enough scope for performance improvement. Recent advancements show that transformer based pre-trained models (e.g., BERT, RoBERTa, ALBERT, etc.) have displayed better results in the text classification task. Following this context, the present research explores different BERT-based models to analyze the sentences in GitHub comments, Jira comments, and Stack Overflow posts. The paper presents three different strategies to analyse BERT based model for sentiment analysis, where in the first strategy the BERT based pre-trained models are fine-tuned; in the second strategy an ensemble model is developed from BERT variants, and in the third strategy a compressed model (Distil BERT) is used. The experimental results show that the BERT based ensemble approach and the compressed BERT model attain improvements by 6-12% over prevailing tools for the F1 measure on all three datasets.

Paper PDF Code

Code

Arnav382/sentiment-analysis mentioned on GitHubtf report

Arnav382/sentiment-analysis-of-Telecom-industry-of-India mentioned on GitHubtf report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Recommendation Systems Sentiment Analysis Text Classification text-classification

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

ALBERT Adam Attention Attention Dropout BERT Dense Connections Dropout LAMB Layer Normalization Linear Layer Linear Warmup With Linear Decay Multi-Head Attention Residual Connection RoBERTa Softmax Weight Decay WordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections