Papers › On the Use of BERT for Automated Essay Scoring: Joint Learning of Multi-Scale Essay...

On the Use of BERT for Automated Essay Scoring: Joint Learning of Multi-Scale Essay Representation

8 May 2022NAACL 2022 7arXiv:2205.03835archive 2025-07-28

Yongjie Wang, Chuan Wang, Ruobing Li, Hui Lin

In recent years, pre-trained models have become dominant in most natural language processing (NLP) tasks. However, in the area of Automated Essay Scoring (AES), pre-trained models such as BERT have not been properly used to outperform other deep learning models such as LSTM. In this paper, we introduce a novel multi-scale essay representation for BERT that can be jointly learned. We also employ multiple losses and transfer learning from out-of-domain essays to further improve the performance. Experiment results show that our approach derives much benefit from joint learning of multi-scale essay representation and obtains almost the state-of-the-art result among all deep learning models in the ASAP task. Our multi-scale essay representation also generalizes well to CommonLit Readability Prize data set, which suggests that the novel text representation proposed in this paper may be a new and effective choice for long-text tasks.

Paper PDF Conference PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

lingochamp/multi-scale-bert-aes officialmentioned in paperpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Automated Essay Scoring Transfer Learning

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Automated Essay Scoring	ASAP-AES	Tran-BERT-MS-ML-R	Quadratic Weighted Kappa	0.791	#2 of 8	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Adam Attention Attention Dropout BERT Dense Connections Dropout LSTM Layer Normalization Linear Layer Linear Warmup With Linear Decay Multi-Head Attention Residual Connection Sigmoid Activation Softmax Tanh Activation Weight Decay WordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections