Papers › Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture...
Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Data
Wei Zhao, Liang Wang, Kewei Shen, Ruoyu Jia, Jingming Liu
Neural machine translation systems have become state-of-the-art approaches for Grammatical Error Correction (GEC) task. In this paper, we propose a copy-augmented architecture for the GEC task by copying the unchanged words from the source sentence to the target sentence. Since the GEC suffers from not having enough labeled training data to achieve high accuracy. We pre-train the copy-augmented architecture with a denoising auto-encoder using the unlabeled One Billion Benchmark and make comparisons between the fully pre-trained model and a partially pre-trained model. It is the first time copying words from the source context and fully pre-training a sequence to sequence model are experimented on the GEC task. Moreover, We add token-level and sentence-level multi-task learning for the GEC task. The evaluation results on the CoNLL-2014 test set show that our approach outperforms all recently published state-of-the-art results by a large margin. The code and pre-trained models are released at https://github.com/zhawe01/fairseq-gec.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Grammatical Error Correction | CoNLL-2014 Shared Task | Copy-augmented Model (4 Ensemble +Denoising Autoencoder) | F0.5 | 61.15 | #18 of 23 | Archive leaderboard | report |
| Grammatical Error Correction | CoNLL-2014 Shared Task | Copy-augmented Model (4 Ensemble +Denoising Autoencoder) | Precision | 71.57 | #18 of 23 | Archive leaderboard | report |
| Grammatical Error Correction | CoNLL-2014 Shared Task | Copy-augmented Model (4 Ensemble +Denoising Autoencoder) | Recall | 38.65 | #18 of 23 | Archive leaderboard | report |
| Grammatical Error Correction | JFLEG | Copy-augmented Model (4 Ensemble +Denoising Autoencoder) | GLEU | 61.0 | #4 of 6 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections