Papers › ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation
ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation
Long Phan, Hieu Tran, Hieu Nguyen, Trieu H. Trinh
We present ViT5, a pretrained Transformer-based encoder-decoder model for the Vietnamese language. With T5-style self-supervised pretraining, ViT5 is trained on a large corpus of high-quality and diverse Vietnamese texts. We benchmark ViT5 on two downstream text generation tasks, Abstractive Text Summarization and Named Entity Recognition. Although Abstractive Text Summarization has been widely studied for the English language thanks to its rich and large source of data, there has been minimal research into the same task in Vietnamese, a much lower resource language. In this work, we perform exhaustive experiments on both Vietnamese Abstractive Summarization and Named Entity Recognition, validating the performance of ViT5 against many other pretrained Transformer-based encoder-decoder models. Our experiments show that ViT5 significantly outperforms existing models and achieves state-of-the-art results on Vietnamese Text Summarization. On the task of Named Entity Recognition, ViT5 is competitive against previous best results from pretrained encoder-based Transformer models. Further analysis shows the importance of context length during the self-supervised pretraining on downstream performance across different settings.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Abstractive Text Summarization | vietnews | ViT5 large | Rouge-1 | 63.37 | #2 of 7 | Archive leaderboard | report |
| Abstractive Text Summarization | vietnews | ViT5 large | Rouge-2 | 34.24 | #2 of 7 | Archive leaderboard | report |
| Abstractive Text Summarization | vietnews | ViT5 large | Rouge-L | 43.55 | #2 of 7 | Archive leaderboard | report |
| Abstractive Text Summarization | vietnews | ViT5 base | Rouge-1 | 62.77 | #3 of 7 | Archive leaderboard | report |
| Abstractive Text Summarization | vietnews | ViT5 base | Rouge-2 | 33.16 | #3 of 7 | Archive leaderboard | report |
| Abstractive Text Summarization | vietnews | ViT5 base | Rouge-L | 42.75 | #3 of 7 | Archive leaderboard | report |
| Named Entity Recognition In Vietnamese | PhoNER COVID19 | ViT5 base | F1 (%) | 94.5 | #1 of 3 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections