Papers › Leveraging ParsBERT and Pretrained mT5 for Persian Abstractive Text Summarization

Leveraging ParsBERT and Pretrained mT5 for Persian Abstractive Text Summarization

21 Dec 2020arXiv:2012.11204archive 2025-07-28

Mehrdad Farahani, Mohammad Gharachorloo, Mohammad Manthouri

Text summarization is one of the most critical Natural Language Processing (NLP) tasks. More and more researches are conducted in this field every day. Pre-trained transformer-based encoder-decoder models have begun to gain popularity for these tasks. This paper proposes two methods to address this task and introduces a novel dataset named pn-summary for Persian abstractive text summarization. The models employed in this paper are mT5 and an encoder-decoder version of the ParsBERT model (i.e., a monolingual BERT model for Persian). These models are fine-tuned on the pn-summary dataset. The current work is the first of its kind and, by achieving promising results, can serve as a baseline for any future work.

PaperPDFCode

In Syntology View this paper on Syntology: its repositories, every harvested function with whether it ran, its licence and the call to fetch it.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

hooshvare/pn-summary officialmentioned in papermentioned on GitHub report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Abstractive Text SummarizationDecoderText Summarization

Datasets

Introduced by this paper, per the archive.

pn-summary

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

AdafactorAdamAttentionAttention DropoutBERTBPEDense ConnectionsDropoutGated Linear UnitInverse Square Root ScheduleLayer NormalizationLinear LayerLinear Warmup With Linear DecayMulti-Head AttentionResidual ConnectionSentencePieceSoftmaxT5Weight DecayWordPiecemT5

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections