Papers › CATT: Character-based Arabic Tashkeel Transformer

CATT: Character-based Arabic Tashkeel Transformer

3 Jul 2024arXiv:2407.03236archive 2025-07-28

Faris Alasmary, Orjuwan Zaafarani, Ahmad Ghannam

Tashkeel, or Arabic Text Diacritization (ATD), greatly enhances the comprehension of Arabic text by removing ambiguity and minimizing the risk of misinterpretations caused by its absence. It plays a crucial role in improving Arabic text processing, particularly in applications such as text-to-speech and machine translation. This paper introduces a new approach to training ATD models. First, we finetuned two transformers, encoder-only and encoder-decoder, that were initialized from a pretrained character-based BERT. Then, we applied the Noisy-Student approach to boost the performance of the best model. We evaluated our models alongside 11 commercial and open-source models using two manually labeled benchmark datasets: WikiNews and our CATT dataset. Our findings show that our top model surpasses all evaluated models by relative Diacritic Error Rates (DERs) of 30.83\% and 35.21\% on WikiNews and CATT, respectively, achieving state-of-the-art in ATD. In addition, we show that our model outperforms GPT-4-turbo on CATT dataset by a relative DER of 9.36\%. We open-source our CATT models and benchmark dataset for the research community\footnote{https://github.com/abjadai/catt}.

PaperPDFCode

Code

abjadai/catt officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Arabic Text DiacritizationDecoderMachine TranslationText to Speechtext-to-speech

Datasets

Introduced by this paper, per the archive.

CATT

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Arabic Text Diacritization CATT CATT ED DER(%) 8.624 #1 of 12 Archive leaderboard report
Arabic Text Diacritization CATT CATT ED WER (%) 34.191 #1 of 12 Archive leaderboard report
Arabic Text Diacritization CATT CATT EO DER(%) 8.762 #2 of 12 Archive leaderboard report
Arabic Text Diacritization CATT CATT EO WER (%) 35.597 #2 of 12 Archive leaderboard report
Arabic Text Diacritization CATT GPT-4 DER(%) 9.515 #3 of 12 Archive leaderboard report
Arabic Text Diacritization CATT GPT-4 WER (%) 38.311 #3 of 12 Archive leaderboard report
Arabic Text Diacritization CATT CBHG DER(%) 10.808 #4 of 12 Archive leaderboard report
Arabic Text Diacritization CATT CBHG WER (%) 42.680 #4 of 12 Archive leaderboard report
Arabic Text Diacritization CATT Command R+ DER(%) 13.169 #5 of 12 Archive leaderboard report
Arabic Text Diacritization CATT Command R+ WER (%) 48.518 #5 of 12 Archive leaderboard report
Arabic Text Diacritization CATT Shakkala DER(%) 13.494 #7 of 12 Archive leaderboard report
Arabic Text Diacritization CATT Shakkala WER (%) 50.387 #7 of 12 Archive leaderboard report
Arabic Text Diacritization CATT Sakhr DER(%) 13.841 #8 of 12 Archive leaderboard report
Arabic Text Diacritization CATT Sakhr WER (%) 56.661 #8 of 12 Archive leaderboard report
Arabic Text Diacritization CATT Alkhalil DER(%) 14.232 #9 of 12 Archive leaderboard report
Arabic Text Diacritization CATT Alkhalil WER (%) 53.413 #9 of 12 Archive leaderboard report
Arabic Text Diacritization CATT Multilevel Diacritizer DER(%) 16.482 #10 of 12 Archive leaderboard report
Arabic Text Diacritization CATT Multilevel Diacritizer WER (%) 60.844 #10 of 12 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

AdamAttentionAttention DropoutBERTDense ConnectionsDropoutLayer NormalizationLinear LayerLinear Warmup With Linear DecayMulti-Head AttentionResidual ConnectionSoftmaxWeight DecayWordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections