Arabic Syntactic Diacritics Restoration Using BERT Models

Figure 2

The BERT base tagger has 12 layers of transformer encoder layer and a token classification layer to predict the syntactic diacritics tags.