AfroLM

AfroLM is a self-active learning-based multilingual pre-trained language model for 23 African languages, presented at EMNLP 2022 (third SustainNLP workshop) with the arXiv paper published in November 2022. It was trained from scratch on Amharic, Hausa, Igbo, Yoruba, Swahili, Zulu, Xhosa and 16 more African languages.

Its novelty is data efficiency: it matches or beats AfriBERTa, mBERT and XLMR-base while using a training dataset more than 14 times smaller, thanks to an active-learning loop that selects the most informative data. The code and models are publicly released.