Published November 9, 2024 | Version v1
Other Open

AI Low-Resource Language Models for African Indigenous Languages: Bridging the Digital Divide Through Innovative AI Natural Language Processing Solutions.

Authors/Creators

Contributors

Contact person:

Data manager:

Description

This research investigates the development and optimization of natural language processing (NLP) models for African indigenous languages, addressing the critical digital divide that affects over 2 billion speakers across the continent. Through a comprehensive analysis of existing pre-trained language models and novel methodologies, this study examines the effectiveness of transformer-based architectures, specifically focusing on multilingual BERT variants, sentiment analysis systems, and speech recognition models for low-resource African languages. The research employs a design thinking framework to develop scalable solutions that enhance digital inclusion and educational accessibility. Our findings demonstrate that ensemble methods combining multiple pre-trained language models achieve superior performance, with weighted F1 scores exceeding 77% for closely related language families. The study contributes to the growing body of work in Afrocentric NLP by providing empirical evidence for effective cross-linguistic transfer learning techniques and proposing a framework for sustainable language model development in resource-constrained environments

Files

Low-Resource Language Models for African Indigenous Languages.pdf

Files (357.4 kB)

Additional details

Identifiers

DOI
10.1038/s41586-020-2649-2

Related works

Describes
Publication: 10.1038/s41586-020-2649-2 (DOI)

Dates

Available
2024-11-09

References

  • 1. Mabokela, K.R., Primus, M. and Celik, T. (2025). Advancing sentiment analysis for low-resourced African languages using pre-trained language models. PLoS One, 20(6), e0325102. https://doi.org/10.1371/journal.pone.0325102 2. Adebara, I. and Abdul-Mageed, M. (2022). Towards Afrocentric NLP for African languages: where we are and where we can go. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pp. 3814-3841. 3. Magueresse, A., Carles, V. and Heetderks, E. (2020). Low-resource languages: a review of past work and future challenges. arXiv preprint, arXiv:2006.07264. 4. Ruder, S. (2020). Why you should do NLP beyond English. Available at: http://ruder.io/nlp-beyond-english 5. Smartling (2024). How the African Languages Lab empowers low-resource languages. Available at: https://www.smartling.com/blog/african-languages-lab-empowers-low-resource-languages-with-ai 6. Davel, M., Barnard, E., van Heerden, C., Wet, F. and Badenhorst, J. (2014). The NCHLT speech corpus of the South African languages. In Proceedings of the 4th International Workshop on Spoken Language Technologies for Under-Resourced Languages, pp. 194-200. 7. Naveed, H., Khan, A., Qiu, S., Saqib, M., Anwar, S. and Usman, M. (2023). A comprehensive overview of large language models. arXiv preprint, arXiv:2307.06435. 8. Africa World Initiative (2024). Lelapa AI launches Africa's first AI large language model. Available at: https://africaworld.princeton.edu/news/2024/lelapa-ai-launches-africa's-first-ai-large-language-model 9. Ogueji, K., Zhu, Y. and Lin, J. (2021). Small data? No problem! Exploring the viability of pretrained multilingual language models for low-resourced languages. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pp. 116-126. 10. Alabi, J.O., Adelani, D.I., Mosbach, M. and Klakow, D. (2022). Adapting pre-trained language models to African languages via multilingual adaptive fine-tuning. In Proceedings of the 29th International Conference on Computational Linguistics, pp. 4336-4349. 11. Devlin, J., Chang, M.W., Lee, K. and Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint, arXiv:1810.04805. 12. Hedderich, M.A., Adelani, D., Zhu, D., Alabi, J., Markus, U. and Klakow, D. (2020). Transfer learning and distant supervision for multilingual transformer models: A study on African languages. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 2580-2591. 13. IEEE Xplore (2021). Improving transformer model translation for low resource South African languages using BERT. In IEEE Conference Publication. https://ieeexplore.ieee.org/document/9659923/ 14. Korir, K. (2025). The Untapped Potential: African Languages in Natural Language Processing. Medium. Available. 15. Muhammad, S., Abdulmumin, I., Ayele, A., Ousidhoum, N., Adelani, D. and Yimam, S. (2023). AfriSenti: a Twitter sentiment analysis benchmark for African languages. arXiv preprint, arXiv:2302.08956. 16. Brown, T. and Kocielnik, R. (2019). Design thinking for AI: Conceptualizing how designers work with machine learning. In Proceedings of the 2019 on Designing Interactive Systems Conference, pp. 297-312. 17. Mabokela, K.R. and Schlippe, T. (2022). A sentiment corpus for South African under-resourced languages in a multilingual context. In The 1st Annual Meeting of the ELRA/ISCA Special Interest Group on Under-Resourced Languages, pp. 70-77. 18. Alexander, M. (2023). The 11 languages of South Africa. Available at: https://southafrica-info.com/arts-culture/11-languages-south-africa/ 19. Duvenhage, B. (2019). Short text language identification for under resourced languages. arXiv preprint, arXiv:1911.07555. 20. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M. and Chen, D. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint, arXiv:1907.11692. 21. Adebara, I., Elmadany, A., Abdul-Mageed, M. and Alcoba Inciarte, A. (2023). Serengeti: massively multilingual language models for Africa. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 1498-1537. 22. Ogunremi, T., Jurafsky, D. and Manning, C. (2023). Mini but mighty: efficient multilingual pretraining with linguistically-informed data selection. In Findings of the Association for Computational Linguistics: EACL 2023, pp. 1251-1266. 23. Perry, T. (2021). LightTag: text annotation platform. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 20-27. 24. Mohammad, S.M. and Turney, P.D. (2012). Crowdsourcing a word–emotion association lexicon. Computational Intelligence, 29(3), 436-465. 25. Mabokela, R., Roborife, M. and Celik, T. (2023). Investigating sentiment-bearing words- and emoji-based distant supervision approaches for sentiment analysis. In Proceedings of the Fourth Workshop on Resources for African Indigenous Languages, pp. 115-125. 26. Wang, M., Adel, H., Lange, L., Strötgen, J. and Schütze, H. (2023). NLDE at SemEval-2023 task 12: Adaptive pretraining and source language selection for low-resource multilingual sentiment analysis. In Proceedings of the 17th International Workshop on Semantic Evaluation, pp. 488-497. 27. Azime, I., Al-azzawi, S., Tonja, A., Shode, I., Alabi, J. and Awokoya, A. (2023). Masakhane-AfriSenti at SemEval-2023 task 12: Sentiment analysis using Afro-centric language models and adapters for low-resource African languages. In Proceedings of the 17th International Workshop on Semantic Evaluation, pp. 1311-1316. 28. Jin, Z. and Mihalcea, R. (2023). Natural language processing for policymaking. In Advances in Artificial Intelligence and Data Engineering, Springer, pp. 141-162. 29. Chakravarthi, B.R., Jose, N., Suryawanshi, S., Sherly, E. and McCrae, J.P. (2020). A sentiment analysis dataset for code-mixed Malayalam-English. In Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced Languages and Collaboration and Computing for Under-Resourced Languages, pp. 177-184. 30. VentureBeat (2024). What happens when the most powerful language technologies ignore the voices of an entire continent? Available at: https://venturebeat.com/ai/what-happens-when-the-most-powerful-language-technologies-ignore-the-voices-of-an-entire-continent/ 31. Wankhade, M., Rao, A.C.S. and Kulkarni, C. (2022). A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55(7), 5731-5780. 32. Vilares, D., Alonso, M.A. and Gómez-Rodríguez, C. (2017). Supervised sentiment analysis in multilingual environments. Information Processing & Management, 53(3), 595-607. 33. vanderWesthuizen, E. and Niesler, T. (2018). A first South African corpus of multilingual code-switched soap opera speech. In Proceedings of the International Conference on Language Resources and Evaluation, pp. 2854-2859. 34. Marivate, V., Sefara, T., Chabalala, V., Makhaya, K., Mokgonyane, T. and Mokoena, R. (2020). Low resource language dataset creation, curation and classification: Setswana and Sepedi. arXiv preprint, arXiv:2004.13842. 35. Nurse, D. and Philippson, G. (2003). The Bantu Languages. 1st edition, Routledge, London. 36. Fehn, A.M., Amorim, B. and Rocha, J. (2022). The linguistic and genetic landscape of southern Africa. Journal of Anthropological Sciences, 100, 243-265. 37. Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G. and Guzmán, F. (2019). Unsupervised cross-lingual representation learning at scale. arXiv preprint, arXiv:1911.02116. 38. Kudo, T. and Richardson, J. (2018). SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. arXiv preprint, arXiv:1808.06226. 39. Clark, K., Luong, M.T., Le, Q.V. and Manning, C.D. (2020). ELECTRA: Pre-training text encoders as discriminators rather than generators. In International Conference on Learning Representations. 40. Sciforce (2019). NLP for low-resource settings. Available at: https://medium.com/sciforce/nlp-for-low-resource-settings-b1b4e4b7ba9c 41. Muhammad, S.H., Adelani, D.I., Ruder, S., Ahmad, I.S., Abdulmumin, I. and Bello, B.S. (2022). NaijaSenti: a Nigerian sentiment corpus for multilingual sentiment analysis. arXiv preprint, arXiv:2201.08277. 42. Dossou, B.F.P., Tonja, A.L., Yousuf, O., Osei, S., Oppong, A., Shode, I., Ahia, O., Marivate, V., Davis, D., Hourrane, O. and Emezue, C. (2022). AfroLM: A self-active learning-based multilingual pretrained language model for 23 African languages. In Proceedings of The Third Workshop on Simple and Efficient Natural Language Processing, pp. 52-64. 43. Marivate, V., Mots'Oehli, M., Wagner, V., Lastrucci, R. and Dzingirai, I. (2022). PuoBERTa: Training and evaluation of a curated language model for Setswana. In Artificial Intelligence Research, Springer Nature, pp. 253-266. 44. Herbert, R. and Bailey, R. (2002). The Bantu languages: sociohistorical perspectives. Cambridge University Press, pp. 50-78. 45. Gunner, E. and Scheub, H. (2002). African literature — history, writers, books, characteristics, themes, & facts. Available at: https://www.britannica.com/art/African-literature 46. Hutto, C.J. and Gilbert, E. (2014). VADER: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the 8th International Conference on Weblogs and Social Media, pp. 216-225. 47. Nielsen, F. (2011). A new ANEW: Evaluation of a word list for sentiment analysis in microblogs. arXiv preprint, arXiv:1103.2903. 48. Kralj Novak, P., Smailović, J., Sluban, B. and Mozetič, I. (2015). Sentiment of Emojis. PLoS One, 10(12), e0144296. 49. Mohammad, S. (2016). A practical guide to sentiment annotation: challenges and solutions. In Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pp. 174-179. 50. Heugh, K. and Stroud, C. (2019). Multilingualism in South African education: A southern perspective. Cambridge University Press, pp. 216-238.