Development and evaluation of a framework for detecting hate speech and abusive language in Zambia using machine learning

Abstract

The advent of artificial intelligence (AI) has revolutionized various fields, including information technology, intelligent transportation systems, virtual personal assistants, robotic surgery, and natural language processing (NLP) applications. However, along with the numerous benefits brought about by technological advancements, there are also drawbacks, such as the widespread dissemination of abusive language, fake news, and hate speech, which can easily be propagated in the digital world. Social media platforms like Facebook and Twitter have played a significant role in the rapid spread of rumors, conspiracy theories, hatred, xenophobia, racism, and prejudice. The misuse of technology has not only influenced public opinion but also impacted religious views worldwide, enabling targeting of individuals based on various attributes. Zambia's social media landscape has witnessed a dynamic shift, particularly following the transition of the government in 2021, which has led to greater freedom of expression but also an upsurge in hate speech and abusive language associated with political, ethnic, and religious divisions. The freedom of expression (FoE) in Zambia has facilitated the sharing of diverse views and ideas, contributing to development, democracy, and dialogue. However, this freedom has also led to the proliferation of hate speech on various online platforms, including social media. Despite the efforts of governments, the technology industry, and individual researchers to address the issue of hate speech, challenges persist. Legislative measures have been attempted to suppress hate speech, but their effectiveness is often limited. The main objective of this study was to develop and evaluate the framework for detecting hate speech and abusive language in Zambia. Cross-Industry Standard Procedure for Data Mining (CRISP-DM) methodology, a commonly used method for overseeing data science projects, was used to perform this study. precision, recall, and F1 score was used to evaluate the framework. Gradient-boosted decision tree was picked over the other algorithms (KNeighbors Classifier, logistic Regression, Random Forest, Decision Tree and Naïve Bayes) because apart from being a powerful machine learning algorithm that has become increasingly popular in recent years, especially in tasks such as classification and regression.