Research registry
Research
Papers, benchmarks, standards and reference architectures for African digital infrastructure.
5 results in NLP benchmark
AfriHate: A Multilingual Collection of Hate Speech and Abusive Language Datasets for African Languages
AfriHate is a multilingual benchmark of hate speech and abusive language datasets covering 15 African languages, annotated by native speakers. The paper contributes classification baselines and hate speech and offensive language lexicons, and analyses why keyword-based moderation fails for low-resource African languages. It was released on arXiv in January 2025.
AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languages
AfriQA is the first cross-lingual open-retrieval question answering benchmark for African languages, with more than 12,000 XOR-QA examples across 10 African languages. The paper shows that current automatic translation and multilingual retrieval methods perform poorly for these languages, where in-language digital content is scarce.
AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages
AfriSenti is a sentiment analysis benchmark of more than 110,000 tweets in 14 African languages spanning four language families, annotated by native speakers. It underpinned SemEval-2023 Task 12, a shared task that attracted more than 200 participants, and documents data collection, annotation and baseline methods for low-resource languages.
IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models
IrokoBench is a human-translated evaluation benchmark covering 17 typologically diverse low-resource African languages across three tasks: natural language inference (AfriXNLI), mathematical reasoning (AfriMGSM) and knowledge-based multiple-choice QA (AfriMMLU). The paper evaluates open and proprietary LLMs and documents a large gap between high-resource languages and African languages, with the best open model reaching about 63 percent of GPT-4o performance. It was published at NAACL 2025.
MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition
MasakhaNER 2.0 introduces the largest human-annotated named entity recognition dataset for 20 African languages and studies Africa-centric cross-lingual transfer learning. The paper reports that choosing the best transfer language improves zero-shot F1 by an average of 14 points across the 20 languages compared with transferring from English.