IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models
IrokoBench is a human-translated evaluation benchmark covering 17 typologically diverse low-resource African languages across three tasks: natural language inference (AfriXNLI), mathematical reasoning (AfriMGSM) and knowledge-based multiple-choice QA (AfriMMLU). The paper evaluates open and proprietary LLMs and documents a large gap between high-resource languages and African languages, with the best open model reaching about 63 percent of GPT-4o performance. It was published at NAACL 2025.
- Category
- Research
- Pricing
- Free / open
- Country
- 馃實 Pan-African
- Last verified
- 5 Jul 2026
Tags
Compare IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models
Side-by-side, verified specs against its closest nlp benchmark alternatives.
Related in Research
AfriHate: A Multilingual Collection of Hate Speech and Abusive Language Datasets for African Languages
AfriHate is a multilingual benchmark of hate speech and abusive language datasets covering 15 African languages, annotated by native speakers. The paper contributes classification baselines and hate speech and offensive language lexicons, and analyses why keyword-based moderation fails for low-resource African languages. It was released on arXiv in January 2025.
AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages
AfriSenti is a sentiment analysis benchmark of more than 110,000 tweets in 14 African languages spanning four language families, annotated by native speakers. It underpinned SemEval-2023 Task 12, a shared task that attracted more than 200 participants, and documents data collection, annotation and baseline methods for low-resource languages.
AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languages
AfriQA is the first cross-lingual open-retrieval question answering benchmark for African languages, with more than 12,000 XOR-QA examples across 10 African languages. The paper shows that current automatic translation and multilingual retrieval methods perform poorly for these languages, where in-language digital content is scarce.