IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models vs MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition

IrokoBench is a human-translated evaluation benchmark covering 17 typologically diverse low-resource African languages across three tasks: natural language inference (AfriXNLI), mathematical reasoning (AfriMGSM) and knowledge-based multiple-choice QA (AfriMMLU). The paper evaluates open and proprietary LLMs and documents a large gap between high-resource languages and African languages, with the best open model reaching about 63 percent of GPT-4o performance. It was published at NAACL 2025.

MasakhaNER 2.0 introduces the largest human-annotated named entity recognition dataset for 20 African languages and studies Africa-centric cross-lingual transfer learning. The paper reports that choosing the best transfer language improves zero-shot F1 by an average of 14 points across the 20 languages compared with transferring from English.