Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages

By W. Nekoto, V. Marivate, T. Matsila, Timi E. Fasubaa, T. Kolawole, T. Fagbohungbe, S. Akinola, Shamsuddeen Hassan Muhammad, Salomon Kabongo KABENAMUALU, Salomey Osei and 38 more. 2020

322 citations recorded by the source corpus.

Research in NLP lacks geographic diversity, and the question of how NLP can be scaled to low-resourced languages has not yet been adequately solved. ‘Low-resourced’-ness is a complex problem going beyond data availability and reflects systemic problems in society. In this paper, we focus on the task of Machine Translation (MT), that plays a crucial role for information accessibility and communication worldwide. Despite immense improvements in MT over the past decade, MT is centered around a few high-resourced languages. As MT researchers cannot solve the problem of low-resourcedness alone, we propose participatory research as a means to involve all necessary agents required in the MT development process. We demonstrate the feasibility and scalability of participatory research with a case study on MT for African languages. Its implementation leads to a collection of novel translation datasets, MT benchmarks for over 30 languages, with human evaluations for a third of them, and enables participants without formal training to make a unique scientific contribution. Benchmarks, models, data, code, and evaluation results are released at https://github.com/masakhane-io/masakhane-mt.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Beyond English-Centric Multilingual Machine Translation (2020, background): “Further, community efforts such as masakhane (∀ et al., 2020) and the African Language data sets Challenge (Siminyu et al.”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “…(2020); Clark et al. (2020)) represent another source of subpopulation shifts with corresponding disparities in model performance: training sets might contain fewer examples in low-resource languages (Nekoto et al., 2020), but we would still hope for high model performance on these minority groups.”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 (2021, background): “One particularly exciting example is the Masakhane project [91], which explores participatory research techniques for developing MT for African languages.”
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice (2023, background): “3420157 [156] Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Tajudeen Kolawole, Taiwo Fagbohungbe, Solomon Oluwole Akinola, Shamsuddeen Hassan Muhammad, Salomon Kabongo, Salomey Osei, et al.”
Neural Machine Translation for Low-resource Languages: A Survey (2021, background): “Within about two years, the Masakhane community has covered more than 38 African languages and resulted in multiple publications [138].”