A massively parallel corpus: the Bible in 100 languages

By Christos Christodoulopoulos, Mark Steedman. 2014

259 citations recorded by the source corpus.

We describe the creation of a massively parallel corpus based on 100 translations of the Bible. We discuss some of the difficulties in acquiring and processing the raw material as well as the potential of the Bible as a corpus for natural language processing. Finally we present a statistical analysis of the corpora collected and a detailed comparison between the English translation and other English corpora.

Subfields: Artificial Intelligence, Language and Linguistics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Language and Linguistics

Citations in the corpus

The Flores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation (2021, background): “…are some benchmarks that have high coverage, but these are often in speciﬁc domains, like COVID-19 (Anastasopoulos et al., 2020) or religious texts (Christodouloupoulos and Steedman, 2015; Malaviya et al., 2017; Tiede-mann, 2018; Agi´c and Vuli´c, 2019); or have low quality because they are built…”
JW300: A Wide-Coverage Parallel Corpus for Low-Resource Languages (2019, background): “6 Moving from OPUS and Tatoeba towards greater linguistic breadth, there are several publicly available Bible datasets, most notably those by Mayer and Cysouw (2014) and Christodouloupoulos and Steedman (2015).”
Scaling Speech Technology to 1, 000+ Languages (2023, methodology): “For machine translation, researchers used data from the Bible both for training and evaluation [Christodouloupoulos and Steedman, 2015, McCarthy et al., 2020, NLLB_Team et al., 2022].”
MaSS: A Large and Clean Multilingual Corpus of Sentence-aligned Spoken Utterances Extracted from the Bible (2019, background): “These resources feed truly multilingual approaches, such as machine translation [13], syntax parsing [14], automatic speech recognition [15, 16], lexical disambiguation [17, 18] and computational dialectology [19].”
Bi-Directional Neural Machine Translation with Synthetic Parallel Data (2018, methodology): “We split a parallel Bible corpus (Christodoulopoulos and Steedman, 2015) into sample, dev, and test sets, using the sample data as the in-domain seed for data selection.”
Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation (2020, methodology): “Mimicking Guzmán et al. (2019) we use Bible data (Christodouloupoulos and Steedman, 2015), GNOME/KDE/Ubuntu documentation and Global Voices (version 2018q4) data from the OPUS repository (Tiedemann, 2012).”
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2020, background): “Sources for parallel text can be the OPUS project (Tiedemann, 2012), Bible corpora (Mayer and Cysouw, 2014; Christodoulopoulos and Steedman, 2015) or the re-”
Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages (2021, unknown): “These include localization data (GNOME, KDE4, Ubuntu, Mozilla-I10n), religious text (JW300 (Agić and Vulić, 2019), Bible-eudin (Christodouloupoulos and Steedman, 2015), Tanzil), GlobalVoices, OpenSubtitles (Lison and Tiedemann, 2016), TED2020 (Reimers and Gurevych, 2020), WikiMatrix (Schwenk et al.”
A Survey of Multilingual Neural Machine Translation (2019, background): “The Bible corpus [26] is a massive N-way one originally containing 100 languages.”
MMCR4NLP: Multilingual Multiway Corpora Repository for Natural Language Processing (2017, background): “, 2012), ILCI (Jha, 2010) and Bible (Christodouloupoulos and Steedman, 2015) corpora.”
Enabling Multi-Source Neural Machine Translation By Concatenating Source Sentences In Multiple Languages (2017, background): “…largest to smallest according to number of lines of corpora) like United Nations (Ziemski et al., 2016), Europarl (Koehn, 2005), Ted Talks (Cettolo et al., 2012) ILCI (Jha, 2010) and Bible (Christodouloupoulos and Steedman, 2015) where the same sentences are translated into more than 5 languages.”