Large Language Models in Machine Translation

By Thorsten Brants, Ashok Popat, Peng Xu, Franz Joseph Och, Joseph Dean. 2007

720 citations recorded by the source corpus.

Subfields: Large Language Models, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Large Language Models

In Natural Language Processing

Citations in the corpus

LLaMA: Open and Efficient Foundation Language Models (2023, methodology): “Brants et al. (2007) showed the benefits of using language models trained on 2 trillion tokens, resulting in 300 billion n -grams, on the quality of machine translation.”
Efficient Estimation of Word Representations in Vector Space (2013, methodology): “An example is the popular N-gram model used for statistical language modeling - today, it is possible to train N-grams on virtually all available data (trillions of words [3]).”
CTRL: A Conditional Transformer Language Model for Controllable Generation (2019, background): “In natural language processing, language models are often trained as conditional language models for speciﬁc tasks that require text generation (Brants et al., 2007; Sutskever et al., 2014; Rush et al., 2015).”
Improving language models by retrieving from trillions of tokens (2021, background): “Brants et al. (2007) show that scaling the training data to trillions of tokens improves the machine translation performance of 𝑛 -gram models.”
Tagged Back-Translation (2019, methodology): “Approaches using monolingual data for machine translation include language model fusion for both phrasebased (Brants et al., 2007; Koehn, 2009) and neural MT (Gülçehre et al.”
Dual Learning for Machine Translation (2016, methodology): “In the ﬁrst category [2, 5], monolingual corpora in the target language are used to train a language model, which is then integrated with the MT models trained from parallel bilingual corpora to improve the translation quality.”
Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition (2014, methodology): “More recent works such as [1, 2, 21] make use of over-segmentation methods, guided by a supervised classiﬁer to generate candidate proposals, which are subsequently classiﬁed as characters or false positives.”
Gmail Smart Compose: Real-Time Assisted Writing (2019, neutral): “The traditional n-gram language model permits easy factorization in probability computation and compact representation forms like finite state automaton [38], which makes it amenable to large-scale distributed serving [6, 14].”
SYSTRAN's Pure Neural Machine Translation Systems (2016, background): “bing (13)In 2007, Google already mentions using 2 trillion words in their language models for machine translation (Brants et al., 2007).”
Text Repair Model for Neural Machine Translation (2019, methodology): “There are several approaches to leverage monolingual data for NMT: Language model fusion for both phrase-based (Brants et al., 2007) and neural MT (Gülçehre et al., 2015, 2017), back-translation (Sennrich et al., 2016b), unsupervised NMT (Lample et al., 2018a; Artetxe et al., 2018a), dual…”
Book Reviews: Data-Intensive Text Processing with MapReduce by Jimmy Lin and Chris Dyer (2010, background): “Yet another example concerns smoothing in web-scale language models [16].”
Textually Pretrained Speech Language Models (2023, methodology): “While Brants et al. [2007] showed the benefit of optimizing LLMs back in 2007, recently with the help of neural networks such models became foundational models, serving as a base model for different downstream tasks [Shoeybi et al., 2019, Radford et al., 2019, Brown et al., 2020, Lieber et al.,…”
Semi-supervised model adaptation for statistical machine translation (2007, background): “It has been shown that adding large amounts of target language text improves translation quality considerably , as improved language model estimates about potential output translations can be used by the decoder in order to improve translation quality (Brants et al., 2007).”
Putting Machine Translation in Context with the Noisy Channel Model (2019, background): “The beneﬁts of strong language models estimated from nonparallel data has long been recognized in translation (Brants et al., 2007), and more recently in noisy channel approaches to translation based on neural network component distributions (Yu et al., 2017; Yee et al., 2019; Ng et al., 2019).”