The Curious Case of Hallucinations in Neural Machine Translation

By Vikas Raunak, Arul Menezes, Marcin Junczys-Dowmunt. 2021

246 citations recorded by the source corpus.

In this work, we study hallucinations in Neural Machine Translation (NMT), which lie at an extreme end on the spectrum of NMT pathologies. Firstly, we connect the phenomenon of hallucinations under source perturbation to the Long-Tail theory of Feldman, and present an empirically validated hypothesis that explains hallucinations under source perturbation. Secondly, we consider hallucinations under corpus-level noise (without any source perturbation) and demonstrate that two prominent types of natural hallucinations (detached and oscillatory outputs) could be generated and explained through specific corpus-level noise patterns. Finally, we elucidate the phenomenon of hallucination amplification in popular data-generation processes such as Backtranslation and sequence-level Knowledge Distillation. We have released the datasets and code to replicate our results.

Subfields: Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Survey of Hallucination in Natural Language Generation (2022, methodology): “[109], Wang and Sennrich [142], Bengio et al.”
Counterfactual Memorization in Neural Language Models (2021, positive): “Further extensions were shown to work well on detecting mislabeled data in classification problems [Wang and Jia, 2022] and characterizing hallucinations in Neural Machine Translation [Raunak et al., 2021].”
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation (2021, background): “Oscillatory hallucinations are one class of hallucinations characterized by repeating bi-gram structure in the output(Raunak et al., 2021).”
xcomet: Transparent Machine Translation Evaluation through Fine-grained Error Detection (2023, background): “We create detached and oscillatory hallucinations (Raunak et al., 2021; Guerreiro et al., 2023): (i) detached hallucinations, replacing the translation with a random sentence or an unrelated one semantically similar to the source sentence; 6 and (ii) oscillatory hallucinations, where we randomly…”
How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation (2023, methodology): “First, not using parallel data might imply that LLMs are protected against the noise associated with parallel data, which leads to problems such as the memorization of noisy/atypical (Raunak et al., 2021) or low-quality samples (Raunak and Menezes, 2022) and biases towards particular language charac”
On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models? (2022, background): “Hallucination in neural language generation has recently attracted the attention of several researchers in many areas including neural machine translation (NMT) (Raunak et al., 2021; Wang and Sen-nrich, 2020) and summarization (Durmus et al., 2020; Kang and Hashimoto, 2020).”
The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation (2021, background): “There may be interesting connections to recent ﬁndings about output hallucinations occurring on memorized examples (Raunak et al., 2021), or with respect to translation bias (Koppel and Ordan, 2011).”
How Language Model Hallucinations Can Snowball (2023, unknown): “Hallucinations Hallucination in text generation is a well-studied problem (Rohrbach et al., 2018; Maynez et al., 2020; Raunak et al., 2021, i.a.) that has recently become more prominent due to Chat-GPT’s tendency to produce plausible-sounding falsehoods.”
Benchmarking Large Language Models in Retrieval-Augmented Generation (2023, background): “…models The knowledge stored in large language models is commonly out-of-date (He, Zhang, and Roth 2022) and they also sometimes generate hallu-cination (Cao et al. 2020; Raunak, Menezes, and Junczys-Dowmunt 2021; Ji et al. 2023) i.e., they may generate irrelevant or factually incorrect contents.”