On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation

By Chaojun Wang, Rico Sennrich. 2020

200 citations recorded by the source corpus.

The standard training algorithm in neural machine translation (NMT) suffers from exposure bias, and alternative algorithms have been proposed to mitigate this. However, the practical impact of exposure bias is under debate. In this paper, we link exposure bias to another well-known problem in NMT, namely the tendency to generate hallucinations under domain shift. In experiments on three datasets with multiple test domains, we show that exposure bias is partially to blame for hallucinations, and that training with Minimum Risk Training, which avoids exposure bias, can mitigate this. Our analysis explains why exposure bias is more problematic under domain shift, and also links exposure bias to the beam search problem, i.e. performance deterioration with increasing beam size. Our results provide a new justification for methods that reduce exposure bias: even if they do not increase performance on in-domain test sets, they can increase model robustness to domain shift.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Survey of Hallucination in Natural Language Generation (2022, methodology): “[145], Wang and Sennrich [180] Bengio et al.”
Investigating Catastrophic Forgetting During Continual Training for Neural Machine Translation (2020, background): “Wang and Sennrich (2020) links the exposure bias problem (Ranzato et al., 2016; Shao et al., 2018; Zhang et al., 2019) to the phenomenon of NMT tends to generate hallucinations under domain shift.”
Detecting Hallucinated Content in Conditional Neural Sequence Generation (2020, methodology): “Previous work (Wang and Sennrich, 2020; Müller et al., 2019; Koehn and Knowles, 2017) has shown that translation models are particularly prone to hallucination when tested out of domain.”
On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models? (2022, background): “Hallucination in neural language generation has recently attracted the attention of several researchers in many areas including neural machine translation (NMT) (Raunak et al., 2021; Wang and Sen-nrich, 2020) and summarization (Durmus et al., 2020; Kang and Hashimoto, 2020).”
How Language Model Hallucinations Can Snowball (2023, background): “Prior work linked this to compounding hallucinations in machine translation (Wang and Sennrich, 2020) and open-ended text generation (Arora et al.”
ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports (2022, background): “Hallucinations are an intrinsic problem of generative models like LLMs [15] and difficult to remedy [16, 17].”
AgreeSum: Agreement-Oriented Multi-Document Summarization (2021, background): “It is timely to revisit the problem also because recent stronger neural abstractive summarization models are prone to hallucination, as are neural text generation in general (Wiseman et al., 2017; Tian et al., 2019; Wang and Sennrich, 2020; Pang and He, 2020).”
The Curious Case of Hallucinations in Neural Machine Translation (2021, unknown): “Wang and Sennrich (2020) consider hallucinations as outputs detached from the source, and demonstrate that NMT models are more prone to hallucinations under out-of-domain settings by manually ascertaining whether an output generated is hallucinated or not.”
Relating Neural Text Degeneration to Exposure Bias (2021, methodology): “Since then, there have been many methods proposed to tackle this problem (Bengio et al., 2015; Huszár, 2015; Ranzato et al., 2016; Lamb et al., 2016; Yu et al., 2017; Wiseman and Rush, 2016; Schmidt, 2019; Zhang et al., 2019a; Wang and Sennrich, 2020).”
Do Long-Range Language Models Actually Use Long-Range Context? (2021, methodology): “When these models are deployed practically to generate text, they have to rely on their previous predictions instead of ground-truth text, and several prior works have noted different behavior in this setting (Wang and Sennrich, 2020; Holtzman et al., 2020; Welleck et al., 2020).”
Text Generation by Learning from Demonstrations (2020, background): “In practice, prior work has observed problems such as repetition and hallucination partly due to exposure bias (Holtzman et al., 2020; Wang & Sennrich, 2020).”
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation (2022, background): “Finally, previous works have linked exposure bias to out-of-domain (Wang and Sennrich, 2020) and out of distribution (Schmidt, 2019) generalization, and hallucinations (Wang and Sennrich, 2020) but these claims remain weak in absence of a clear and principled formalization of”
Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation? (2019, background): “Finally, Wang and Sennrich (2020) discusses the potential link between exposure bias and hallucination in the context of machine translation.”
Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation (2021, background): “4832 plan leads to inconsistencies; (2) even if a perfect content-plan is provided, the autoregressive model used in the second stage is still prone to hallucinate unfaithful contents due to the well-known exposure bias (Wang and Sennrich, 2020) problem; (3) there is no guarantee that the selected k”
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, unknown): “Such inconsistency can result in hallucinations (Wang and Sennrich, 2020), especially when an erroneous token generated by the model cascades errors throughout the subsequent sequence, akin to a snowball effect (Zhang et al., 2023c).”
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models (2023, unknown): “…NLG models are typically designed for a single task, and thus, hallucination studies on them are usually task-specific (Maynez et al., 2020; Wang and Sennrich, 2020; Xiao and Wang, 2021); however, current LLMs are expected to excel in multi-task, multi-lingual, and multi-domain settings…”
Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding (2021, background): “, 2018), exposure bias (Wang and Sennrich, 2020), and question answering (Feng et al.”
Chain-of-Verification Reduces Hallucination in Large Language Models (2023, background): “Further, in longform tasks consisting of generating multiple sentences or paragraphs, the hallucination problem can be exacerbated due to the issue of exposure bias (Wang and Sennrich, 2020).”
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation (2021, background): “, 2020), machine translation (Wang and Sennrich, 2020), datato-text generation (Rebuffel et al.”