Beam Search Strategies for Neural Machine Translation

By Markus Freitag, Yaser Al-Onaizan. NMT@ACL, 2017

513 citations recorded by the source corpus.

The basic concept in Neural Machine Translation (NMT) is to train a large Neural Network that maximizes the translation performance on a given parallel corpus. NMT is then using a simple left-to-right beam-search decoder to generate new translations that approximately maximize the trained conditional probability. The current beam search strategy generates the target sentence word by word from left-to-right while keeping a fixed amount of active candidates at each time step. First, this simple search is less adaptive as it also expands candidates whose scores are much worse than the current best. Secondly, it does not expand hypotheses if they are not within the best scoring candidates, even if their scores are close to the best one. The latter one can be avoided by increasing the beam size until no performance improvement can be observed. While you can reach better performance, this has the drawback of a slower decoding speed. In this paper, we concentrate on speeding up the decoder by applying a more flexible beam search strategy whose candidate size may vary at each time step depending on the candidate scores. We speed up the original decoder by up to 43% for the two language pairs German to English and Chinese to English without losing any translation quality.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a8e4580471908d17e279000d328f39654359bd6e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Interactive Visualization and Manipulation of Attention-based Neural Machine Translation (2017, methodology): “(Wu et al., 2016; Freitag and Al-Onaizan, 2017) proposed several penalty functions and pruning methods for beam search.”
Analyzing and Mitigating Object Hallucination in Large Vision-Language Models (2023, methodology): “In language modeling, beam search (Holtzman et al., 2019; Freitag & Al-Onaizan, 2017) is employed to predict words iteratively, introducing inherent uncertainty into the search process (Please refer to illustrative examples in Appendix D.1).”
LaMP: When Large Language Models Meet Personalization (2023, methodology): “We employ beam search (Freitag and Al-Onaizan, 2017) with a beam size of 4 in all experiments to improve the model’s ability to generate high-quality predictions.”
Why So Toxic?: Measuring and Triggering Toxic Behavior in Open-Domain Chatbots (2022, background): “Compared to the traditional greedy or beam search decoding methods [18], nucleus sampling provides open-domain generation leading to high sentence diversity.”
Neural Machine Translation: A Review (2019, methodology): “Wu et al. (2016), Freitag and Al-Onaizan (2017) suggested to use a variable beam size, using various heuristics to decide the beam size at each time step.”
Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (2023, methodology): “From the adjusted output distribution p vcd , we can apply various sampling strategies, such as nucleus sampling [25] and beam search [15].”
Answer Complex Questions: Path Ranker Is All You Need (2021, background): “The second group tends to utilize the retrieval scores of the candidate set in each hop for re-ranking the retrieval paths through beam or greedy search [8].”
LSDSCC: a Large Scale Domain-Specific Conversational Corpus for Response Generation with Diversity Oriented Evaluation Metrics (2018, background): “…metrics, that can be taken to measure both candidate response’s diversity and its relevance to the given query, to effectively guide the training of NRG models towards the state of promoting meaningful and diverse responses (Li et al., 2016a; Shao et al., 2017; Freitag and Al-Onaizan, 2017).”
Empirical Analysis of Beam Search Performance Degradation in Neural Sequence Models (2019, background): “Freitag & Al-Onaizan (2017) analyzed pruning techniques for a beam search decoder in machine translation.”