A Systematic Exploration of Diversity in Machine Translation

By Kevin Gimpel, Dhruv Batra, Chris Dyer, Gregory Shakhnarovich. 2013

99 citations recorded by the source corpus.

This paper addresses the problem of producing a diverse set of plausible translations. We present a simple procedure that can be used with any statistical machine translation (MT) system. We explore three ways of using diverse translations: (1) system combination, (2) discriminative reranking with rich features, and (3) a novel post-editing scenario in which multiple translations are presented to users. We find that diversity can improve performance on these tasks, especially for sentences that are difficult for MT.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

A Diversity-Promoting Objective Function for Neural Conversation Models (2015, background): “Carbonell and Goldstein (1998) and Gimpel et al. (2013) produce multiple outputs that are mutually diverse, either non-redundant summary sentences or N-best lists.”
Posterior calibration and exploratory analysis for natural language processing models (2015, background): “, 2008), or explicitly diverse lists (Gimpel et al., 2013); often the goal is to marginalize over structures to calculate and minimize an expected loss function, as in minimum Bayes risk decoding (Goodman, 1996; Kumar and Byrne, 2004), or to perform joint inference between early and later stages of ”
A Simple, Fast Diverse Decoding Algorithm for Neural Generation (2016, background): “The most relevant is work from Gimpel et al. (2013) and Batra et al. (2012) that produces diverse N-best lists by adding a dissimilarity function based on N-gram overlaps, distancing the current translation from already-generated ones by choosing translations that have high scores but are distinct…”
Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models (2016, methodology): “Most related to our proposed approach is the work of Gimpel et al. (2013), who applied DivMBest to machine translation using beam search as a black-box inference algorithm. Specifically, in this approach, DivMBest knows nothing about the inner-workings of BS and simply makesB sequential calls to BS ”
Analyzing Uncertainty in Neural Machine Translation (2018, background): “in the data distribution, or decoding strategies which attempt to predict diverse outputs while leaving the model unchanged (Gimpel et al., 2013; Vijayakumar et al., 2016; Li & Jurafsky, 2016; Cho, 2016).”
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints (2018, neutral): “Gimpel et. al.”
Models and Inference for Prefix-Constrained Machine Translation (2016, background): “However, it is well-known that n -best lists are poor approximations of MT structured output spaces (Macherey et al., 2008; Gimpel et al., 2013).”
Learning the Curriculum with Bayesian Optimization for Task-Specific Word Representation Learning (2016, background): “Entropy is the bestknown measure of diversity in statistical research, but there are many others (Tang et al., 2006; Gimpel et al., 2013).”
Making 360° Video Watchable in 2D: Learning Videography for Click Free Viewing (2017, methodology): “Diverse solution search Generating a diverse set of candidate solutions has been widely discussed in different domains [6, 11, 25, 28, 41].”
Neural Machine Translation: A Review (2019, methodology): “An issue with using beam search is that the hypotheses found by the decoder are very similar to each other and often differ only by one or two words (Li & Jurafsky, 2016; Li et al., 2016b; Gimpel et al., 2013).”
Submodular Optimization-based Diverse Paraphrasing and its Effectiveness in Data Augmentation (2019, methodology): “Although diversity in paraphrasing models has not been explored extensively, methods have been proposed to address diversity in other NLP tasks (Li et al., 2016, 2015; Gimpel et al., 2013).”
Mechanism-Aware Neural Machine for Dialogue Response Generation (2017, methodology): “Gimpel et al. (2013) proposed the methods, namely system combination and discriminating re-ranking, to produce a diverse set of plausible translations.”
Sequence-to-Sequence Data Augmentation for Dialogue Language Understanding (2018, methodology): “During testing, we use beam search with beam size of 10 to yield more than one translation following Gimpel et al. (2013) and Vijayakumar et al. (2016).”
Mutual Information and Diverse Decoding Improve Neural Machine Translation (2016, background): “Neural machine translation models map source x = { x 1 , x 2 , ...x N x } to a continuous vector representation, from which target output y = { y 1 , y 2 , ..., y N y } is to be generated.”
Diverse Beam Search for Improved Description of Complex Scenes (2018, methodology): “By running multiple beam searches in parallel and at staggered time offsets, we obtain large time savings, making our method comparable to a single run of classical BS and M times faster than (Gimpel et al. 2013).”
First the Worst: Finding Better Gender Translations During Beam Search (2021, background): “This lack of output diversity has been noted in machine translation, both statistical (Gimpel et al., 2013) and neural (Vanmassenhove et al.”
Creativity: Generating Diverse Questions Using Variational Autoencoders (2017, background): “However in [50] only a single question is generated per image, while the approach discussed in [63] generates diverse questions by sampling from a complicated energy landscape, which is intractable in general [23, 5].”
multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning (2021, background): “Multiple prior studies focus on generating diverse unstructured texts (Gimpel et al., 2013; Dai et al., 2017; Xu et al., 2018; Raffel et al., 2020). which broadly span two categories – (1) using improved decoding techniques like beam search with inter-sibling ranking penalty (Li et al., 2016),…”
An Attentional Neural Conversation Model with Improved Specificity (2016, background): “Prior work to potentially increase specificity or diversity aims at producing multiple outputs (Carbonell and Goldstein, 1998; Gimpel et al., 2013) and our work is the same as in (Li et al.”