Guiding Neural Machine Translation with Retrieved Translation Pieces

By Jingyi Zhang, Masso Utiyama, Eiichiro Sumita, Graham Neubig, Satoshi Nakamura. 2018

148 citations recorded by the source corpus.

One of the difficulties of neural machine translation (NMT) is the recall and appropriate translation of low-frequency words or phrases. In this paper, we propose a simple, fast, and effective method for recalling previously seen translation examples and incorporating them into the NMT decoding process. Specifically, for an input sentence, we use a search engine to retrieve sentence pairs whose source sides are similar with the input sentence, and then collect n-grams that are both in the retrieved target sentences and aligned with words that match in the source sentences, which we call “translation pieces”. We compute pseudo-probabilities for each retrieved sentence based on similarities between the input sentence and the retrieved source sentences, and use these to weight the retrieved translation pieces. Finally, an existing NMT model is used to translate the input sentence, with an additional bonus given to outputs that contain the collected translation pieces. We show our method improves NMT translation results up to 6 BLEU points on three narrow domain translation tasks where repetitiveness of the target sentences is particularly salient. It also causes little increase in the translation time, and compares favorably to another alternative retrieval-based method with respect to accuracy, speed, and simplicity of implementation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Improving language models by retrieving from trillions of tokens (2021, methodology): “Work in machine translation such as Zhang et al. (2018) and Gu et al. (2018) retrieve translation pairs based on edit distance between source sentences and guide the translation output using the closest retrieved target sentences.”
Nearest Neighbor Machine Translation (2020, methodology): “…to prior work on non-parametric methods for MT, our approach is arguably simpler (in that it requires no training, as compared to Gu et al. (2018)) and more expressive (in that it provides access to billions of key-value pairs during inference, as compared to Zhang et al. (2018); Gu et al. (2018)).”
Attributed Text Generation via Post-hoc Research and Revision (2022, background): “…be effective for a wide range of tasks such as question answering (Chen et al., 2017; Lee et al., 2019; Nakano et al., 2021), machine translation (Zhang et al., 2018), code generation (Hayati et al., 2018), language modeling (Khandelwal et al., 2020), and other knowledge-intensive tasks (Lewis…”
Neural Fuzzy Repair: Integrating Fuzzy Matches into Neural Machine Translation (2019, methodology): “…imposed on the NMT search algorithms (Hokamp and Liu, 2017), rewards attached to retrieved and matched translation pieces that guide the NMT output (Zhang et al., 2018), by explicitly providing the NMT system with access to a list of retrieved TM matches during decoding (Gu et al., 2018), or by…”
Deep Learning for Text Style Transfer: A Survey (2020, methodology): “For the architecture of the classifier, Zhang et al. (2018c) use LSTM, and Sudhakar, Upadhyay, and Maheswaran (2019) use a BERT classifier, where the BERT classifier has shown higher detection accuracy for the attribute markers.”
Case-based Reasoning for Natural Language Queries over Knowledge Bases (2021, methodology): “There has been work in machine translation (Zhang et al., 2018; Gu et al., 2018; Khandelwal et al., 2021) that uses nearest neighbor translation pair to guide the decoding pro-”
Controllable Semantic Parsing via Retrieval Augmentation (2021, methodology): “And some use the retrievals to explicitly manipulate the token scores at each decoding step (Zhang et al., 2018; Hayati et al., 2018; Peng et al., 2019; Khandelwal et al., 2020, 2021).”
Retrieval-based Neural Source Code Summarization (2020, methodology): “The NMT-based methods generally prefer high-frequency words in the corpus and may have trouble with low-frequency words [3, 64].”
RARR: Researching and Revising What Language Models Say, Using Language Models (2022, background): “, 2021), machine translation (Zhang et al., 2018), code generation (Hayati et al.”
Retrieval-Based Neural Code Generation (2018, methodology): “In machine translation (MT) problems (Zhang et al., 2018; Gu et al., 2018; Amin Farajian et al., 2017; Li et al., 2018), hybrid methods combining retrieval of salient examples and neural models have proven successful in dealing with rare words.”
Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning (2021, methodology): “…texts from a certain database (such as the training set) has been widely applied in tasks such as neural machine translation (Gu et al., 2018; Zhang et al., 2018; Xu et al., 2020), text generation (Weston et al., 2018; Kim et al., 2020), semantic parsing (Hashimoto et al., 2018; Guo et al.,…”