In-context Examples Selection for Machine Translation

By Sweta Agrawal, Chunting Zhou, Michael Lewis, Luke S. Zettlemoyer, Marjan Ghazvininejad. 2022

260 citations recorded by the source corpus.

Large-scale generative models show an impressive ability to perform a wide range of Natural Language Processing (NLP) tasks using in-context learning, where a few examples are used to describe a task to the model. For Machine Translation (MT), these examples are typically randomly sampled from the development dataset with a similar distribution as the evaluation set. However, it is unclear how the choice of these in-context examples and their ordering impacts the output translation quality. In this work, we aim to understand the properties of good in-context examples for MT in both in-domain and out-of-domain settings. We show that the translation quality and the domain of the in-context examples matter and that 1-shot noisy unrelated example can have a catastrophic impact on output quality. While concatenating multiple random examples reduces the effect of noise, a single good prompt optimized to maximize translation quality on the development dataset can elicit learned information from the pre-trained language model. Adding similar examples based on an n-gram overlap with the test source significantly and consistently improves the translation quality of the outputs, outperforming a strong kNN-MT baseline in 2 out of 4 out-of-domain datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Prompting & Reasoning, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Philipp Koehn
  2. Felix Stahlberg
  3. Colin Cherry
  4. Dacheng Tao
  5. George F. Foster
  6. Rico Sennrich
  7. Markus Freitag
  8. David Vilar

In Computer Vision and Pattern Recognition

  1. Bo Du
  2. David Vilar
  3. Colin Cherry
  4. George F. Foster
  5. Dacheng Tao
  6. Felix Stahlberg
  7. Markus Freitag
  8. Amr Sharaf

In Machine Translation

  1. Dacheng Tao
  2. Colin Cherry
  3. David Vilar
  4. George F. Foster
  5. Zhaopeng Tu
  6. Markus Freitag

Citations in the corpus