FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation

By Parker Riley, Timothy Dozat, Jan A. Botha, Xavier García, Dan Garrette, Jason Riesa, Orhan Firat, Noah Constant. 2022

27 citations recorded by the source corpus.

We present FRMT, a new dataset and evaluation benchmark for Few-shot Region-aware Machine Translation, a type of style-targeted translation. The dataset consists of professional translations from English into two regional variants each of Portuguese and Mandarin Chinese. Source documents are selected to enable detailed analysis of phenomena of interest, including lexically distinct terms and distractor terms. We explore automatic evaluation metrics for FRMT and validate their correlation with expert human evaluation across both region-matched and mismatched rating scenarios. Finally, we present a number of baseline models for this task, and offer guidelines for how researchers can train, evaluate, and compare their own models. Our dataset and evaluation code are publicly available: https://bit.ly/frmt-task.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Graham Neubig
  2. Quoc V. Le
  3. Luke S. Zettlemoyer
  4. Mohit Bansal
  5. Jianfeng Gao
  6. Percy Liang
  7. Noah A. Smith
  8. Yi Tay

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Quoc V. Le
  3. Dacheng Tao
  4. Andrew Zisserman
  5. Trevor Darrell
  6. Yi Yang
  7. Cordelia Schmid
  8. Alan L. Yuille

Citations in the corpus