Exploring the Planet of the APEs: a Comparative Study of State-of-the-art Methods for MT Automatic Post-Editing

By Rajen Chatterjee, Marion Weller, Matteo Negri, Marco Turchi. 2015

58 citations recorded by the source corpus.

Downstream processing of machine translation (MT) output promises to be a solution to improve translation quality, especially when the MT system’s internal decoding process is not accessible. Both rule-based and statistical automatic postediting (APE) methods have been proposed over the years, but with contrasting results. A missing aspect in previous evaluations is the assessment of different methods: i) under comparable conditions, and ii) on different language pairs featuring variable levels of MT quality. Focusing on statistical APE methods (more portable across languages), we propose the first systematic analysis of two approaches. To understand their potential, we compare them in the same conditions over six language pairs having English as source. Our results evidence consistent improvements on all language pairs, a relation between the extent of the gain and MT output quality, slight but statistically significant performance differences between the two methods, and their possible complementarity.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Philipp Koehn
2. Ondrej Bojar
3. Graham Neubig
4. Markus Freitag
5. Felix Stahlberg
6. Luke S. Zettlemoyer
7. Lucia Specia
8. Rico Sennrich

In Evaluation & Benchmarks

1. Ondrej Bojar
2. Lucia Specia
3. Philipp Koehn
4. Markus Freitag
5. Yvette Graham
6. Matt Post
7. Barry Haddow
8. Christian Federmann

In Machine Translation

1. Philipp Koehn
2. Lucia Specia
3. Matt Post
4. Varvara Logacheva
5. Barry Haddow
6. Matteo Negri
7. Marco Turchi
8. Ondrej Bojar

Citations in the corpus

• Findings of the 2017 Conference on Machine Translation (WMT17) (2017, methodology): “As pointed out by (Chatterjee et al., 2015b), from the application point of view the task is motivated by its possible uses to: • Improve MT output by exploiting information unavailable to the decoder, or by performing deeper text analysis that is too expensive at the decoding stage; • Cope with…”
• Findings of the 2021 Conference on Machine Translation (WMT21) (2021, methodology): “As pointed out by (Chatterjee et al., 2015), from the application point of view, the task is motivated by its possible uses to:”
• Findings of the WMT 2020 Shared Task on Automatic Post-Editing (2020, methodology): “As pointed out by (Chatterjee et al., 2015), from the application point of view, the task is motivated by its possible uses to: • Improve MT output by exploiting information unavailable to the decoder, or by performing deeper text analysis that is too expensive at the decoding stage; • Cope with…”
• The FBK Participation in the WMT 2016 Automatic Post-editing Shared Task (2016, background): “A recent study on APE by Chatterjee et al. (2015b) over six language pairs have reported consistent improvement (7.3% to 14.7% TER reduction) in the quality of machine translated text across all language pairs.”
• The FBK Participation in the WMT15 Automatic Post-editing Shared Task (2015, background): “Recently, Chatterjee et al. (2015) showed a fair systematic comparison of these two approaches over multiple language pairs and revealed that inclusion of source information in the form of context-aware variant is useful to improve translation quality over standard monolingual translation approach.”