A Shared Attention Mechanism for Interpretation of Neural Automatic Post-Editing Systems

By Inigo Jauregi Unanue, Ehsan Zare Borzeshi, Massimo Piccardi. 2018

5 citations recorded by the source corpus.

Automatic post-editing (APE) systems aim to correct the systematic errors made by machine translators. In this paper, we propose a neural APE system that encodes the source (src) and machine translated (mt) sentences with two separate encoders, but leverages a shared attention mechanism to better understand how the two inputs contribute to the generation of the post-edited (pe) sentences. Our empirical observations have showed that when the mt is incorrect, the attention shifts weight toward tokens in the src sentence to properly edit the incorrect translation. The model has been trained and evaluated on the official data from the WMT16 and WMT17 APE IT domain English-German shared tasks. Additionally, we have used the extra 500K artificial data provided by the shared task. Our system has been able to reproduce the accuracies of systems trained with the same data, while at the same time providing better interpretability.

Subfields: Safety, Robustness & Interpretability, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

In Machine Translation

Citations in the corpus

Analysis Methods in Neural Language Processing: A Survey (2018, background): “…have produced visualizations ranging from tasks like NLI (Rocktäschel et al., 2016; Yin et al., 2016), summarization (Rush et al., 2015), MT post-editing (Jauregi Unanue et al., 2018), and morphological inﬂection (Aharoni and Goldberg, 2017), to matching users on social media (Tay et al., 2018).”