A Reinforcement Learning Approach to Interactive-Predictive Neural Machine Translation

By Tsz Kin Lam, Julia Kreutzer, Stefan Riezler. 2018

33 citations recorded by the source corpus.

We present an approach to interactivepredictive neural machine translation that attempts to reduce human effort from three directions: Firstly, instead of requiring humans to select, correct, or delete segments, we employ the idea of learning from human reinforcements in form of judgments on the quality of partial translations. Secondly, human effort is further reduced by using the entropy of word predictions as uncertainty criterion to trigger feedback requests. Lastly, online updates of the model parameters after every interaction allow the model to adapt quickly. We show in simulation experiments that reward signals on partial translations significantly improve character F-score and BLEU compared to feedback on full translations only, while human effort can be reduced to an average number of 5 feedback requests for every input.

Subfields: Artificial Intelligence, Reinforcement Learning, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Stefan Riezler
2. Percy Liang
3. Julia Kreutzer
4. Mohit Bansal
5. Yejin Choi
6. Jianfeng Gao
7. Tatsunori B. Hashimoto
8. Luke S. Zettlemoyer

In Reinforcement Learning

1. Julia Kreutzer
2. Stefan Riezler
3. Carolin (Haas) Lawrence
4. Shahram Khadivi
5. Evgeny Matusov
6. Pavel Petrushkov
7. Percy Liang
8. Carlos Guestrin

In Machine Translation

1. Percy Liang
2. Carolin (Haas) Lawrence
3. Stefan Riezler
4. Julia Kreutzer

Citations in the corpus

• Offline Reinforcement Learning from Human Feedback in Real-World Sequence-to-Sequence Tasks (2020, background): “The majority of RL research in NLP has focused on learning from online feedback (Sokolov et al., 2016; He et al., 2016; Li et al., 2016; Bahdanau et al., 2017; Nguyen et al., 2017; Nogueira and Cho, 2017; Lam et al., 2018).”
• Learning from Chunk-based Feedback in Neural Machine Translation (2018, background): “(Lam et al., 2018) extended the work of (Nguyen et al.”
• AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback (2023, methodology): “RL research applied to NLP has a long history [75, 62, 27, 49, 42, 30, 29, 53, 61], and we expect future work in this direction to benefit from the ideas and artifacts in AlpacaFarm.”