Counterfactual Learning for Machine Translation: Degeneracies and Solutions

By Carolin (Haas) Lawrence, Pratik Gajane, Stefan Riezler. 2017

7 citations recorded by the source corpus.

Counterfactual learning is a natural scenario to improve web-based machine translation services by offline learning from feedback logged during user interactions. In order to avoid the risk of showing inferior translations to users, in such scenarios mostly exploration-free deterministic logging policies are in place. We analyze possible degeneracies of inverse and reweighted propensity scoring estimators, in stochastic and deterministic settings, and relate them to recently proposed techniques for counterfactual learning under deterministic logging.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback (2018, methodology): “This results in a deterministic propensity matching (DPM) objec-tive (Lawrence et al., 2017b), without the possibility to correct the sampling bias of the logging policy by inverse propensity scoring (Rosenbaum and Rubin, 1983): This objective can show degenerate behavior in that it overﬁts to the…”
Can Neural Machine Translation be Improved with User Feedback? (2018, methodology): “…signal for response prediction in online advertising (Bottou et al., 2013), bandit learning for machine translation has so far been restricted to simulation experiments (Sokolov et al., 2016b; Lawrence et al., 2017b; ∗The work for this paper was done while the first author was an intern at eBay.”
Offline Reinforcement Learning from Human Feedback in Real-World Sequence-to-Sequence Tasks (2020, methodology): “for a collected log Dlog with δ ∈ [0, 1] because IPS and DPM can trivially be minimized by setting all probabilities in the dataset D to 1 for any δt > 0 (Lawrence et al., 2017a).”