Offline Reinforcement Learning from Human Feedback in Real-World Sequence-to-Sequence Tasks
By Julia Kreutzer, Stefan Riezler, Carolin (Haas) Lawrence. 2020
18 citations recorded by the source corpus.
Large volumes of interaction logs can be collected from NLP systems that are deployed in the real world. How can this wealth of information be leveraged? Using such interaction logs in an offline reinforcement learning (RL) setting is a promising approach. However, due to the nature of NLP tasks and the constraints of production systems, a series of challenges arise. We present a concise overview of these challenges and discuss possible solutions.
Subfields: Artificial Intelligence, Industrial and Manufacturing Engineering, Natural Language Processing, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Hannaneh Hajishirzi
- Yejin Choi
- Jack Hessel
- Prithviraj Ammanabrolu
- Mohit Bansal
- Luke S. Zettlemoyer
- Rafet Sifa
- Christian Bauckhage
In Industrial and Manufacturing Engineering
In Natural Language Processing
- Hannaneh Hajishirzi
- Yejin Choi
- Jack Hessel
- Prithviraj Ammanabrolu
- Mohit Bansal
- Luke S. Zettlemoyer
- Kianté Brantley
- Christian Bauckhage
In Reinforcement Learning
- Hannaneh Hajishirzi
- Yejin Choi
- Kianté Brantley
- Christian Bauckhage
- Prithviraj Ammanabrolu
- Rafet Sifa
- Rajkumar Ramamurthy
- Jack Hessel
Citations in the corpus
- Is Reinforcement Learning (Not) for Natural Language Processing?: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization (2022, background): “, 2022) but progress in this line of work is heavily hindered by a lack of open-source benchmarks and algorithmic implementations—resulting in perception that RL is a challenging paradigm for NLP (Choshen et al., 2020; Kreutzer et al., 2021).”