Preference-based reinforcement learning: evolutionary direct policy search using a preference-based racing algorithm
By Róbert Istvan Busa-Fekete, Balázs Szörényi, Paul Weng, Weiwei Cheng, Eyke Hüllermeier. 2014
73 citations recorded by the source corpus.
Subfields: Artificial Intelligence, Computational Theory and Mathematics, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Mohit Bansal
- Pieter Abbeel
- Percy Liang
- Douwe Kiela
- Yoshua Bengio
- Luke S. Zettlemoyer
- Ruslan Salakhutdinov
In Computational Theory and Mathematics
- Rafael Rafailov
- Shamim Nemati
- Christopher D. Manning
- Mark Rowland
- Michael I. Jordan
- Douwe Kiela
- Dan Jurafsky
- Bilal Piot
In Reinforcement Learning
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Stefano Ermon
- Aldo Pacchiano
- Bilal Piot
- Michael I. Jordan
- Rafael Rafailov
Citations in the corpus
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model (2023, methodology): “Similarly, preference-based RL (PbRL) learns from binary preferences generated by an unknown ‘scoring’ function rather than rewards [9, 35].”
- Dueling RL: Reinforcement Learning with Trajectory Preferences (2021, methodology): “…applied PbRL problem inspired by the problems of reward hacking, reward shaping, difficulty to model infinite rewards or multiobjective trade-offs (Busa-Fekete et al., 2014; Wirth et al., 2016, 2017; Christiano et al., 2017) etc. Novoseller et al. (2019) made the first attempt to analyze the…”
- Reinforcement Learning in Healthcare: A Survey (2019, methodology): “Busa-Fekete et al. [24] introduced a preference-based variant of a direct PS method in the medical treatment design for cancer clinical trials.”
- Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisons (2023, background): “And a majority of the empirical literature focuses on the framework of learning to rank (MLE) under general function approximation, especially when the reward is parameterized by a neural network (Liu et al., 2009; Xia et al., 2008; Cao et al., 2007; Christiano et al., 2017a; Ouyang et al., 2022; Br”
- Model Alignment as Prospect Theoretic Optimization (2024, background): “This single stage of optimization distinguishes DPO from the conventional approach in preference-based RL, which learns a reward and then fits the policy to those rewards (Jain et al., 2013; Busa-Fekete et al., 2014).”
- A General Theoretical Paradigm to Understand Learning from Human Preferences (2023, background): “Notable exceptions, that consider specific special cases, are (Wang et al., 2023; Chen et al., 2022) and prior work on preference-based (Busa-Fekete et al., 2014, 2013) and dueling bandits and RL (Novoseller et al., 2020; Pacchiano et al., 2023).”