A Survey of Reinforcement Learning from Human Feedback
By Timo Kaufmann, Paul Weng, Viktor Bengs, Eyke Hüllermeier. arXiv.org, 2023
338 citations recorded by the source corpus.
Reinforcement learning from human feedback (RLHF) is a variant of reinforcement learning (RL) that learns from human feedback instead of relying on an engineered reward function. Building on prior work on the related setting of preference-based reinforcement learning (PbRL), it stands at the intersection of artificial intelligence and human-computer interaction. This positioning provides a promising approach to enhance the performance and adaptability of intelligent systems while also improving the alignment of their objectives with human values. The success in training large language models (LLMs) has impressively demonstrated this potential in recent years, where RLHF has played a decisive role in directing the model's capabilities towards human objectives. This article provides an overview of the fundamentals of RLHF, exploring how RL agents interact with human feedback. While recent focus has been on RLHF for LLMs, our survey covers the technique across multiple domains. We provide our most comprehensive coverage in control and robotics, where many fundamental techniques originate, alongside a dedicated LLM section. We examine the core principles that underpin RLHF, how algorithms and human feedback work together, and the main research trends in the field. Our goal is to give researchers and practitioners a clear understanding of this rapidly growing field.
Subfields: Information Systems
View Paper: https://www.semanticscholar.org/paper/867c82da010e0cb2c69e7d8fe12f94ba6a49ee74
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Information Systems
Citations in the corpus
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning (2025, positive): “RL was introduced to LLM tuning by Ouyang et al. (2022) through RL from human feedback (RLHF) (Kaufmann et al., 2023).”
- HybridFlow: A Flexible and Efficient RLHF Framework (2024, neutral): “Researchers are actively exploring novel RLHF algorithms [41, 70, 91] and integrating traditional RL methods into RLHF domains [37].”
- ToolRL: Reward is All Tool Learning Needs (2025, positive): “The development of RL methods for LLMs has evolved from reinforcement learning from human feedback (RLHF) (Kaufmann et al., 2023) and proximal policy optimization (PPO) (Schulman et al., 2017) to more advanced techniques such as direct preference optimization (DPO) (Rafailov et al., 2023), SimPO…”