RLAIF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
By Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune, Abhinav Rastogi. arXiv.org, 2023
404 citations recorded by the source corpus.
Subfields: Artificial Intelligence
View Paper: https://www.semanticscholar.org/paper/cb587eaea753ee38013afb7e5b6bc8fba1248d04
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Maosong Sun
- Jianfeng Gao
- Zhiyuan Liu
- Richard Socher
- Mohit Bansal
- Caiming Xiong
- Graham Neubig
- Percy Liang
Citations in the corpus
- Self-Rewarding Language Models (2024, neutral): “Lee et al. [2023] compare RLAIF and RLHF procedures and find the methods they compare perform roughly equally.”
- HybridFlow: A Flexible and Efficient RLHF Framework (2024, positive): “RLHF aligns the linguistic space of LLMs with human values, using a set of human-ranked candidates of given prompts [7, 19, 41, 43, 55, 70, 91].”
- UltraFeedback: Boosting Language Models with High-quality Feedback (2023, neutral): “Although many researchers pointed out that GPT-4 is capable of providing meticulous judgments and feedback (Dubois et al., 2023; Bai et al., 2022b; Lee et al., 2023; Zheng et al., 2023a), and exhibits high agreement with human preference, we still concern whether the GPT-4 preferences are qualified.”
- From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge (2024, positive): “Several studies have also incorporated swapping operations in self-alignment (Lee et al., 2023; Sun et al., 2024; Lee et al., 2024) to obtain more accurate pairwise feed-back from the judge LLM. Zhu et al. (2024a) proposed a CoT-like prompting technique to mitigate the positional bias by asking the…”
- ULTRAFEEDBACK: Boosting Language Models with Scaled AI Feedback (2023, positive): “Specifically, we explore AI feedback (Bai et al., 2022b; Lee et al., 2023), which substitutes human annotators with advanced LLMs. Compared with human feedback, AI feed-back is more scalable , which means (1) it is easier to collect and expand with lower cost; (2) its quality improves as the LLM…”
- GraphGPT: Graph Instruction Tuning for Large Language Models (2023, neutral): “This significant reduction in token consumption translates to a substantial decrease in training and inference resource requirements.”
- Large Language Models: A Survey (2024, positive): “Reinforcement learning from AI feedback on the other hand, directly connects a pretrained and well-aligned model to the LLM and helps it to learn from larger and more aligned models [138].”
- A Survey on Large Language Models for Code Generation (2024, neutral): “LLM evaluation metrics are criteria used to quantify the performance of LLM systems in aspects such as correctness of the answers, semantic similarity, and hallucination [50].”