RLHF Workflow: From Reward Modeling to Online RLHF

By Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Ying-Bo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, Tong Zhang. 2024

Subfields: Alignment & RLHF

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Jun Qin
2. Archit Sharma
3. Judy Hanwen Shen

Citations in the corpus

• S-SPPO: Semantic-Calibrated Self-Play Preference Optimization (2026, unknown): “…have extended this idea to an iterative, online setting, either by continually updating the reference model to the latest policy or by generating fresh preference pairs at each training iteration (Xiong et al., 2024; Dong et al., 2024; Rosset et al., 2024; Kim et al., 2025; Wu et al., 2025).”
• Towards Data-Centric RLHF: Simple Metrics for Preference Dataset Comparison (2024, methodology): “These datasets are selected based on their frequent use in prior works [4, 8] 2 .”