Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs

By Chris Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, Yahui Zhou. arXiv.org, 2024

317 citations recorded by the source corpus.

In this report, we introduce a collection of methods to enhance reward modeling for LLMs, focusing specifically on data-centric techniques. We propose effective data selection and filtering strategies for curating high-quality open-source preference datasets, culminating in the Skywork-Reward data collection, which contains only 80K preference pairs -- significantly smaller than existing datasets. Using this curated dataset, we developed the Skywork-Reward model series -- Skywork-Reward-Gemma-27B and Skywork-Reward-Llama-3.1-8B -- with the former currently holding the top position on the RewardBench leaderboard. Notably, our techniques and datasets have directly enhanced the performance of many top-ranked models on RewardBench, highlighting the practical impact of our contributions in real-world preference learning applications.

View Paper: https://www.semanticscholar.org/paper/7796f56c7b9c152ac9573c8bb34f716aa78b4e6d

Citations in the corpus

• Visual-RFT: Visual Reinforcement Fine-Tuning (2025, unknown): “Unlike previous Reinforcement Learning from Human Feedback (RLHF) [19, 28, 49], which relies on a trained reward model, RLVR instead uses a direct verification function to assess correctness.”
• VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model (2025, unknown): “More recently, DeepSeek R1 [17] further advanced this insight by showing that simple rule-based rewards, without the need for a separate learned reward model [28, 39, 56], are sufficient to autonomously equip LLMs with complex reasoning performance.”