Defining and Characterizing Reward Gaming

By Joar Max Viktor Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov, D. Krueger. Neural Information Processing Systems, 2022

379 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/597f869a3ae3572e16de3643a7bd285ea2f1eabd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Yi Tay
2. Yoshua Bengio
3. Sergey Levine
4. Quoc V. Le
5. Percy Liang
6. Mohit Bansal
7. Armand Joulin
8. Ruslan Salakhutdinov

Citations in the corpus

• The alignment problem from a deep learning perspective (2022, background): “Gaining high reward by exploiting reward misspecification is known as reward hacking [Skalse et al., 2022].”
• Gemma: Open Models Based on Gemini Research and Technology (2024, methodology): “Similar to the SFT phase, and in order to tune hyperparameters and additionally mitigate reward hacking (Amodei et al., 2016; Skalse et al., 2022) we relied on a high capacity model as an automatic rater and computed side-by-side comparisons against baseline models.”