The Alignment Problem for Bayesian History-Based Reinforcement Learners∗

By Tom Everitt, Marcus Hutter. 2019

Subfields: Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

1. Jan Leike
2. Tom Everitt
3. David Krueger
4. Shane Legg
5. Miljan Martic
6. Vishal Maini
7. Tegan Maharaj
8. Sergey Levine

Citations in the corpus

• Scalable agent alignment via reward modeling: a research direction (2018, methodology): “Sources of undesired loopholes are reward gaming (Leike et al., 2017) where the agent exploits some misspecification in the reward function, and reward tampering (Everitt & Hutter, 2018) where the agent interferes with the process computing the reward.”
• Hidden Incentives for Auto-Induced Distributional Shift (2020, background): “Examples include tampering with human-generated reward signals (Everitt & Hutter, 2018) (e.g. selecting news articles which manipulate user interests), and creating “self-fulfilling prophecies” (e.g. driving up the value of a held asset by publicly predicting its value will increase (Armstrong &…”