On the Theory of Reinforcement Learning with Once-per-Episode Feedback

By Niladri S. Chatterji, Aldo Pacchiano, Peter L. Bartlett, Michael I. Jordan. 2021

37 citations recorded by the source corpus.

We study a theory of reinforcement learning (RL) in which the learner receives binary feedback only once at the end of an episode. While this is an extreme test case for theory, it is also arguably more representative of real-world applications than the traditional requirement in RL practice that the learner receive feedback at every time step. Indeed, in many real-world applications of reinforcement learning, such as self-driving cars and robotics, it is easier to evaluate whether a learner’s complete trajectory was either “good” or “bad,” but harder to provide a reward signal at each step. To show that learning is possible in this more challenging setting, we study the case where trajectory labels are generated by an unknown parametric model, and provide a statistically and computationally efficient algorithm that achieves sub-linear regret.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Management Science and Operations Research, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Management Science and Operations Research

In Reinforcement Learning

Citations in the corpus

Dueling RL: Reinforcement Learning with Trajectory Preferences (2021, methodology): “Throughout this section we will make heavy use of some of the results from Chatterji et al. (2021).”
Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisons (2023, background): “Chatterji et al. (2022) studies a close setting where in each episode only binary feedback is received.”