Lipschitz Continuity of Value Functions in Markovian Decision Processes

By Karl Hinderer. 2005

73 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Industrial and Manufacturing Engineering, Management Science and Operations Research, Reinforcement Learning, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Yuanzhi Li
2. Tengyu Ma
3. Trevor Darrell
4. Marcello Restelli
5. Marc G. Bellemare
6. Luca Bascetta
7. Huazhe Xu
8. Yuandong Tian

In Industrial and Manufacturing Engineering

1. Trevor Darrell
2. Marcello Restelli
3. Marc G. Bellemare
4. Luca Bascetta
5. Saurabh Kumar
6. Yuandong Tian
7. Matteo Pirotta
8. Ofir Nachum

In Management Science and Operations Research

1. Carles Gelada
2. Ofir Nachum
3. Saurabh Kumar
4. Jacob Buckman
5. Trevor Darrell
6. Marcello Restelli
7. Marc G. Bellemare
8. Tengyu Ma

In Reinforcement Learning

1. Sergey Levine
2. Pieter Abbeel
3. Ofir Nachum
4. Tengyu Ma
5. Marc G. Bellemare
6. Carles Gelada
7. Huazhe Xu
8. Saurabh Kumar

In Statistical Learning Theory

1. Yuanzhi Li
2. Tengyu Ma
3. Trevor Darrell
4. Marcello Restelli
5. Marc G. Bellemare
6. Luca Bascetta
7. Yuandong Tian
8. Carles Gelada

Citations in the corpus

• DeepMDP: Learning Continuous Latent Space Models for Representation Learning (2019, background): “1 and if for all a P A : Several works have studied Lipschitz norm constraints on the transition and reward functions (Hinderer, 2005; Asadi et al., 2018) to provide conditions for value functions to be Lipschitz.”
• Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees (2018, background): “…RL mostly focuses on either the finite-state MDPs (Jaksch et al., 2010; Bartlett & Tewari, 2009; Fruit et al., 2018; Lakshmanan et al., 2015; Hinderer, 2005; Pirotta et al., 2015; 2013), or the linear parametrization of the dynamics, policy, or value function (Abbasi-Yadkori & Szepesvári,…”
• Policy gradient in Lipschitz Markov Decision Processes (2015, methodology): “Finally, the choice of the L1Wasserstein metric rather than other sophisticated distribution distances is motivated by the fact that it has been frequently used for MDPs (Rachelson and Lagoudakis 2010; Hinderer 2005; Ferns et al. 2005).”
• Lipschitz Continuity in Model-based Reinforcement Learning (2018, unknown): “…the theory of reinforcement learning (Bertsekas, 1975; Bertsekas & Tsitsiklis, 1995; Littman & Szepesv ´ ari, 1996; M ¨ uller, 1996; Ferns et al., 2004; Hinderer, 2005; Rachelson & Lagoudakis, 2010; Szepesv´ari, 2010; Pazis & Parr, 2013; Pirotta et al., 2015; Pires & Szepesv ´ ari, 2016;…”