Metrics for Finite Markov Decision Processes

By N. Ferns, Prakash Panangaden, Doina Precup. 2004

373 citations recorded by the source corpus.

We present metrics for measuring the similarity of states in a finite Markov decision process (MDP). The formulation of our metrics is based on the notion of bisimulation for MDPs, with an aim towards solving discounted infinite horizon reinforcement learning tasks. Such metrics can be used to aggregate states, as well as to better structure other value function approximators (e.g., memory-based or nearest-neighbor approximators). We provide bounds that relate our metric distances to the optimal values of states in the given MDP.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Reinforcement Learning, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Pieter Abbeel
  3. Yoshua Bengio
  4. Ofir Nachum
  5. Chelsea Finn
  6. Ruslan Salakhutdinov
  7. Mohit Bansal
  8. Joelle Pineau

In Computational Theory and Mathematics

  1. Sergey Levine
  2. Ofir Nachum
  3. Pablo Samuel Castro
  4. Doina Precup
  5. Pieter Abbeel
  6. Joelle Pineau
  7. Marc G. Bellemare
  8. Michael L. Littman

In Reinforcement Learning

  1. Sergey Levine
  2. Ofir Nachum
  3. Pieter Abbeel
  4. Doina Precup
  5. Marc G. Bellemare
  6. Lihong Li
  7. Shixiang Shane Gu
  8. Chelsea Finn

In Game & Decision Theory

  1. Pablo Samuel Castro
  2. Doina Precup
  3. Michael L. Littman
  4. Ofir Nachum
  5. Prashant Doshi
  6. Mengjiao Yang
  7. Gheorghe Comanici
  8. Saurabh Arora

Citations in the corpus