Information-Theoretic Considerations in Batch Reinforcement Learning

By Jinglin Chen, Nan Jiang. 2019

421 citations recorded by the source corpus.

Value-function approximation methods that operate in batch mode have foundational importance to reinforcement learning (RL). Finite sample guarantees for these methods often crucially rely on two types of assumptions: (1) mild distribution shift, and (2) representation conditions that are stronger than realizability. However, the necessity ("why do we need them?") and the naturalness ("when do they hold?") of such assumptions have largely eluded the literature. In this paper, we revisit these assumptions and provide theoretical results towards answering the above questions, and make steps towards a deeper understanding of value-function approximation.

Subfields: Artificial Intelligence, Computer Networks and Communications, Industrial and Manufacturing Engineering, Reinforcement Learning, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Sham M. Kakade
  3. Ruslan Salakhutdinov
  4. Nan Jiang
  5. Yoshua Bengio
  6. Pieter Abbeel
  7. Qiang Liu
  8. Lihong Li

In Computer Networks and Communications

  1. Masatoshi Uehara
  2. Nan Jiang
  3. Andrea Zanette
  4. Thorsten Joachims
  5. Jiawei Huang
  6. Yisong Yue
  7. Emma Brunskill
  8. Lihong Li

In Industrial and Manufacturing Engineering

  1. Tiancheng Yu
  2. Akshay Krishnamurthy
  3. Chi Jin
  4. Ruosong Wang
  5. Sham M. Kakade
  6. Tengyang Xie
  7. Alekh Agarwal
  8. Yu Bai

In Reinforcement Learning

  1. Sergey Levine
  2. Nan Jiang
  3. Masatoshi Uehara
  4. Lihong Li
  5. Emma Brunskill
  6. Nathan Kallus
  7. Jiawei Huang
  8. Ofir Nachum

In Information Theory

  1. Ruosong Wang
  2. Sham M. Kakade
  3. Tengyang Xie
  4. Alekh Agarwal
  5. Emma Brunskill
  6. Ruslan Salakhutdinov
  7. Zhuoran Yang
  8. Zhaoran Wang

Citations in the corpus