Provably Good Batch Off-Policy Reinforcement Learning Without Great Exploration

98 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Continuous Doubly Constrained Batch Reinforcement Learning (2021, methodology): “[38] considers a pessimistic and conservative approach to update Q-value by utilizing the marginalized state-action distribution of available data.”
Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism (2021, methodology): “Examples include model-based methods that learn pessimistic MDPs [37], [39], [40], model-free methods that reduce the Q-functions on unseen state-action pairs [41]–[43], and policy-based methods that minimize the divergence between the learned policy and the behavior policy [20], [21], [44]–[49].”
Bellman-consistent Pessimism for Offline Reinforcement Learning (2021, methodology): “The closest prior result to our work is that of [Liu et al., 2020], who develop a pessimistic estimator that truncates Bellman backups from state-action pairs infrequently visited by μ, and analyzes the resulting pessimistic policy and value iteration algorithms under general function approximation.”
Representation Learning for Online and Offline RL in Low-rank MDPs (2021, background): “Empirically, it can work on simulation control tasks (Kidambi et al., 2020; Yu et al., 2020; Kumar et al., 2020; Liu et al., 2020; Chang et al., 2021).”