Counterfactual Data-Fusion for Online Reinforcement Learners

By Andrew Forney, Judea Pearl, Elias Bareinboim. 2017

94 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Causal Inference & Discovery

Citations in the corpus

Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation (2020, background): “In addition, the work by [33] leverages counterfactual data in multi-armed bandit problems under heterogeneous conditions, with counterfactual quantities being estimated by active agents empirically.”
Simpson's Paradox in COVID-19 Case Fatality Rates: A Mediation Analysis of Age-Related Causal Effects (2020, background): “in sequential decision making contexts, such as the multiarmed bandit problem (see [37], [38]).”
Causally Correct Partial Models for Reinforcement Learning (2020, background): “…a growing interest in combining causal inference with RL research in the directions of non-model-based bandit algorithms (Bareinboim et al., 2015; Forney et al., 2017; Zhang & Bareinboim, 2017; Lee & Bareinboim, 2018; Bradtke & Barto, 1996; Lu et al., 2018) and causal discovery with RL (Zhu &…”
Causal Reasoning from Meta-reinforcement Learning (2019, background): “Despite advances in both (Or-tega and Stocker, 2015; Lattimore et al., 2016; Bramley et al., 2017; Forney et al., 2017; Sen et al., 2017; Parida et al., 2018), inducing models is expensive and typically requires simplifying assumptions.”
Deconfounding Reinforcement Learning in Observational Settings (2018, background): “Forney et al. (2017) circumvented some problems caused by unobserved confounders in multi-armed bandits by using counterfactual-based decision making.”
Balanced Linear Contextual Bandits (2018, background): “There is a number of recent works which approach contextual bandits through the framework of causality (Bareinboim, Forney, and Pearl 2015), (Bareinboim and Pearl 2015), (Forney, Pearl, and Bareinboim 2017), (Lattimore, Lattimore, and Reid 2016).”
Estimation Considerations in Contextual Bandits (2017, background): “There is a number of recent works which approach contextual bandits through the framework of causality [14, 15, 31, 41].”
A Survey of Learning Causality with Data: Problems and Methods (2018, background): “Other recent work bridging RL and causality includes causal approaches for transfer learning in RL models [142] and data-fusion for online reinforcement learners [40].”
A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learning (2022, background): “…learning (de Haan et al., 2019; Zhang et al., 2020c; Kumor et al., 2021), batch learning (Bannon et al., 2020), and partial observability settings (Forney et al., 2017; Kallus & Zhou, 2018; Zhang et al., 2019a; Lu et al., 2018) in the online environment (Lyle et al., 2021; Zhang & Bareinboim,…”
MultiVerse: Causal Reasoning using Importance Sampling in Probabilistic Programming (2019, background): “, 2017), and reinforcement learning (Foerster et al., 2018; Forney et al., 2017; Buesing et al., 2019).”
A Survey of Learning Causality with Data (2018, background): “t of this approach is also mentioned. Other recent work bridging RL and causality includes causal approaches for transfer learning in RL models [142] and data-fusion for online reinforcement learners [40]. 6 CONCLUSIONS AND SOME OPEN PROBLEMS Different from previous surveys, this work aims to solve ”