Bandits with Unobserved Confounders: A Causal Approach

By Elias Bareinboim, Andrew Forney, Judea Pearl. 2015

202 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Causal Inference & Discovery

Citations in the corpus

Toward Causal Representation Learning (2021, background): “An emerging line of work devoted to the intersection of RL and causality includes [1], [13], [22], [38], [51], [165], [276].”
Towards Causal Representation Learning (2021, background): “An emerging line of work devoted to the intersection of RL and causality includes [13, 21, 164, 37, 50, 275, 1].”
Simpson's Paradox in COVID-19 Case Fatality Rates: A Mediation Analysis of Age-Related Causal Effects (2020, background): “in sequential decision making contexts, such as the multiarmed bandit problem (see [37], [38]).”
Causally Correct Partial Models for Reinforcement Learning (2020, neutral): “…this work, there has been a growing interest in combining causal inference with RL research in the directions of non-model-based bandit algorithms (Bareinboim et al., 2015; Forney et al., 2017; Zhang & Bareinboim, 2017; Lee & Bareinboim, 2018; Bradtke & Barto, 1996; Lu et al., 2018) and causal…”
Causality for Machine Learning (2019, background): “Recent work devoted to the field between RL and causality includes (Bareinboim et al., 2015; Bengio et al., 2017; Lu et al., 2018; Buesing et al., 2018; Dasgupta et al., 2019; Zhang and Bareinboim, 2019).”
Deconfounding Reinforcement Learning in Observational Settings (2018, background): “To the best of our knowledge, however, little work has been done in this promising area of integrating RL with causal inference (Bareinboim et al., 2015; Forney et al., 2017; Buesing et al., 2019).”
Balanced Linear Contextual Bandits (2018, background): “There is a number of recent works which approach contextual bandits through the framework of causality (Bareinboim, Forney, and Pearl 2015), (Bareinboim and Pearl 2015), (Forney, Pearl, and Bareinboim 2017), (Lattimore, Lattimore, and Reid 2016).”
Estimation Considerations in Contextual Bandits (2017, background): “There is a number of recent works which approach contextual bandits through the framework of causality [14, 15, 31, 41].”
Fairness in Decision-Making - The Causal Explanation Formula (2018, methodology): “Further, the distribution P (yx|x′) and P (yx,w|x′, w′) can be estimated following the new counterfactual randomization procedure introduced in (Bareinboim, Forney, and Pearl 2015).”
A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learning (2022, background): “Lee & Bareinboim (2018); Bareinboim et al. (2015); Lattimore et al. (2016); Moziﬁan et al. (2020); Volodin et al. (2020) also explore how to design an optimal intervention policy in bandits or RL settings.”
Causal Induction from Visual Observations for Goal Directed Tasks (2019, unknown): “Both causal induction (constructing SCMs from observational data) (Shimizu et al., 2006; Hoyer et al., 2009; Peters et al., 2014; Ortega & Braun, 2014) and causal inference (using SCMs to estimate causal effects) (Bareinboim et al., 2015; Bareinboim & Pearl, 2016) algorithms have been developed.”
Counterfactual Data-Fusion for Online Reinforcement Learners (2017, methodology): “To tie these concepts to the MABUC problem, one important tool introduced in [2] is what is known as the agent’s intent.”
Causal Analysis of Agent Behavior for AI Safety (2021, background): “Resolving the use of spurious correlations in learned policies is ongoing research—see for instance (Bareinboim et al., 2015; Arjovsky et al., 2019; Volodin et al., 2020).”
A Survey of Learning Causality with Data: Problems and Methods (2018, methodology): “Specifically, the Causal Thompson Sampling algorithm [16] is proposed to deal with unobserved confounders in MAB problems.”
Contextual Bandits with Latent Confounders: An NMF Approach (2016, background): “In [5], the authors investigate a similar latent confounder model.”
Causal Bandits: Learning Good Interventions via Causal Inference (2016, background): “Bareinboim et al. (2015) demonstrate that in the presence of confounding variables the value that a variable would have taken had it not been intervened on can provide important contextual information.”
A Survey of Learning Causality with Data (2018, background): “t time step based on whether the customer flow increases or not. Unobserved Confounders in RL. Unobserved confounders raise issues of learning policies for RL models such as multi-armed bandits (MAB) [16]. Without knowing the causal model, MAB algorithms can perform as badly as randomly taking an ac”
Transfer Learning in Multi-Armed Bandits: A Causal Approach (2017, unknown): “Connections between causal models with UCs and RL were first established in [Bareinboim et al., 2015].”