Explainable Reinforcement Learning: A Survey

By Erika Puiutta, Eric M. S. P. Veith. International Cross-Domain Conference on Machine Learning and Knowledge Extraction, 2020

321 citations recorded by the source corpus.

Explainable Artificial Intelligence (XAI), i.e., the development of more transparent and interpretable AI models, has gained increased traction over the last few years. This is due to the fact that, in conjunction with their growth into powerful and ubiquitous tools, AI models exhibit one detrimential characteristic: a performance-transparency trade-off. This describes the fact that the more complex a model's inner workings, the less clear it is how its predictions or decisions were achieved. But, especially considering Machine Learning (ML) methods like Reinforcement Learning (RL) where the system learns autonomously, the necessity to understand the underlying reasoning for their decisions becomes apparent. Since, to the best of our knowledge, there exists no single work offering an overview of Explainable Reinforcement Learning (XRL) methods, this survey attempts to address this gap. We give a short summary of the problem, a definition of important terms, and offer a classification and assessment of current XRL methods. We found that a) the majority of XRL methods function by mimicking and simplifying a complex model instead of designing an inherently simple one, and b) XRL (and XAI) methods often neglect to consider the human side of the equation, not taking into account research from related fields like psychology or philosophy. Thus, an interdisciplinary effort is needed to adapt the generated explanations to a (non-expert) human user in order to effectively progress in the field of XRL and XAI in general.

Subfields: Artificial Intelligence, Safety Research

View Paper: https://www.semanticscholar.org/paper/e1b77eb03d753da13d1c6ce7cb45dd98e64c7fb6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

Citations in the corpus

Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond (2021, background): “DRL is also applicable and shows potentials of application in healthcare, finance and business management [13, 105], where human security and property safety issues should be considered, leading to the demands of explainable RL [134].”
A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, background): “Task visual tasks [117,118,99,7,2,84,147] reinforcement learning [107,65]”
Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial (2020, background): “We refer the reader to [67], [68], and the references therein for a detailed overview of”
Causal Analysis of Agent Behavior for AI Safety (2021, background): “Other recent approaches for analyzing AI systems have mostly focused on white-box approaches for improving understanding (see for instance Mott et al., 2019; Verma et al., 2018; Montavon et al., 2018; Puiutta & Veith, 2020) and developing safety guarantees (Uesato et al., 2018).”