Defining Explainable AI for Requirements Analysis
By Raymond Ka-Man Sheh, Isaac Monteath. 2018
42 citations recorded by the source corpus.
Explainable artificial intelligence (XAI) has become popular in the last few years. The artificial intelligence (AI) community in general, and the machine learning (ML) community in particular, is coming to the realisation that in many applications, for AI to be trusted, it must not only demonstrate good performance in its decisionmaking, but it also must explain these decisions and convince us that it is making the decisions for the right reasons. However, different applications have different requirements on the information required of the underlying AI system in order to convince us that it is worthy of our trust. How do we define these requirements? In this paper, we present three dimensions for categorising the explanatory requirements of different applications. These are Source, Depth and Scope. We focus on the problem of matching up the explanatory requirements of different applications with the capabilities of underlying ML techniques to provide them. We deliberately avoid including aspects of explanation that are already well-covered by the existing literature and we focus our discussion on ML although the principles apply to AI more broadly.
Subfields: Artificial Intelligence, Safety Research
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Safety Research
Citations in the corpus
- Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI (2019, neutral): “…and Treu, 1977; Antunes, Herskovic, Ochoa, and Pino, 2012; Kulesza, et al., 2010, 2012, 2011, 2013, 2015; Nataksu, 2004; Ribiero, et al., 2016a,b; Sheh and Monteath, 2008; Treu, 1998; Wickens, 1994), and the challenges for the evaluation of explanation systems (Doshi-Velez and Kim, 2017ab;…”
- Explaining Explanations to Society (2019, background): “Other work examining best practices for explanation [30] provides a set of categories, but does not evaluate the questions that explanatory systems should be able to answer; which is necessary for policy makers and societal trust in DNN decision processes.”
- Software Engineering for AI-Based Systems: A Survey (2021, background): “Further examples include methodological aids such as the notion of supplier’s declaration of conformity [12] or conceptual frameworks to improve the specification of requirements for explainable [184] or safe [15] AI-based systems.”