DARPA's Explainable Artificial Intelligence (XAI) Program
By D. Gunning, David William Aha. 2019
1,701 citations recorded by the source corpus.
Dramatic success in machine learning has led to a new wave of AI applications (for example, transportation, security, medicine, finance, defense) that offer tremendous benefits but cannot explain their decisions and actions to human users. DARPA’s explainable artificial intelligence (XAI) program endeavors to create AI systems whose learned models and decisions can be understood and appropriately trusted by end users. Realizing this goal requires methods for learning more explainable models, designing effective explanation interfaces, and understanding the psychologic requirements for effective explanations. The XAI developer teams are addressing the first two challenges by creating ML techniques and developing principles, strategies, and human-computer interaction techniques for generating effective explanations. Another XAI team is addressing the third challenge by summarizing, extending, and applying psychologic theories of explanation to help the XAI evaluator define a suitable evaluation framework, which the developer teams will use to test their systems. The XAI teams completed the first of this 4-year program in May 2018. In a series of ongoing evaluations, the developer teams are assessing how well their XAM systems’ explanations improve user understanding, user trust, and user task performance.
Subfields: Artificial Intelligence, Health Informatics, Information Systems and Management, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Alexey Ignatiev
- Bernhard Schölkopf
- Emilio Carrizosa
- Percy Liang
- Dolores Romero Morales
- Mohit Bansal
- Bo Li
- Yoshua Bengio
In Health Informatics
- Gesina Schwalbe
- Bettina Finzel
- Joao Marques-Silva
- Przemyslaw Biecek
- Isabel Valera
- Alexey Ignatiev
- Gilles Barthe
- Anna Y. Q. Huang
In Information Systems and Management
- Gesina Schwalbe
- Bettina Finzel
- Ashraf Abdul
- Mohan Kankanhalli
- Bo Li
- Przemyslaw Biecek
- Isabel Valera
- Alexey Ignatiev
In AI Ethics, Fairness & Society
- Bernhard Schölkopf
- Joao Marques-Silva
- Przemyslaw Biecek
- Isabel Valera
- Alexey Ignatiev
- Gilles Barthe
- Anna Y. Q. Huang
- Amir-Hossein Karimi
Citations in the corpus
- Learning Optimal Decision Trees with SAT (2018, background): “, 2012] among others), by a number of recent events dedicated to the topic [IJCAI XAI Workshop, 2017; ICML WHI Workshop, 2017; NIPS IML Symposium, 2017], but also by recent research programs [DARPA, 2016].”
- Mathematical optimization in classification and regression trees (2021, background): “Although the term Explainable Artificial Intelligence ( XAI ) was coined a while ago, it is now tracking a lot of attention from different communities, see, e.g., Barredo Arrieta et al. (2020), Gunning and Aha (2019), Holter et al. (2018), Miller (2019).”
- HARK Side of Deep Learning - From Grad Student Descent to Automated Machine Learning (2019, background): “Having such causal explanations will provide substantial leap in reaching human-like perception of AI systems and anthropomorphism [65].”
- Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics (2021, background): “The issue of ML explanation has experienced a significant surge in interest since the launch of the USA’s Defense Advanced Research Projects Agency (DARPA) initiative [12], from the international research community [13–16] and to various application domains in recent years [17–19], which can be demo”
- Model-Agnostic Counterfactual Explanations for Consequential Decisions (2019, background): “In the context of consequential decision making, it is widely agreed that a good explanation should provide answers to the following two questions [7, 12, 32]: (i) “why the model outputs a certain”
- Trustworthy AI: From Principles to Practices (2021, background): “To promote the research on AI trustworthiness, stakeholders are actively setting up various forms of collaboration such as research workshops on trustworthy AI and cooperative projects like DARPA XAI [162].”
- Algorithmic Recourse: from Counterfactual Explanations to Interventions (2020, background): “…predictor h θ , predicting the eligibility of an individual to receive a loan. the different stakeholders involved, is one of the main focuses of explainable machine learning (Kodratoff, 1994; R ¨ uping, 2006; Doshi-Velez & Kim, 2017; Lipton, 2018; Rudin, 2018; Gunning, 2019; Murdoch et al., 2019).”
- Artificial intelligence in intelligent tutoring systems toward sustainable education: a systematic review (2023, methodology): “Similar to the 4-year project conducted by DARPA (Gunning & Aha, 2019), the technical teams were requested to provide an explainable model and explanation interface to support users making decisions.”
- Explainable AI Methods - A Brief Overview (2020, background): “The term explainable AI (xAI) was coined by DARPA [28] and gained mean-while a lot of popularity.”
- A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts (2021, background): “As a consequence, XAI aims for increasing the trust in learned models and to allow for an efficient partnership between human and artificial agents [58].”
- Six Human-Centered Artificial Intelligence Grand Challenges (2023, neutral): “The development of systems that satisfy the needs of both user groups is a major challenge and most interfaces are tailored either to model developers (Gunning & Aha, 2019) or end users (Mitchell et al., 2019).”
- Survey of Explainable AI Techniques in Healthcare (2023, background): “Later, in 2021, a trust AI project showed that the XAI can be used in interdisciplinary types of application problems, including psychology, statistics, and computer science, and may provide explanations that increase the trust of users [4].”
- COGAM: Measuring and Moderating Cognitive Load in Machine Learning Model Explanations (2020, background): “The growing importance of Artificial Intelligence (AI) in society has driven calls for it to be explainable to improve user understanding and trust [3, 17, 22, 67].”
- Explainable AI: A Review of Machine Learning Interpretability Methods (2020, background): “They arranged all of the scientific work in the field of explainable AI along four main axes and stressed the need for more formalism to be introduced in the field of XAI and for more interaction between humans and machines.”