The challenge of crafting intelligible intelligence

By Daniel S. Weld, Gagan Bansal. 2018

265 citations recorded by the source corpus.

To trust the behavior of complex AI algorithms, especially in mission-critical settings, they must be made intelligible.

Subfields: Artificial Intelligence, Information Systems and Management, Management Science and Operations Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems and Management

In Management Science and Operations Research

In Safety, Robustness & Interpretability

Citations in the corpus

Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI (2020, background): “26 The training data itself, in addition to the mechanics of training, also contribute to the objective mismatch problem.”
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2020, background): “y one is increasing the effectiveness of decision making, while reducing the cost of operations. But increased use of AI brings several well-known dangers as well, including bias and erratic behavior [64]. Many researchers have argued that if AI could explain its reasoning, these , Vol. 1, No. 1, Ar”
Trustworthy AI: From Principles to Practices (2021, methodology): “Various approaches of interaction have been studied to enhance the explainability of AI, including the visualization of ML models [72] and interactive parameter-tuning [351].”
Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey (2020, background): “Recent interest in XAI, even from governments especially with the European General Data Protection Regulation (GDPR) [21] regulation, shows the important realization of the ethics [22]–[26], trust [27]–[29], bias [30]–[33] of AI, as well as the”
Principles and Practice of Explainable Machine Learning (2020, background): “She again tries various candidates and she finds out that Random Forests achieve the 5Note that this informal view encourages a notional plot of explainability versus accuracy, as is common in informal discussions on the challenge of XAI [32, 88].”
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations (2020, background): “These contributions are important to make progress towards creating conversational explanations between machines and human users [53, 33].”
Towards Trustable Explainable AI (2020, background): “Explainable AI is an emerging ﬁeld, with a growing number of areas of research [ Weld and Bansal, 2019 ] .”
Explainability fact sheets: a framework for systematic assessment of explainable approaches (2019, methodology): “Adjusting an explanation to users requires the explainability technique to model their background knowledge and mental model [48, 55].”
Questioning the AI: Informing Design Practices for Explainable AI User Experiences (2020, background): “Prior work repeatedly pointed out that a prerequisite for explanations to be truly human-like is to be interactive [61, 62, 93], because explanation is a grounding process where people incrementally close the belief gaps.”
Expanding Explainability: Towards Social Transparency in AI systems (2021, background): “Recent work has also begun to explore interactive explanations that could address users’ follow-up questions as a way to fill individual knowledge gaps [91, 104].”