Glass-Box: Explaining AI Decisions With Counterfactual Statements Through Conversation With a Voice-enabled Virtual Assistant

By Kacper Sokol, Peter A. Flach. 2018

90 citations recorded by the source corpus.

The prevalence of automated decision making, influencing important aspects of our lives -- e.g., school admission, job market, insurance and banking -- has resulted in increasing pressure from society and regulators to make this process more transparent and ensure its explainability, accountability and fairness. We demonstrate a prototype voice-enabled device, called Glass-Box, which users can question to understand automated decisions and identify the underlying model's biases and errors. Our system explains algorithmic predictions with class-contrastive counterfactual statements (e.g., ``Had a number of conditions been different:...the prediction would change...''), which show a difference in a particular scenario that causes an algorithm to ``change its mind''. Such explanations do not require any prior technical knowledge to understand, hence are suitable for a lay audience, who interact with the system in a natural way -- through an interactive dialogue. We demonstrate the capabilities of the device by allowing users to impersonate a loan applicant who can question the system to understand the automated decision that he received.

Subfields: Artificial Intelligence, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

Citations in the corpus

Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review (2020, unknown): “Sokol and Flach [306] list several desirable properties of counterfactuals inspired from Miller [238] and state how the method of flipping logical conditions in a decision tree satisfies most of them.”
A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence (2021, methodology): “, see [155]) answering the question: ‘‘How is the prediction altered when the observation changes, given a classifier and an observation?’’ Furthermore, these changes are normally expected to”
The Use and Misuse of Counterfactuals in Ethical Machine Learning (2021, background): “The use of counterfactuals has become increasingly popular in the machine learning community for many reasons such as making sense of algorithmic fairness or explainability in automated decision-making for consequential social contexts [4, 9, 14, 18, 29, 35, 42, 47, 51].”