RuleMatrix: Visualizing and Understanding Classifiers with Rules

By Yao Ming, Huamin Qu, Enrico Bertini. 2018

243 citations recorded by the source corpus.

With the growing adoption of machine learning techniques, there is a surge of research interest towards making machine learning systems more transparent and interpretable. Various visualizations have been developed to help model developers understand, diagnose, and refine machine learning models. However, a large number of potential but neglected users are the domain experts with little knowledge of machine learning but are expected to work with machine learning systems. In this paper, we present an interactive visualization technique to help users with little expertise in machine learning to understand, explore and validate predictive models. By viewing the model as a black box, we extract a standardized rule-based knowledge representation from its input-output behavior. Then, we design RuleMatrix, a matrix-based visualization of rules to help users navigate and verify the rules and the black-box model. We evaluate the effectiveness of RuleMatrix via two use cases and a usability study.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

FairSight: Visual Analytics for Fairness in Decision Making (2019, methodology): “Examples include RuleMatrix [29]’s rule-based visual interface for explaining decision rules based on the secondary decision tree model, the distribution-based visual representation for global-level explanation, and Rivelo’s instance-and feature-level representation [38].”
ViCE: visual counterfactual explanations for machine learning models (2020, methodology): “For example, Manifold [24] enables the comparison of data distribution at different levels of granularity; RuleMatrix [14] visualizes extracted rules for a given model; iForest [25] and EnsembleMatrix [19] attempt to explain ensemble models.”
An Interpretable Model with Globally Consistent Explanations for Credit Risk (2018, methodology): “Our visualization interface could be extended to be much fancier, like [Ming et al., 2018] for rule-list exploration.”
COGAM: Measuring and Moderating Cognitive Load in Machine Learning Model Explanations (2020, background): “Furthermore, while visual explanations are widely used [29, 40, 42, 47], there is growing interest in verbal explanations using text [18, 27, 55].”
Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models (2019, methodology): “Existing work supports this idea by using surrogate models for improved interpretability, for example, employing model distillation [61] or visualizing extracted rule-based knowledge representations [46].”
Benchmarking and survey of explanation methods for black box models (2021, background): “RuleMatrix [88] is a post-hoc agnostic explanator tailored for the visualization of the rules extracted.”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, background): “d inspect for problems like bias [4]. Another important aspect of model visualization and inspection for domain experts is to identify and analyze failure cases of machine learning models and systems [144]. Therefore, the main challenge for data-analysis and decision-support systems is to improve mo”
The What-If Tool: Interactive Probing of Machine Learning Models (2019, methodology): “Despite attempts to use visualizations for generating these explanations via rules [21], analogies, and layer activations [20], most approaches run into issues of visual complexity, the need to support multiple exploratory workﬂows, and reliance on simple, interpretable data for meaningful insights…”
Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond (2021, background): “Ming et al. [93] developed a visualisation approach to derive rules by approximating a complicated model via model induction at different tasks such as diagnosis of breast cancer and the classification of diabetes.”
Interpretable Machine Learning - A Brief History, State-of-the-Art and Challenges (2020, methodology): “Many IML methods are surrogate model approaches [89,75,72,95,34,10,18,61] and differ, e.”