Anchors: High-Precision Model-Agnostic Explanations

By Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin. 2018

2,399 citations recorded by the source corpus.

We introduce a novel model-agnostic system that explains the behavior of complex models with high-precision rules called anchors, representing local, "sufficient" conditions for predictions. We propose an algorithm to efficiently compute these explanations for any black-box model with high-probability guarantees. We demonstrate the flexibility of anchors by explaining a myriad of different models for different domains and tasks. In a user study, we show that anchors enable users to predict how a model would behave on unseen instances with less effort and higher precision, as compared to existing linear explanations or no explanations.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “Others: [67, 224, 230, 234, 235, 236, 237]”
On the Art and Science of Machine Learning Explanations (2018, background): “While residual and sensitivity analysis typically play a role in model debugging exercises, readers are encouraged to investigate newer methods such as anchors and burgeoning work in adversarial examples [24], [32].”
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, background): “Newer, related, and highly anticipated work from the creators of LIME, called anchors [85], extends LIME using decision rules.”
Local Rule-Based Explanations of Black Box Decision Systems (2018, methodology): “[31] uses a bandit algorithm that randomly constructs the anchors with the highest coverage and respecting a precision threshold.”
A Survey of Methods for Explaining Black Box Models (2018, methodology): “An extension of LIME using decision rules as local interpretable classifier cl is presented in Reference [99].”
A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective (2018, methodology): “The difference between the models can identified using model interpretation tools like LIME [120] and Anchors [121] and be used by the user to further debug and improve the labeling functions.”
Computing Optimal Decision Sets with SAT (2020, background): “Hence the rising interest in explainable artificial intelligence (XAI) [4, 14, 16, 18, 22, 23, 26, 29, 30, 36–38, 42, 43, 48, 49, 52], including research programs [3, 24] and legislation [17, 21].”
"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations (2019, background): “, ensuring that the explanations accurately mimic the predictions of black box model [12, 17, 18].”
Interpretable and Steerable Sequence Learning via Prototypes (2019, methodology): “Recently, model distillation is also used to explain deep sequence models [5, 23, 26, 27].”
An Interpretable Model with Globally Consistent Explanations for Credit Risk (2018, background): “they provide useful patterns, which has been a popular form of explanation in the state-of-the-art literature on model explanations [7, 4, 13].”
BERTology Meets Biology: Interpreting Attention in Protein Language Models (2020, background): “Methods that learn to explain by extracting rationales from the input and only using those rationales to make a prediction are faithful by design [8, 13, 43, 62].”
Actionable Recourse in Linear Classification (2018, background): “Other topics that are broadly related to recourse in machine learning include: inverse classification, which aims to determine how the inputs to a model can be manipulated to obtain a desired outcome [1, 10]; strategic classification, which considers how to build classifiers that are robust to manip”
Generating Natural Adversarial Examples (2017, background): “l network through different levels of representation erasure. However, erasure of words or phrases directly often harms text integrity, resulting in semantically or grammatically incorrect sentences. Ribeiro et al. (2018) replace tokens by random words of the same POS tag with probability proportion”
The challenge of crafting intelligible intelligence (2018, background): “describe a similar optimization algorithm that balances faithfulness and coverage in its search for summary rules.(34) Indeed, all methods for rendering an inscrutable model intelligible require mapping the complex model to a simpler one.”
Evaluation of Text Generation: A Survey (2020, background): “The field of explainable AI is growing, particularly in generating explanations of classifier predictions in NLP tasks (Ribeiro et al., 2016, 2018; Thorne et al., 2019).”
Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods (2019, background): “For instance, anchors [21] use (non-linear) rules to express more actionable local explanations.”
PRover: Proof Generation for Interpretable Reasoning over Rules (2020, background): “…have focused on formalizing the notion of interpretability (Rudin, 2019; Doshi-Velez and Kim, 2017; Hase and Bansal, 2020), tweaking features for local model interpretability (Ribeiro et al., 2016, 2018) and exploring interpretability in latent spaces (Joshi et al., 2018; Samangouei et al., 2018).”