Or's of And's for Interpretable Classification, with Application to Context-Aware Recommender Systems

By Tong Wang, Cynthia Rudin, Finale Doshi-Velez, Yimin Liu, Erica Klampfl, Perry MacNeille. 2015

41 citations recorded by the source corpus.

We present a machine learning algorithm for building classifiers that are comprised of a small number of disjunctions of conjunctions (or's of and's). An example of a classifier of this form is as follows: If X satisfies (x1 = 'blue' AND x3 = 'middle') OR (x1 = 'blue' AND x2 = '<15') OR (x1 = 'yellow'), then we predict that Y=1, ELSE predict Y=0. An attribute-value pair is called a literal and a conjunction of literals is called a pattern. Models of this form have the advantage of being interpretable to human experts, since they produce a set of conditions that concisely describe a specific class. We present two probabilistic models for forming a pattern set, one with a Beta-Binomial prior, and the other with Poisson priors. In both cases, there are prior parameters that the user can set to encourage the model to have a desired size and shape, to conform with a domain-specific definition of interpretability. We provide two scalable MAP inference approaches: a pattern level search, which involves association rule mining, and a literal level search. We show stronger priors reduce computation. We apply the Bayesian Or's of And's (BOA) model to predict user behavior with respect to in-vehicle context-aware personalized recommender systems.

Subfields: Artificial Intelligence, Information Systems, Safety, Robustness & Interpretability, Recommender Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Safety, Robustness & Interpretability

In Recommender Systems

Citations in the corpus

Learning Optimal Decision Trees with SAT (2018, background): “Many other representations have been studied, including rule lists [Angelino et al. , 2017], decision sets [Lakkaraju et al. , 2017], and sums of terms (i.e. DNF) [Wang et al. , 2015; Hauser et al. , 2010], among others.”
Anchors: High-Precision Model-Agnostic Explanations (2018, background): “A number of approaches construct globally interpretable models, many based on rules (Lakkaraju, Bach, and Leskovec 2016; Letham et al. 2015; Wang and Rudin 2015; Wang et al. 2015).”
Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model (2015, methodology): “Wang et al. (2015) build disjunctions of conjunctive rules using a Bayesian framework similar to the one in this work.”
Interpretable Decision Sets: A Joint Framework for Description and Prediction (2016, methodology): “While metrics accounting for conciseness have been studied in the context of interpretable rule-based models [34, 53], the property of overlap has not been previously considered.”
Learning Optimized Or's of And's (2015, methodology): “The closest work to ours are that of Hauser et al. (2010) in the marketing literature, and Wang et al. (2015) on Bayesian modeling of DNF formulae.”
Learning Classification Models of Cognitive Conditions from Subtle Behaviors in the Digital Clock Drawing Test (2015, background): “…(Steinhart 2006; Andrade 2009; Ridgeway 2013), national defense (ABS Consulting 2002), marketing (Hauser et al. 2010; Verbeke et al. 2011; Wang et al. 2015), medical diagnosis (Tian and Tibshirani 2011; Van Belle et al. 2012; Letham et al. 2015; Wang and Rudin 2015), and scientific…”
A Survey on the Explainability of Supervised Machine Learning (2020, methodology): “…Classification Rule-based Ustun and Rudin (2017) Two-level Boolean Rules Classification Rule-based Su et al. (2015) BOA Classification Rule-based Wang et al. (2015b) ORC Classification Rule-based Bertsimas et al. (2011) BLM Classification Rule-based Letham et al. (2012) BRL Classification…”
Interpretable Optimal Stopping (2018, background): “Another stream of research considers the design of disjunctive rules and rule lists; recent examples include Wang et al. (2015), Wang and Rudin (2015), Wang et al. (2017), Letham et al. (2015), Angelino et al. (2017) and Lakkaraju et al. (2016).”
Snuba: Automating Weak Supervision to Label Training Data (2018, background): “Other works also generate heuristics to help interpret the underlying data labels [56, 55], but neither methods use unlabeled data since the programs generated either mimic the desired program perfectly or provide interpretations for existing labels.”
Explanation and Justification in Machine Learning : A Survey Or (2017, methodology): “This family of models includes Bayesian Rule Lists [ Letham et al. , 2015 ] , an algorithm that generates a posterior distribution of decision lists that encourages sparsity as well as accuracy; Bayesian Or’s of And’s [ Wang et al. , 2015 ] , highly efﬁcient disjunctive rule lists; and Falling Rule ”