Supersparse linear integer models for optimized medical scoring systems

By Berk Ustun, Cynthia Rudin. 2015

386 citations recorded by the source corpus.

Scoring systems are linear classification models that only require users to add, subtract and multiply a few small numbers in order to make a prediction. These models are in widespread use by the medical community, but are difficult to learn from data because they need to be accurate and sparse, have coprime integer coefficients, and satisfy multiple operational constraints. We present a new method for creating data-driven scoring systems called a Supersparse Linear Integer Model (SLIM). SLIM scoring systems are built by using an integer programming problem that directly encodes measures of accuracy (the 0–1 loss) and sparsity (the $$\ell _0$$ℓ0-seminorm) while restricting coefficients to coprime integers. SLIM can seamlessly incorporate a wide range of operational constraints related to accuracy and sparsity, and can produce acceptable models without parameter tuning because of the direct control provided over these quantities. We provide bounds on the testing and training accuracy of SLIM scoring systems, and present a new data reduction technique that can improve scalability by eliminating a portion of the training data beforehand. Our paper includes results from a collaboration with the Massachusetts General Hospital Sleep Laboratory, where SLIM is being used to create a highly tailored scoring system for sleep apnea screening.

Subfields: Statistics and Probability, Mathematical Optimization, Medical & Health AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Mathematical Optimization

In Medical & Health AI

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “There are several methods that can solve (3) directly (Ustun et al., 2013; Ustun and Rudin, 2016, 2017, 2019; Rudin and Ustun, 2018).”
Please Stop Explaining Black Box Models for High Stakes Decisions (2018, background): “Figure 4: Scoring system for risk of recidivism from Rudin and Ustun [2018] [which grew out of Ustun and Rudin, 2017, Zeng et al., 2017, Ustun and Rudin, 2015].”
On the Art and Science of Machine Learning Explanations (2018, background): “They have developed several types of rule-based models [3], [31], linear model variants [28], and many other novel algorithms suitable for use in high stakes, mission-critical prediction and decision-making scenarios.”
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, background): “Ustun and Rudin [68] presented a sparse linear models for creating a data-driven scoring systems called SLIM,”
Anchors: High-Precision Model-Agnostic Explanations (2018, background): “As a consequence of the need for users to understand the behavior of these models, interpretable machine learning has seen a resurgence in recent years, ranging from the design of novel globally-interpretable machine learning models (Lakkaraju, Bach, and Leskovec 2016; Ustun and Rudin 2015; Wang and”
Interpretability via Model Extraction (2017, background): “Interpretability is a promising approach to address these challenges [12, 24]—we can help human users diagnose issues and verify correctness of machine learning models by providing insight into the model’s reasoning [3, 18, 20, 23, 26].”
Model-Agnostic Interpretability of Machine Learning (2016, methodology): “The prevailing solution to this explanation problem is to use so called “interpretable” models, such as decision trees, rules (Letham et al., 2015; Wang & Rudin, 2015), additive models (Caruana et al., 2015), attention-based networks (Xu et al., 2015), or sparse linear models (Ustun & Rudin, 2015).”
Interpretable classification models for recidivism prediction (2015, methodology): “A Supersparse Linear Integer Model (SLIM) is a new optimization-based method for creating scoring systems – that is, linear classiﬁcation models that only require users to add, subtract and multiply a few small numbers to make a prediction (Ustun and Rudin 2015).”
Learning Fair Rule Lists (2019, background): “For instance, when they have small or of reasonable size [44], ∗ Principal contributor. the following models can be considered as interpretable: rules sets [18, 43, 46, 54], rule lists [3, 59, 61], decision trees [9, 51] and scoring systems [39, 57, 64].”
An Interpretable Model with Globally Consistent Explanations for Credit Risk (2018, background): “…+ βp,0bp,0(x·,p) can be equivalently written as fp(x·,p) = (βp,1 + βp,2 + βp,3 + βp,0)1[x·,p 10] + (βp,2 + βp,3 + βp,0)1[10 ≤ x·,p 50] +(βp,3 + βp,0)1[50 ≤ x·,p 75] + βp,01[75 ≤ x·,p], which can be displayed as a traditional scoring system [Ustun and Rudin, 2016, Medical calculators].”
Deep Weighted Averaging Classifiers (2018, background): “This category includes classic models that can easily be simulated by humans, such as decision lists, as well as sparse linear models, where the prediction is based on a weighted sum of features [6, 24, 29, 49, 50, 56].”
Learning customized and optimized lists of rules with mathematical programming (2018, background): “linear models (as discussed by [64,65]), as well as other model forms, and for many other types of algorithms.”
Interval Coded Scoring: a toolbox for interpretable scoring systems (2018, neutral): “In contrast to similar work by Ustun and Rudin (2016) it focuses on value intervals rather than variables, allowing for more ﬁne-tuned models.”
Manipulating and Measuring Model Interpretability (2018, background): “ﬁrst is to employ simple models in which the impact of each feature on the model’s prediction is arguably easy to understand. Examples include generative additive models [3, 18, 19] and point systems [10, 27]. The second is to provide post-hoc explanations for (potentially complex) models. One threa”
Learning Optimized Risk Scores (2016, background): “RiskSLIM risk scores are the risk assessment counterpart to SLIM scoring systems (Ustun et al., 2013; Ustun and Rudin, 2016; Zeng et al., 2016; Ustun et al., 2016).”