Monotonic Calibrated Interpolated Look-Up Tables

By Maya R. Gupta, Andrew Cotter, Jan Pfeifer, Konstantin Voevodski, Kevin Robert Canini, Alexander Mangylov, Wojtek Moczydlowski, Alexander Van Esbroeck. 2015

148 citations recorded by the source corpus.

Real-world machine learning applications may have requirements beyond accuracy, such as fast evaluation times and interpretability. In particular, guaranteed monotonicity of the learned function with respect to some of the inputs can be critical for user confidence. We propose meeting these goals for low-dimensional machine learning problems by learning flexible, monotonic functions using calibrated interpolated look-up tables. We extend the structural risk minimization framework of lattice regression to monotonic functions by adding linear inequality constraints. In addition, we propose jointly learning interpretable calibrations of each feature to normalize continuous features and handle categorical or missing data, at the cost of making the objective non-convex. We address large-scale learning through parallelization, mini-batching, and random sampling of additive regularizer terms. Case studies on real-world problems with up to sixteen features and up to hundreds of millions of training samples demonstrate the proposed monotonic functions can achieve state-of-the-art accuracy in practice while providing greater transparency to users.

Subfields: Artificial Intelligence, Statistics and Probability, Optimization & Training, Algorithms & Complexity

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Optimization & Training

In Algorithms & Complexity

Citations in the corpus

Please Stop Explaining Black Box Models for High Stakes Decisions (2018, background): “…model is constrained in model form so that it is either useful to someone, or obeys structural knowledge of the domain, such as monotonicity [e.g., Gupta et al., 2016], causality, structural (generative) constraints, additivity [Lou et al., 2013], or physical constraints that come from domain…”
Learning Optimized Risk Scores (2016, background): “5 Our approach is a promising option to build models that obey constraints related to, for example, interpretability (see e.g. Caruana et al., 2015; Gupta et al., 2016; Chen et al., 2018; Li et al., 2018, where interpretability is addressed through constraints on model form), safety (Amodei et al.,…”
Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals (2018, background): “…the model is restricted to functions with a certain shape such as monotonic functions (e.g. Barlow et al. (1972); Groeneboom and Jongbloed (2014); Gupta et al. (2016); Canini et al. (2016); Luss and Rosset (2017); You et al. (2017); Bonakdarpour et al. (2018)), or other shapes (e.g. Chetverikov…”
Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints (2018, methodology): “On each dataset, we trained one of three different types of models: linear, calibrated lattice [Gupta et al., 2016], or a neural network with one hidden layer containing 50 ReLU neurons. The neural network models are more complex than necessary for these datasets, and are used here because they over”
A Roadmap for a Rigorous Science of Interpretability (2017, background): “In monotone ways [Gupta et al., 2016]?”
A Qualitative Exploration of Perceptions of Algorithmic Fairness (2018, background): “A wide range of techniques have been proposed to scrutinize algorithms, such as model interpretability, audits, expert analysis, and reverse engineering [22,42,76,77].”
Towards A Rigorous Science of Interpretable Machine Learning (2017, unknown): “In monotone ways [Gupta et al., 2016]?”
xGEMs: Generating Examplars to Explain Black-Box Models (2018, background): “In this work, we posit that there need not be an inherent tradeoff between model performance and explanability, as is generally assumed (and found in [22, 16, 20]).”
Towards Complementary Explanations Using Deep Neural Networks (2018, background): “Another way of facilitating the model interpretability is to guarantee the monotonicity of the learnt function in relation to some of the inputs [4].”
Proxy Fairness (2018, background): “There are N = 15 , 415 training and 3 , 854 test examples, all IID, and the classiﬁer is a non-linear lattice model [8] trained on D = 9 features.”
Unconstrained Monotonic Neural Networks (2019, unknown): “(c) NAF - Huang et al. [2018] −0.”
Fast and Flexible Monotonic Functions with Ensembles of Lattices (2016, methodology): “Generalization to larger look-up tables is trivial [11, 2].”
The Case for Learned Index Structures (2017, methodology): “To overcome this problem, one option is to force our RMI model to be monotonic, as has been studied in machine learning [41, 71].”
Learning Global Additive Explanations for Neural Nets Using Model Distillation (2018, background): “This way of representing the relationship between input features and model predictions has precedence in interpretability, from work that learned monotonic (Gupta et al., 2016) or concave/convex (Pya & Wood, 2015) feature shapes from original data (i.”