Node harvest: simple and interpretable regression and classication

By Nicolai Meinshausen. 2009

64 citations recorded by the source corpus.

When choosing a suitable technique for regression and classication with multivariate pre- to analyze than single trees and are often criticized, perhaps unfairly, as 'black box' predictors. Node harvest is trying to reconcile the two aims of interpretability and predictive accuracy by combining positive aspects of trees and tree ensembles. Results are very sparse and interpretable and predictive accuracy is extremely competitive, especially for low signal-to-noise data. The procedure is very simple: an initial set of a few thousand nodes is generated randomly. If a new observation falls into just a single node, its prediction is the mean response of all training observation within this node, identical to a tree-like prediction. However, a new observation falls typically into several nodes and its prediction is then the weighted average of the mean responses across all these nodes. The only role of node harvest is to 'pick' the right nodes from the initial large ensemble of nodes by choosing node weights, which amounts in the proposed algorithm to a quadratic programming problem with linear inequality constraints. The solution is sparse in the sense that only very few nodes are selected with a non-zero weight. This sparsity is not explicitly enforced. Maybe surprisingly, it is not necessary to select a tuning parameter for optimal predictive accuracy. Node harvest can handle mixed data and missing values well and is shown to be simple to interpret and competitive in predictive accuracy on a variety of datasets, with special attention given to an application in climate modelling.

Subfields: Artificial Intelligence, Statistics and Probability, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Safety, Robustness & Interpretability

Citations in the corpus

Interpreting tree ensembles with inTrees (2014, background): “, reduced decision nodes referred to as node harvest [28], rules generated from simplified predictive regions [22,23], single decision trees [3,4,34, 36], and linear models [14].”
Mathematical optimization in classification and regression trees (2021, background): “2020; Meinshausen 2010) because of their rule-based nature.”
Methods and Models for Interpretable Linear Classification (2014, methodology): “interpretability of black-box models is mainly improved by auxiliary methods that extract rules and prototype examples to illustrate the relationship between input variables and the predicted outcome [78, 111, 40, 76, 11]. These rules and prototypes are 2 useful for troubleshooting and generating in”
Decision Forests for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning (2011, methodology): “A related model, made of multiple single nodes is “node harvest” [57].”
An Interpretable Stroke Prediction Model using Rules and Bayesian Analysis (2013, unknown): “Call the rule˜r i .”
An extensive experimental survey of regression methods (2019, methodology): “nodeHarvest is a simple interpretable tree-based ensemble for high926 dimensional regression with sparse results [42] implemented in the node927 Harvest package.”
A Bayesian Approach to Learning Scoring Systems (2015, methodology): “Other methods (Meinshausen, 2010; Van Assche and Blockeel, 2007; Fung, Sandilya, and Rao, 2005; Martens et al., 2007; Bien and Tibshirani, 2011) extract rules and prototype examples to illustrate the relationship between input variables and the predicted outcome.”
A Survey on the Explainability of Supervised Machine Learning (2020, background): “Rule-based models: Decision rules have the structure of IF condition THEN label ELSE other label.”
Rule Covering for Interpretation and Boosting (2020, methodology): “Meinshausen [15] presents a similar approach, called Node Harvest, which relies on rules generated by a tree ensemble.”