Born-Again Tree Ensembles

By Thibaut Vidal, Toni Pacheco, Maximilian Schiffer. 2020

65 citations recorded by the source corpus.

The use of machine learning algorithms in finance, medicine, and criminal justice can deeply impact human lives. As a consequence, research into interpretable machine learning has rapidly grown in an attempt to better control and fix possible sources of mistakes and biases. Tree ensembles offer a good prediction quality in various domains, but the concurrent use of multiple trees reduces the interpretability of the ensemble. Against this background, we study born-again tree ensembles, i.e., the process of constructing a single decision tree of minimum size that reproduces the exact same behavior as a given tree ensemble in its entire feature space. To find such a tree, we develop a dynamic-programming based algorithm that exploits sophisticated pruning and bounding rules to reduce the number of recursive calls. This algorithm generates optimal born-again trees for many datasets of practical interest, leading to classifiers which are typically simpler and more interpretable without any other form of compromise.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Mathematical optimization in classification and regression trees (2021, methodology): “This approach is suggested in Vidal et al. (2020), where the closest tree is 1 3 Mathematical optimization in classification and regression… extracted using dynamic programming.”
Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation (2020, background): “[9] showed how to distill tree ensembles into a single tree without sacrificing accuracy, but their approach is restricted to tree student/teacher models.”
Security and Privacy for 6G: A Survey on Prospective Technologies and Challenges (2021, background): “Explainable AI like the studies of [224] and [225] aims to provide an interpretable and faithful manner to predict the results, by proposing the decision trees, rule”