Variable selection and sensitivity analysis using dynamic trees, with an application to computer code performance tuning

By Robert B. Gramacy, Matt Taddy, Stefan M. Wild. 2011

37 citations recorded by the source corpus.

We investigate an application in the automatic tuning of computer codes, an area of research that has come to prominence alongside the recent rise of distributed scientific processing and heterogeneity in high-performance computing environments. Here, the response function is nonlinear and noisy and may not be smooth or stationary. Clearly needed are variable selection, decomposition of influence, and analysis of main and secondary effects for both real-valued and binary inputs and outputs. Our contribution is a novel set of tools for variable selection and sensitivity analysis based on the recently proposed dynamic tree model. We argue that this approach is uniquely well suited to the demands of our motivating example. In illustrations on benchmark data sets, we show that the new techniques are faster and offer richer feature sets than do similar approaches in the static tree and computer experiment literature. We apply the methods in code-tuning optimization, examination of a cold-cache effect, and detection of transformation errors.

Subfields: Computational Theory and Mathematics, Electrical and Electronic Engineering, Hardware and Architecture

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Electrical and Electronic Engineering

In Hardware and Architecture

Citations in the corpus

Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks (2017, methodology): “Hutter et al. (2014) describe a method to gain insights into the relative importance of hyperparameters by using random forest predictions with functional ANOVA compositions (Gramacy et al., 2013).”
Automated Machine Learning on Big Data using Stochastic Algorithm Tuning (2014, methodology): “We provide explicit comparisons against the BO state of the art for managing large numbers of evaluations, random forest regressors (RFRs) [7, 8].”
Automated Machine Learning using Stochastic Algorithm Tuning (2014, background): “This enables the tackling of challenging algorithm configuration tasks and provides non-trivial acceleration on generic BO problems, while retaining the powerful probabilistic machinery of the GP, which provides better performance than the empirical mean and variance of random forest regressors (RFR”
An Efficient Approach for Assessing Hyperparameter Importance (2014, methodology): “To date, efficient marginalizations had not been available for random forest models, forcing researchers to revert to sampling-based techniques to compute approximate functional ANOVA decompositions (Gramacy et al., 2013).”