Parametric or nonparametric? A parametricness index for model selection

By Wei David Liu, Yuhong Yang. 2011

61 citations recorded by the source corpus.

In model selection literature, two classes of criteria perform well asymptotically in different situations: Bayesian information criterion (BIC) (as a representative) is consistent in selection when the true model is finite dimensional (parametric scenario); Akaike's information criterion (AIC) performs well in an asymptotic efficiency when the true model is infinite dimensional (nonparametric scenario). But there is little work that addresses if it is possible and how to detect the situation that a specific model selection problem is in. In this work, we differentiate the two scenarios theoretically under some conditions. We develop a measure, parametricness index (PI), to assess whether a model selected by a potentially consistent procedure can be practically treated as the true model, which also hints on AIC or BIC is better suited for the data for the goal of estimating the regression function. A consequence is that by switching between AIC and BIC based on the PI, the resulting regression estimator is simultaneously asymptotically efficient for both parametric and nonparametric scenarios. In addition, we systematically investigate the behaviors of PI in simulation and real data and show its usefulness.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Hyperparameter optimization with approximate gradient (2016, background): “As mentioned in the introduction, the goal of hyperparam-eter optimization is to choose the hyperparameters λ that optimizes some criteria, such as a cross-validation loss or a SURE/AIC/BIC criteria.”
Model Selection Techniques: An Overview (2018, background): “, 100– , ), 2 000 a relatively small parametric model among the candidates continues to be the best model [14].”