Model averaging, asymptotic risk, and regressor groups

By B. Hansen. 2014

102 citations recorded by the source corpus.

This paper examines the asymptotic risk of nested least‐squares averaging estimators when the averaging weights are selected to minimize a penalized least‐squares criterion. We find conditions under which the asymptotic risk of the averaging estimator is globally smaller than the unrestricted least‐squares estimator. For the Mallows averaging estimator under homoskedastic errors, the condition takes the simple form that the regressors have been grouped into sets of four or larger. This condition is a direct extension of the classic theory of James–Stein shrinkage. This discovery suggests the practical rule that implementation of averaging estimators be restricted to models in which the regressors have been grouped in this manner. Our simulations show that this new recommendation results in substantial reduction in mean‐squared error relative to averaging over all nested submodels. We illustrate the method with an application to the regression estimates of Fryer and Levitt (2013).

Subfields: Finance, Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/9aee5fdb90de952ebe7cd7ea961362adb776ddbf

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Finance

1. Xinyu Zhang
2. Jihai Yu
3. Hua Liang
4. Dalei Yu
5. Guohua Zou

In Management Science and Operations Research

1. Jihai Yu
2. Hua Liang
3. Dalei Yu
4. Xinyu Zhang
5. Guohua Zou

In Statistics and Probability

1. Xinyu Zhang
2. Guohua Zou
3. Hua Liang
4. Dalei Yu
5. Jihai Yu
6. J. R. Magnus
7. Giuseppe De Luca
8. Mark F. J. Steel

Citations in the corpus

• Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models (2016, methodology): “Using variable selection (or model selection), we end up putting all our inferential eggs in one unevenly woven basket (Longford 2005), while model averaging is a smoothed extension of model selection for substantially reducing risk relative to selection (Hansen 2014).”
• INFERENCE AFTER MODEL AVERAGING IN LINEAR REGRESSION MODELS (2018, unknown): “Similar to Hansen (2014), for all the following theoretical results, we do not impose any assumption on the ordering of regressors, i.e., the ordering is not required to be “correct” in any sense.”
• Generalized Least Squares Model Averaging (2016, unknown): “Specifically, the kth group contains the (4(k − 1) + 1)th to the (4k)th regressors, and the kth submodel includes the regressors in the first k groups, as in Hansen (2014).”
• Spatial weights matrix selection and model averaging for spatial autoregressive models (2017, background): “By doing so, model averaging provides a kind of insurance against selecting a very poor model and can substantially reduce the risk compared to model selection; see Leung and Barron (2006) and Hansen (2014).”