High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: Risk underestimation

By Noureddine El Karoui. 2010

110 citations recorded by the source corpus.

We study the properties of solutions of quadratic programs with linear equality constraints whose parameters are estimated from data in the high-dimensional setting where p, the number of variables in the problem, is of the same order of magnitude as n, the number of observations used to estimate the parameters. The Markowitz problem in Finance is a subcase of our study. Assuming normality and independence of the observations we relate the ecient frontier computed empirically to the \true" ecient frontier. Our computations show that there is a separation of the errors induced by estimating the mean of the observations and estimating the covariance matrix. In particular, the price paid for estimating the covariance matrix is an underestimation of the variance by a factor roughly equal to 1 p=n. Therefore the risk of the optimal population solution is underestimated when we estimate it by solving a similar quadratic program with estimated parameters. We also characterize the statistical behavior of linear functionals of the empirical optimal vector and show that they are biased estimators of the corresponding population quantities. We investigate the robustness of our Gaussian results by extending the study to certain elliptical models and models where our n observations are correlated (in \time"). We show a lack of robustness of the Gaussian results, but are still able to get results concerning rst order properties of the quantities of interest, even in the case of relatively heavy-tailed data (we require two moments). Risk underestimation is still present in the elliptical case and more pronounced that in the Gaussian case. We discuss properties of the non-parametric and parametric bootstrap in this context. We show several results, including the interesting fact that standard applications of the bootstrap generally yields inconsistent estimates of bias. Finally, we propose some strategies to correct these problems and practically validate them in some simulations. In all the paper, we will assume that p, n and n p tend to innity, and p < n.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/8ae4dd5ecce2c8a1e9cdbfcdbb183d5d759c06d8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

High-dimensional minimum variance portfolio estimation based on high-frequency data (2020, unknown): “The first result is a direct consequence of Theorem 4.1 in El Karoui (2010), by which we have 1ᵀS−11 1ᵀΣ−11 →p 1 (1− ρ) , (A.12) The conclusion in the theorem follows by noticing that R̂p Rmin = wᵀpSwp wᵀoptΣwopt = 1ᵀΣ−11 1ᵀS−11 .”
Using Principal Component Analysis to Estimate a High Dimensional Factor Model with High-Frequency Data (2017, unknown): “The second approach consists in imposing constraints on the portfolio weights (Jagannathan and Ma (2003), Pesaran and Zaffaroni (2008), DeMiguel et al. (2009a), El Karoui (2010), Fan et al. (2012), Gandy and Veraart (2013)) or penalties (Brodie et al. (2009)).”
The Joint Distribution of Stock Returns is Not Elliptical (2010, unknown): “This topic has been the focus of intense research in the recent years, some inspired by Random Matrix Theory (for a review see [13,38]) or clustering ideas [32,42,43].”
Cleaning large correlation matrices: tools from random matrix theory (2016, unknown): “We refer to [137] for a rigorous study of this problem.”
Optimal Shrinkage-Based Portfolio Selection in High Dimensions (2016, background): “The sample estimator of the EU portfolio behaves badly in this case both from the theoretical and practical points of view (see, e.g., El Karoui (2010)).”
Random matrix theory in statistics: A review (2014, background): “El Karoui (2010c) also suggested ways of correcting this bias.”
An Overview on the Estimation of Large Covariance and Precision Matrices (2015, unknown): “To estimate large covariance matrices, one of the key assumptions made in the literature is that the target matrix of interest is sparse, namely, many entries are zero or nearly so (Bickel and Levina, 2008; Lam and Fan, 2009; El Karoui, 2010; Rigollet and Tsybakov, 2012).”
A Nodewise Regression Approach to Estimating Large Portfolios (2016, background): “Other approaches include that of Karoui (2010), where the properties of the Markowitz portfolio based on quadratic optimization are established in a setting where p is of the same magnitude as n, and with independent, normally distributed asset returns.”