Optimal selection of reduced rank estimators of high-dimensional matrices

By Florentina Bunea, Yiyuan She, Marten H. Wegkamp. 2010

262 citations recorded by the source corpus.

We introduce a new criterion, the Rank Selection Criterion (RSC), for selecting the optimal reduced rank estimator of the coefficient matrix in multivariate response regression models. The corresponding RSC estimator minimizes the Frobenius norm of the fit plus a regularization term proportional to the number of parameters in the reduced rank model. The rank of the RSC estimator provides a consistent estimator of the rank of the coefficient matrix; in general, the rank of our estimator is a consistent estimate of the effective rank, which we define to be the number of singular values of the target matrix that are appropriately large. The consistency results are valid not only in the classic asymptotic regime, when n, the number of responses, and p, the number of predictors, stay bounded, and m, the number of observations, grows, but also when either, or both, n and p grow, possibly much faster than m. We establish minimax optimal bounds on the mean squared errors of our estimators. Our finite sample performance bounds for the RSC estimator show that it achieves the optimal balance between the approximation error and the penalty term. Furthermore, our procedure has very low computational complexity, linear in the number of candidate models, making it particularly appealing for large scale problems. We contrast our estimator with the nuclear norm penalized least squares (NNP) estimator, which has an inherently higher computational complexity than RSC, for multivariate regression models. We show that NNP has estimation properties similar to those of RSC, albeit under stronger conditions. However, it is not as parsimonious as RSC. We offer a simple correction of the NNP estimator which leads to consistent rank estimation. We verify and illustrate our theoretical findings via an extensive simulation study.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/8124c3f9bf7fb7cef9cb28ba77117f0923039b14

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Estimation of high-dimensional low-rank matrices (2009, background): “We also mention a more recent work by Bunea, She and Wegkamp (2010) dealing with a special case of our model and analyzing matrix estimators penalized by the rank.”
Penalized Least Squares Regression Methods and Applications to Neuroimaging (2010, unknown): “By refining the method in Bunea et al. (2011), we will develop procedures that select the best biomarker predictors of a collection of, possibly correlated, outcome variables.”
High-dimensional regression with unknown variance (2011, background): “[20] propose to plug an estimate of the variance in their selection criterion [which works when rank(X) < n], whereas Giraud [36] introduces a penalized log-likelihood criterion independent of the variance.”
High-dimensional covariance matrix estimation with missing observations (2012, unknown): “See also [5, 21] for rank minimization approach.”
Analysis of elliptical copula correlation factor model with Kendall's tau (2013, methodology): “A similar reasoning is used in the proof of [3], Theorem 2.”
Robust regression via mutivariate regression depth (2017, background): “Some recent progresses on this topic were made by [4, 23] and references therein.”
Joint variable and rank selection for parsimonious estimation of high-dimensional matrices (2011, methodology): “easures of several white-matter regions of interest, such as fractional anisotropy, mean diﬀusivity, axial diﬀusivity, and radial diﬀusivity, along with all volumetrics ×DTI interactions. We refer to Bunea et al. (2011a) for details. The ﬁnal model has p = 235 predictors, much greater than the sampl”
Nuclear norm penalization and optimal rates for noisy low rank matrix completion (2010, methodology): “The most popular methods are based on penalized empirical risk minimization with nuclear norm penalty [4, 2, 3, 5, 7, 8, 9, 13, 18, 23].”