Sparse Multivariate Regression With Covariance Estimation

By Adam J. Rothman, Elizaveta Levina, Ji Zhu. Journal of Computational And Graphical Statistics, 2010

358 citations recorded by the source corpus.

We propose a procedure for constructing a sparse estimator of a multivariate regression coefficient matrix that accounts for correlation of the response variables. This method, which we call multivariate regression with covariance estimation (MRCE), involves penalized likelihood with simultaneous estimation of the regression coefficients and the covariance structure. An efficient optimization algorithm and a fast approximation are developed for computing MRCE. Using simulation studies, we show that the proposed method outperforms relevant competitors when the responses are highly correlated. We also apply the new method to a finance example on predicting asset returns. An R-package containing this dataset and code for computing MRCE and its approximation are available online.

Subfields: Applied Mathematics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/e4e872b2cc8d04076fc23139bc4e5721722ada70

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Statistics and Probability

Citations in the corpus

Multi-Task Learning with Gaussian Matrix Generalized Inverse Gaussian Model (2013, methodology): “In the real data experiment, we compare the GMGIG regression model for multi-task learning (MTL-GMGIG) with the single task learning method, the ridge logistic regression (STL), and the other state-of-the-art multitask learning methods with least square loss including clustered multi-task learning (”
Reconciling meta-learning and continual learning with online mixtures of tasks (2018, background): “Moreover, explicitly estimating relatedness between tasks has the potential to aid the efﬁcacy of a meta-learning algorithm by modulating both positive and negative transfer [52, 59, 45, 61], and by identifying outlier tasks that require a more signiﬁcant degree of adaptation [56, 23].”
Multi-task Sparse Structure Learning with Gaussian Copula Models (2016, methodology): “Xue et al. (2007) employed a Dirichlet process prior over the task coefficients to encourage task clustering and the number of clusters was someway automatically determined by the prior.”
Multi-Task Multiple Kernel Relationship Learning (2016, background): “Second, learn the relationship between the tasks to improve the performance of the related tasks [19, 26, 28, 29].”
Scalable Matrix-valued Kernel Learning for High-dimensional Nonlinear Multivariate Regression and Granger Causality (2012, methodology): “…in Table 1 where we compare our methods in terms of mean test RMSE against standard linear regression (OLS) and linear Lasso independently applied to each output co-ordinate, and the sparse multivariate regression with covariance estimation approaches of [31, 39], labeled MRCE and FES respectively.”
A Tutorial on Regularized Partial Correlation Networks (2016, methodology): “…networks while taking temporal information into account is by using the graphical vectorautoregression model (graphical VAR; Wild et al. 2010; Fisher et al. 2017; Epskamp et al. 2017d), for which LASSO regularization techniques have been worked out (Abegaz and Wit, 2013; Rothman et al., 2010).”
Simultaneously Leveraging Output and Task Structures for Multiple-Output Regression (2012, methodology): “Although some of the existing multiple-output regression m odels have attempted to incorporate such structures [17, 11, 13], most of these models are restrictiv e in the sense that (1) they usually exploit only one of the two structures (output structure o task structure, but not both), and (2) they”
Matrix variate regressions and envelope models (2016, methodology): “, 2005; Rothman et al., 2010; Chen and Huang 2012; Su et al., 2016). In particular, Su et al. (2016) studied sparse multivariate regression with an envelope structure.”
Monitoring the covariance matrix via penalized likelihood estimation (2013, methodology): “As several authors have already pointed out (see, for example, Woodall and Montgomery (1999), Stoumbos et al. (2000), and Woodall (2000)), the multivariate control chart will be an important area of research in the 21st century.”
Online gradient-based mixtures for transfer modulation in meta-learning (2018, background): “Moreover, explicitly estimating relatedness between tasks has the potential to aid the efficacy of a meta-learning algorithm by modulating both positive and negative transfer (Thrun; Zhang and Schneider, 2010; Rothman et al., 2010; Zhang and Yeung, 2014; Xue et al., 2007).”
Identifying demand effects in a large network of product categories (2015, methodology): “It builds further on a sparse estimator of the multivariate regression model (Rothman et al., 2010), and the groupwise lasso for categorical variables (Yuan and Lin, 2006; Meier et al.”