Regularized estimation in sparse high-dimensional time series models

By Sumanta Basu, George Michailidis. 2013

473 citations recorded by the source corpus.

Many scientific and economic problems involve the analysis of high-dimensional time series datasets. However, theoretical studies in high-dimensional statistics to date rely primarily on the assumption of independent and identically distributed (i.i.d.) samples. In this work, we focus on stable Gaussian processes and investigate the theoretical properties of $\ell _1$-regularized estimates in two important statistical problems in the context of high-dimensional time series: (a) stochastic regression with serially correlated errors and (b) transition matrix estimation in vector autoregressive (VAR) models. We derive nonasymptotic upper bounds on the estimation errors of the regularized estimates and establish that consistent estimation under high-dimensional scaling is possible via $\ell_1$-regularization for a large class of stable processes under sparsity constraints. A key technical contribution of the work is to introduce a measure of stability for stationary processes using their spectral properties that provides insight into the effect of dependence on the accuracy of the regularized estimates. With this proposed stability measure, we establish some useful deviation bounds for dependent data, which can be used to study several important regularized estimates in a time series setting.

Subfields: Applied Mathematics, Computational Mechanics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/414c16698cd6dc95151680090784d574e20998e9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Computational Mechanics

In Statistics and Probability

Citations in the corpus

NETS: Network estimation for time series (2019, unknown): “First, it is related to the econometric literature on networks, which includes research by Billio et al. (2012), Diebold and Yılmaz (2014, 2015), Hautsch et al. (2015), Anufriev and Panchenko (2015), Härdle et al.”
On testing for high-dimensional white noise (2018, methodology): “When the dimension is large, most existing works regularize the fitted models by Lasso [6, 17, 19, 32], Dantzig penalization [16], banded autocovariances [7], or banded auto-coefficient matrices [13].”
Granger Causality: A Review and Recent Advances (2021, methodology): “2) and underlie the study of network Granger causality (Basu et al., 2015).”
Machine Learning Advances for Time Series Forecasting (2020, background): “In high-dimensional settings, where the dimension of the series increase with the number of observations, Kock and Callot (2015) and Basu and Michailidis (2015) show oracle bounds and model selection consistency for the LASSO in Gaussian 𝑉𝐴𝑅(𝑝) models, extending previous works.”
High‐dimensional macroeconomic forecasting and variable selection via penalized regression (2019, background): “Basu and Michailidis (2015) establishes the oracle inequality for the lasso with weak dependence under Gaussian assumption.”
Performance bounds for parameter estimates of high-dimensional linear models with correlated errors (2016, unknown): “…studied Lasso with long memory errors with very light tails such that the Cram´er condition is met, Loh [24] considered M -estimators for linear models with i.i.d. data, and in [3] Basu and Michailidis investigated theoretical properties of Lasso estimates for high-dimensional Gaussian processes.”
Neural Granger Causality (2021, methodology): “Sparsity-inducing regularizers, like the Lasso [12] or group lasso [13], help scale linear Granger causality estimation in VAR models to the high-dimensional setting [7], [14].”