Parametric estimation. Finite sample theory

By Vladimir Spokoiny. 2011

117 citations recorded by the source corpus.

The paper aims at reconsidering the famous Le Cam LAN theory. The main features of the approach which make it different from the classical one are: (1) the study is non-asymptotic, that is, the sample size is xed and does not tend to infinity; (2) the parametric assumption is possibly misspecified and the underlying data distribution can lie beyond the given parametric family. The main results include a large deviation bounds for the (quasi) maximum likelihood and the local quadratic majorization of the log-likelihood process. The latter yields a number of important corollaries for statistical inference - concentration, confidence and risk bounds, expansion of the maximum likelihood estimate, etc. All these corollaries are stated in a non-classical way admitting a model misspecification and finite samples. However, the classical asymptotic results including the efficiency bounds can be easily derived as corollaries of the obtained non-asymptotic statements. The general results are illustrated for the i.i.d. set-up as well as for generalized linear and median estimation. The results apply for any dimension of the parameter space and provide a quantitative lower bound on the sample size yielding the root-n accuracy. We also discuss the procedures which allows to recover the structure when its e ective dimension is unknown.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Finite Sample Bernstein -- von Mises Theorem for Semiparametric Problems (2013, methodology): “The proof of this lemma can also be found in Spokoiny (2012).”
A modern maximum-likelihood theory for high-dimensional logistic regression (2018, background): “In parallel, it also developed a corresponding finite sample theory, relying on the earlier work [92].”
Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It (2014, methodology): “Learning Theory From the learning theory perspective, generalized Bayesian updating as in (45) with Z(γ) set to 1 can be seen as the result of a simple regularized loss minimization procedure (this was probably first noted by Williams (1980); see in particular Zhang (2006b)), which means that it con”
Bernstein - von Mises Theorem for growing parameter dimension (2013, methodology): “We present one of them concerning an expansion of the qMLE θ̃ ; see Corollary 3.4 in Spokoiny (2012).”
Critical dimension in the semiparametric Bernstein—von Mises theorem (2014, methodology): “In this paper we show that the bracketing approach of [27] can be used for obtaining a finite sample semiparametric version of the BvM theorem even if the full parameter dimension grows with the sample size.”
Adaptive Huber Regression (2017, background): “However, the effect of ARHGAP29 and MALL on breast cancer remains unclear and is worth further investigation.”
Finite-sample Analysis of M-estimators using Self-concordance (2018, methodology): “We mention once again that global assumptions in the vein of [49] about the deviations of the empirical risk, its gradient and Hessian can be avoided by using self-concordance.”