Why your model parameter confidences might be too optimistic - unbiased estimation of the inverse covariance matrix

By J. Hartlap, P. Simon, P. Schneider. 2006

677 citations recorded by the source corpus.

Aims. The maximum-likelihood method is the standard approach to obtain model fits to observational data and the corresponding confidence regions. We investigate possible sources of bias in the log-likelihood function and its subsequent analysis, focusing on estimators of the inverse covariance matrix. Furthermore, we study under which circumstances the estimated covariance matrix is invertible. Methods. We perform Monte-Carlo simulations to investigate the behaviour of estimators for the inverse covariance matrix, depending on the number of independent data sets and the number of variables of the data vectors. Results. We find that the inverse of the maximum-likelihood estimator of the covariance is biased, the amount of bias depending on the ratio of the number of bins (data vector variables), p , to the number of data sets, n . This bias inevitably leads to an – in extreme cases catastrophic – underestimation of the size of confidence regions. We report on a method to remove this bias for the idealised case of Gaussian noise and statistically independent data vectors. Moreover, we demonstrate that marginalisation over parameters introduces a bias into the marginalised log-likelihood function. Measures of the sizes of confidence regions suffer from the same problem. Furthermore, we give an analytic proof for the fact that the estimated covariance matrix is singular if $p>n$.

Subfields: Applied Mathematics, Global and Planetary Change, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/afd394b90a16768ef1cb3e2f6aac110c02e57e07

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Mathematics

In Global and Planetary Change

In Statistics and Probability

Citations in the corpus

Cosmological parameters from the BOSS galaxy power spectrum (2019, methodology): “In our analysis we neglect the Hartlap factor correction [65] which affects the covariance matrix at the level of ∼ 1%.”
The VIMOS Public Extragalactic Redshift Survey (VIPERS) - Galaxy clustering and redshift-space distortions at z ≃ 0.8 in the first data release (2013, methodology): “As a final remark, we note that we use the direct inverse of the covariance matrix without applying the correction discussed by Hartlap et al. (2007), as it is not clear how the size of the correction is affected by the shrinkage estimation technique.”
The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: RSD measurement from the power spectrum and bispectrum of the DR12 BOSS galaxies (2016, methodology): “We account this effect by applying the correction proposed by Hartlap et al. (2007).”
The Subaru FMOS galaxy redshift survey (FastSound). IV. New constraint on gravity theory from redshift space distortions at z ∼ 1.4 (2015, background): “The underestimation of the covariance due to the finite number of realizations is corrected for following Hartlap et al. (2007), C−1 = Nmock −Nbin − 2 Nmock − 1 Ĉ−1.”
HALO OCCUPATION DISTRIBUTION MODELING OF CLUSTERING OF LUMINOUS RED GALAXIES (2008, background): “This holds as long as the number of jackknife realizations, n, is significantly larger than the dimension of the data vector, p. Hartlap et al. (2007) discusses related issues, pointing out a potential bias in a general model fitting which depends on the ratio of p to n.”
Evidence of the accelerated expansion of the Universe from weak lensing tomography with COSMOS (2009, background): “To obtainan unbiased estimate for the inverse covarianceC−1, we apply the correction Ĉ−1 = c Ĉ∗ −1 = n− p− 2 n− 1 Ĉ∗ −1 for p < n− 2 (17) discussed in Hartlap et al. (2007), wheren = 288 is the number of independent realisations andp is the dimension of the data vector.”
The VIMOS Public Extragalactic Redshift Survey (VIPERS): The growth of structure at $0.5 < z < 1.2$ from redshift-space distortions in the clustering of the PDR-2 final sample (2016, background): “It takes into account the typical skewness characterising an inverse Wishart distribution and is capable of providing an unbiased estimate of the precision matrix (Hartlap et al. 2007).”
Putting the precision in precision cosmology: How accurate should your data covariance matrix be? (2012, background): “…of the sampled precision matrix is biased and, assuming the mean in unknown, given by (Kaufman, 1967; see Press, 1982; Anderson, 2003; see also Hartlap et al., 2007, for a first application to cosmology) 〈Ψ̂〉 = NS − 1 NS −ND − 2 Ψ. (24) If NS > ND+2 is not satisfied then the values of M̂ are…”
The completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: BAO and RSD measurements from anisotropic clustering analysis of the quasar sample in configuration space between redshift 0.8 and 2.2 (2020, background): “Consequently, the precision matrix needs to be corrected following Anderson (2003) and Hartlap et al. (2007) C−1debiased = Nm − Nb − 2 Nm − 1 〈 Ĉ−1∗ 〉 for Nb < Nm − 2, (9) where Nb represents the number of bins in the data vector and Nm is the number of synthetic mocks.”
Cosmology from cosmic shear power spectra with Subaru Hyper Suprime-Cam first-year data (2018, background): “This noise in the covariance matrix may affect the analysis in several ways, such as the unstable inversion of the covariance matrix and noise bias in derived constraints (e.g., Hartlap et al. 2007; Norberg et al. 2009; Friedrich et al. 2016).”
The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: observational systematics and baryon acoustic oscillations in the correlation function (2016, methodology): “These factors are defined in Hartlap et al. (2007); Dodelson & Schneider (2013); Percival et al. (2014) and we apply them in the same way as in, e.g., Anderson et al. (2014b).”
COSMOS: Three-dimensional Weak Lensing and the Growth of Structure (2007, background): “We do not include the multiplicative factor suggested by Hartlap et al. (2007).”
The WiggleZ Dark Energy Survey: Direct constraints on blue galaxy intrinsic alignments at intermediate redshifts (2009, methodology): “In order to avoid overly optimistic constraints on model parameters due to inversion of the noisy jackknife covariance matrices, we apply the correction described by Hartlap et al. (2007) (equation 17 in that paper) after inversion.”
Planck2018 results (2018, methodology): “Second, we also rescale our inverse covariance matrix by the factor (Hartlap et al. 2007)”
The completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: BAO and RSD measurements from the anisotropic power spectrum of the quasar sample between redshift 0.8 and 2.2 (2020, methodology): “In this expression, W is the inverse of the covariance matrix computed from mocks which we correct for the finite number of mocks following Hartlap et al. (2007): W = N − n − 2 N − 1 C −1, (18) where N is the total number of mocks, and n is the number of data points.”