Myths and truths concerning estimation of power spectra: the case for a hybrid estimator
By G. Efstathiou. 2003
187 citations recorded by the source corpus.
It is widely believed that maximum-likelihood estimators must be used to provide optimal estimates of power spectra. Since such estimators require the inversion and multiplication of N d x N d matrices, where N d is the size of the data vector, maximum-likelihood estimators require at least of order N 3 d operations and become computationally prohibitive for N d greater than a few tens of thousands. Because of this, a large and inhomogeneous literature exists on approximate methods of power-spectrum estimation. These range from manifestly suboptimal but computationally fast methods, to near-optimal but computationally expensive methods. Furthermore, much of this literature concentrates on the power-spectrum estimates rather than the equally important problem of deriving an accurate covariance matrix. In this paper, I consider the problem of estimating the power spectrum of cosmic microwave background (CMB) anisotropies from large data sets. Various analytic results on power-spectrum estimators are derived, or collated from the literature, and tested against numerical simulations. An unbiased hybrid estimator is proposed that combines a maximum-likelihood estimator at low multipoles and pseudo-C estimates at high multipoles. The hybrid estimator is computationally fast (i.e. it can be run on a laptop computer for Planck-sized data sets), nearly optimal over the full range of multipoles, and returns an accurate and nearly diagonal covariance matrix for realistic experimental configurations (provided certain conditions on the noise properties of the experiment are satisfied). It is argued that, in practice, computationally expensive methods that approximate the O(N 3 d) maximum-likelihood solution are unlikely to improve on the hybrid estimator, and may actually perform worse. The results presented here can be generalized to CMB polarization and to power-spectrum estimation using other types of data, such as galaxy clustering and weak gravitational lensing.
Subfields: Astronomy and Astrophysics
View Paper: https://www.semanticscholar.org/paper/04140dadb040d086b00011fc66583cc44e29e385
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Astronomy and Astrophysics
Citations in the corpus
- Fast estimation of polarization power spectra using correlation functions (2003, methodology): “We used the following approximation to the covariance of the recovered beam-smoothed CEl s (see Efstathiou 2003 for a derivation of this formula for the temperature anisotropies): cov(ĈEl , Ĉ E l′ ) ≈ 1 2π(w2fsky)2 ∑ L [ CEl C E l′ ∑ M |(w2P )LM |2 + 2 √ CEl C E l′ ∑ M (w2Pσ 2 P )LM (w 2 P ) ∗…”
- Planck 2015 results. XI. CMB power spectra, likelihoods, and robustness of parameters (2015, methodology): “The CamSpec, Hillipop, and Mspec codes are, like Plik, based on pseudo-C` estimators and an analytic calculation of the covariance (Efstathiou 2004, 2006), with some differences in the approximations used to calculate this covariance. The Xfaster code (Rocha et al. 2011) is an an approximation to th”
- Planck intermediate results. XLVI. Reduction of large-scale systematic effects in HFI polarization maps and estimation of the reionization optical depth (2016, unknown): “QML estimators have been widely discussed in the literature (e.g., Tegmark 1996; Bond et al. 1998; Efstathiou 2004, 2006).”
- A unified pseudo-Cℓ framework (2018, background): “…the pseudo-C` approach is a good choice for non-steep power spectra for which the typical correlation length is smaller than the mask structure, and in general on small scales, where the impact of the power spectrum shape drops as the number of available modes increases (Efstathiou 2004, 2006).”
- Localized spectral analysis on the sphere (2005, background): “Truncating this matrix to be square, it is possible under some conditions to obtain a deconvolved unbiased estimate of the power spectrum by simple matrix inversion (Efstathiou 2004).”