Extreme deconvolution: Inferring complete distribution functions from noisy, heterogeneous and incomplete observations

By Jo Bovy, David W. Hogg, Sam T. Roweis. 2009

135 citations recorded by the source corpus.

We generalize the well-known mixtures of Gaussians approach to density estimation and the accompanying Expectation-Maximization technique for finding the maximum likelihood parameters of the mixture to the case where each data point carries an individual d-dimensional uncertainty covariance and has unique missing data properties. This algorithm reconstructs the error-deconvolved or “underlying” distribution function common to all samples, even when the individual data points are samples from different distributions, obtained by convolving the underlying distribution with the unique uncertainty distribution of the data point and projecting out the missing data directions. We show how this basic algorithm can be extended with Bayesian priors on all of the model parameters and a “split-and-merge” procedure designed to avoid local maxima of the likelihood. We apply this technique to a few typical astrophysical applications.

Subfields: Astronomy and Astrophysics, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/e85472feaefaa20f00daf5eecfaf170b3217abba

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Astronomy and Astrophysics

In Statistical and Nonlinear Physics

Citations in the corpus

Co-formation of the disc and the stellar halo (2018, methodology): “To track the behavior of the stellar halo’s velocity ellipsoid, we model the stellar distributions shown in Figure 2 with a mixture of multi-variate Gaussians using the Extreme Deconvolution algorithm described in Bovy et al. (2011).”
THE SDSS-III BARYON OSCILLATION SPECTROSCOPIC SURVEY: QUASAR TARGET SELECTION FOR DATA RELEASE NINE (2011, methodology): “However, tests of the Year One data with the XDQSO method (Bovy et al. 2011) showed it selected about 1 z > 2.2 quasar deg−2 more than Likelihood.”
Kinematics in Young Star Clusters and Associations with Gaia DR2 (2018, methodology): “To evaluate this effect, we include uncertainties on median vout in a statistical model akin to the Extreme Deconvolution method of Bovy et al. (2011).”
THE SPATIAL STRUCTURE OF MONO-ABUNDANCE SUB-POPULATIONS OF THE MILKY WAY DISK (2011, methodology): “…uncertainties of 0.2 dex in [Fe/H] and 0.15 dex in [α/Fe], we first model the underlying abundance distribution using two Gaussian components, and fit this model to the observed distribution with the assumed abundance uncertainties using the extreme-deconvolution technique (Bovy et al. 2011).”
Proper motions and dynamics of the Milky Way globular cluster system fromGaiaDR2 (2018, methodology): “This method was generalized by Bovy et al. (2011) to the case of noisy data, under the name Extreme Deconvolution.”
The clustering of intermediate-redshift quasars as measured by the Baryon Oscillation Spectroscopic Survey (2012, methodology): “XD is applied in BOSS to model the distributions of quasars and stars in flux space, and hence to separate quasar targets from stellar contaminants (XDQSO; Bovy et al. 2011).”
Extracting distribution parameters from multiple uncertain observations with selection biases (2018, background): “The problem of extracting the distributional properties of a population of sources based on a set of observations drawn from that distribution is a common one, frequently labeled as hierarchical modelling (e.g., Hogg et al. 2010) (Bovy et al. (2011) call this “extreme deconvolution”).”
THE SDSS-IV EXTENDED BARYON OSCILLATION SPECTROSCOPIC SURVEY: QUASAR TARGET SELECTION (2015, methodology): “2 shows the typical positions of XDQSOz PQSO(z > 0.9) > 0.2 quasars in SDSS colors.”