On the Estimation of Confidence Intervals for Binomial Population Proportions in Astronomy: The Simplicity and Superiority of the Bayesian Approach

By Ewan Cameron. Publications Astronomical Society of Australia, 2010

266 citations recorded by the source corpus.

Abstract I present a critical review of techniques for estimating confidence intervals on binomial population proportions inferred from success counts in small to intermediate samples. Population proportions arise frequently as quantities of interest in astronomical research; for instance, in studies aiming to constrain the bar fraction, active galactic nucleus fraction, supermassive black hole fraction, merger fraction, or red sequence fraction from counts of galaxies exhibiting distinct morphological features or stellar populations. However, two of the most widely-used techniques for estimating binomial confidence intervals — the ‘normal approximation’ and the Clopper & Pearson approach — are liable to misrepresent the degree of statistical uncertainty present under sampling conditions routinely encountered in astronomical surveys, leading to an ineffective use of the experimental data (and, worse, an inefficient use of the resources expended in obtaining that data). Hence, I provide here an overview of the fundamentals of binomial statistics with two principal aims: (I) to reveal the ease with which (Bayesian) binomial confidence intervals with more satisfactory behaviour may be estimated from the quantiles of the beta distribution using modern mathematical software packages (e.g. r, matlab, mathematica, idl, python); and (ii) to demonstrate convincingly the major flaws of both the ‘normal approximation’ and the Clopper & Pearson approach for error estimation.

Subfields: Astronomy and Astrophysics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/85fb6230f93818850274db945af5f5474a9f6910

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Astronomy and Astrophysics

In Statistics and Probability

Citations in the corpus

Galaxy evolution in groups and clusters: satellite star formation histories and quenching time-scales in a hierarchical Universe (2012, background): “1 shows the fraction of infall types for satellites in the simulation at z = 0.05, as a function of and current host halo mass. Shaded widths show 68% confidence interval as given by a beta distribution (Cameron 2011).”
CANDELS: CONSTRAINING THE AGN–MERGER CONNECTION WITH HOST MORPHOLOGIES AT z ∼ 2 (2011, methodology): “The error bars on each fraction reflect the 68.3% binomial confidence limits given the number of sources in each category, calculated using the method of Cameron (2011).”
The Neutral Hydrogen Cosmological Mass Density at z = 5 (2015, methodology): “We calculate the binomial confidence intervals on kreal and kfound using the method described by Cameron (2011).”
The environmental history of group and cluster galaxies in a Λ cold dark matter universe (2011, methodology): “Error bars denote confidence intervals corresponding to a probability of 68 per cent (1σ) from quartiles of the beta distribution (for details on the method, see Cameron 2011).”
Precise Radial Velocities of Giant Stars VII. Occurrence Rate of Giant Extrasolar Planets as a Function of Mass and Metallicity (2014, methodology): “68.3% confidence limits on planet occurrence rates are also given; they were calculated based on binomial statistics following the Bayesian approach (Cameron 2011).”
Evidence of Environmental Dependencies of Type Ia Supernovae from the Nearby Supernova Factory indicated by Local H$\alpha$ (2013, background): “(5) By definition of the Iaα/Ia subgroups, the SNfactory sample sets the normalization to ψ(z = 0.05) = 41/82 = 50 ± 5% (Cameron 2011); hence K = 0.90 ± 0.15.”
The mean star formation rate of X-ray selected active galaxies and its evolution from z~2.5: results from PEP-Herschel (2012, methodology): “The PACS detection fractions, estimated using Bayesi n binomial statistics (Cameron 2011), of AGNs above and below our fiducial boundary between unobscured and obscured AGNs (solid curve:NH = 1022 cm−2) are comparable in all four of our redshift bins.”
The Sloan Digital Sky Survey Reverberation Mapping Project: Hα and Hβ Reverberation Measurements from First-year Spectroscopy and Photometry (2017, background): “Statistically speaking, this suggests that we likely have a similar number of “false-positive” positive Hβ lags as well, which is a 6.3+7.3−2.1% false-positive rate (calculations of uncertainties follow Cameron 2011).”
Why does the environmental influence on group and cluster galaxies extend beyond the virial radius (2012, methodology): “2, are calculated as binomial 1σ confidence intervals using a beta distribution generator as proposed by Cameron (2011).”
Morphologies of z ∼ 0.7 AGN host galaxies in CANDELS: no trend of merger incidence with AGN luminosity (2014, methodology): “Binomial probabilities and their confidence intervals for the AGN sample are derived using a beta statistic (Cameron 2011).”