Information criteria for astrophysical model selection

By Andrew R. Liddle. 2007

932 citations recorded by the source corpus.

Model selection is the problem of distinguishing competing models, perhaps featuring different numbers of parameters. The statistics literature contains two distinct sets of tools, those based on information theory such as the Akaike Information Criterion (AIC), and those on Bayesian inference such as the Bayesian evidence and Bayesian Information Criterion (BIC). The Deviance Information Criterion combines ideas from both heritages; it is readily computed from Monte Carlo posterior samples and, unlike the AIC and BIC, allows for parameter degeneracy. I describe the properties of the information criteria, and as an example compute them from Wilkinson Microwave Anisotropy Probe 3-yr data for several cosmological models. I find that at present the information theory and Bayesian approaches give significantly different conclusions from that data.

Subfields: Astronomy and Astrophysics, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/d084823ff526a10e8329665e0338e69df64c6b8f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Astronomy and Astrophysics

1. Jon Fortney
2. N. Madhusudhan
3. C. Conroy
4. A. Showman
5. D. Charbonneau
6. L. Hernquist
7. H. Knutson
8. P. Hopkins

In Statistical and Nonlinear Physics

1. Will Handley
2. Jesús Torrado
3. V. Poulin
4. A. Lewis
5. Jon Fortney
6. M. Hobson
7. L. Verde
8. Julien Lesgourgues

Citations in the corpus

• Exploring cosmic origins with CORE: Inflation (2016, background): “which is a measure of the amount of information provided by the data D about lnRreh [247, 248].”
• SPITZER PHASE CURVE CONSTRAINTS FOR WASP-43b AT 3.6 AND 4.5 μm (2016, methodology): “We reject model combinations (including those with no visit-long ramp) that have higher Bayesian Information Criterion (BIC, Liddle 2007) values (see Table 2).”
• Spatial variation in the fine-structure constant – new results from VLT/UVES (2012, methodology): “We weight the sum by the penalised likelihood of the fits, via the AICC (see Liddle 2007).”
• The GAPS Programme with HARPS-N at TNG - XIV. Investigating giant planet migration history via improved eccentricity and mass determination for 231 transiting planets (2017, neutral): “Other indicators such as the Bayesian Information Criterion (BIC) or the Akaike Information Criterion (AIC; Liddle 2007) largely used in the literature (e.g. by P11 and H12) may suffer from several problems.”
• RETIRED A STARS AND THEIR COMPANIONS. III. COMPARING THE MASS–PERIOD DISTRIBUTIONS OF PLANETS AROUND A-TYPE STARS AND SUN-LIKE STARS (2009, methodology): “…subject of Bayesian Model Comparison.11 A commonly used model selection tool is the Bayesian Information Criterion (BIC; Schwarz 1978; Liddle 2004; Liddle 2007), which rewards better-fitting models but penalizes models that are overly complex: BIC ≡ −2 lnLmax + k lnN, (1) where Lmax is the…”
• Multimodal nested sampling: an efficient and robust alternative to Markov Chain Monte Carlo methods for astronomical data analyses (2007, background): “Various altern tive information criteria for astrophysical model selection are discussed by Liddle (2007), but the evidence remains the preferred method.”
• A 6% measurement of the Hubble parameter at z∼0.45: direct evidence of the epoch of cosmic re-acceleration (2016, methodology): “In this way, they penalize over-fitting the data (for more details, see [78]).”
• KiDS-450 + 2dFLenS: Cosmological parameter constraints from weak gravitational lensing tomography and overlapping redshift-space galaxy clustering (2017, background): “Best & Carlin 2002, also see Liddle 2007; Trotta 2008; Spiegelhalter et al. 2014; Joudaki et al. 2017a), given by DIC ≡ χ2eff (θ̂ ) + 2pD. (19) The first term of the DIC is the best-fit effective χ2eff (θ̂ ) = −2 lnLmax, where θ̂ is the vector of varied parameters at the maximum likelihoodLmax of…”
• THE STELLAR INITIAL MASS FUNCTION IN EARLY-TYPE GALAXIES FROM ABSORPTION LINE SPECTROSCOPY. II. RESULTS (2012, background): “According to Jeffreys’ scale,∆(AIC) > 5 is judged as ‘strong’ evidence for one model over another, while∆(AIC) > 10 is judged as ‘decisive’ (Liddle 2007).”
• KiDS-450: testing extensions to the standard cosmological model (2016, methodology): “To aid in this aim, we follow Joudaki et al. (2017) in using the DIC (Spiegelhalter, Best & Carlin 2002, also see Kunz, Trotta & Parkinson 2006; Liddle 2007; Trotta 2008, and Spiegelhalter et al. 2014), given by the sum of two terms: DIC ≡ χ2eff (θ̂) + 2pD. (1) Here, the first term consists of the…”
• The Axis of Evil revisited (2006, result): “It is far from peculiar to the AOE effect: see for example the rather disparate conclusions regarding evidence against scale-invariance (nS = 1) as reported in Liddle (2007).”