E-values: Calibration, combination and applications

By Vladimir G Vovk, Ruodu Wang. Annals of Statistics, 2019

317 citations recorded by the source corpus.

Multiple testing of a single hypothesis and testing multiple hypotheses are usually done in terms of p-values. In this paper we replace p-values with their natural competitor, e-values, which are closely related to betting, Bayes factors, and likelihood ratios. We demonstrate that e-values are often mathematically more tractable; in particular, in multiple testing of a single hypothesis, e-values can be merged simply by averaging them. This allows us to develop efficient procedures using e-values for testing multiple hypotheses.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/312a9dd54777bb85f339e96d819aba24c4a0f320

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

False discovery rate control with e‐values (2020, methodology): “r the next studies, whereas a generic e-value can provide a continuum of evidence strength. Even if the e-value is moderate (e.g., 1.5), it is useful and can be easily merged into the next experiment [18]. On the other hand, a p-value of 0.1 is diﬃcult to use for future studies, and may be discarded”
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (2024, methodology): “…and made more formally rigorous by using the theory of nonnegative supermartingales and E-values (Howard et al., 2020; Waudby-Smith & Ramdas, 2020; Vovk & Wang, 2021; Ramdas et al., 2023); this would deal with the dependence, but the variants we tried did not perform well in terms of power.”
Safe Testing (2019, background): “…(Lai, 2009; Siegmund, 2013), pioneered by Wald (1947) Related Work on Relating p -values and e -variables Shafer et al. (2011) and Shafer and Vovk (2019) give a general formula for calibrators f (see also Vovk and Wang (2021) and Vovk (1993); Sellke et al. (2001) for early work in this direction).”
Admissible anytime-valid sequential inference must rely on nonnegative martingales. (2020, background): “As before, a standard e-value [25] is not anytime-valid and exhibits the above property only at fixed times t, which does not suffice for sequential settings.”
Time-uniform central limit theory and asymptotic confidence sequences (2021, background): “7, 32, 33, 63 [56] Vladimir Vovk and Ruodu Wang.”
Admissible ways of merging p-values under arbitrary dependence (2020, methodology): “More generally, choosing a good value of r is discussed in detail in Section 6 of Vovk and Wang (2020b).”