A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation

By Cyril Goutte, Éric Gaussier. 2005

2,192 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

In Evaluation & Benchmarks

Citations in the corpus

Model Cards for Model Reporting (2018, methodology): “Confidence intervals for metrics derived from confusion matrices can be calculated by treating the matrices as probabilistic models of system performance [24].”
PathVQA: 30000+ Questions for Medical Visual Question Answering (2020, background): “…metrics: (1) exact match [2], which measures the percentage of inferred answers that match exactly with the ground-truth; (2) Macro-averaged F1 [33], which measures the average overlap between the predicted answers and ground-truth, where the answers are treated as bag of tokens; (3) BLEU…”
Within the Lack of Chest COVID-19 X-ray Dataset: A Novel Detection Model Based on GAN and Deep Transfer Learning (2020, background): “The most widespread performance measures in the ﬁeld of deep learning are Precision, Sensitivity (recall), F1 Score [65] and they are presented from Equation (9) to Equation (11).”
Assessment of Low Density Full-Waveform Airborne Laser Scanning for Individual Tree Detection and Tree Species Classification (2014, positive): “Three measures were calculated as follows [1,52,53]:”
Individual tree detection from Unmanned Aerial Vehicle (UAV) derived canopy height model in an open canopy mixed conifer forest (2017, positive): “[66], using the following equations [67,68]:”
Semantics-driven Event Clustering in Twitter Feeds (2015, methodology): “To assess the signiﬁcance of the di ↵ erences in F 1 measure be-tween our three systems, we used a Bayesian technique suggested by Goutte et al. [9].”
A systematic analysis of performance measures for classification tasks (2009, positive): “Our formal discussion of ML performance measures complements popular statistical and empirical comparisons such as the ones presented in Goutte and Gaussier (2005). We show that, in some learning settings, the correct identification of positive examples may be important whereas in others, the correc”
A review of ensemble learning and data augmentation models for class imbalanced problems: combination, implementation and evaluation (2023, background): “We note that typical metrics do not apply to CI problems, and specialised metrics have been developed, such as the precision and recall curve [8], and the F1 score [9].”
Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks (2015, methodology): “We computed the following evaluation measurements [34]: recall (R), precision (P ), Fβ score (β = 1, i.”
A data-driven approach to predicting diabetes and cardiovascular disease with machine learning (2019, result): “In other words, F1 score [29] is the harmonic average of the precision and recall allowing for comparison of different model performance in identifying true disease predictions when compared to false positives.”
VirFinder: a novel k-mer based tool for identifying viral sequences from assembled metagenomic data (2017, background): “In fact, P(V|Z = 1) is equivalent to precision, the fraction of predicted viral contigs that are true [64].”
LTF: A Label Transformation Framework for Correcting Label Shift (2020, methodology): “As done in BBSE (Lipton et al., 2018) and RLLS (Aziz-zadenesheli et al., 2019), the accuracy and F1 score (Goutte & Gaussier, 2005) are used as evaluation metrics, allowing us to compare the performance of different methods more comprehensively (Azizzadenesheli et al., 2019).”