Classification accuracy comparison: hypothesis tests and the use of confidence intervals in evaluations of difference, equivalence and non-inferiority

By Giles M. Foody. 2009

246 citations recorded by the source corpus.

The comparison of classification accuracy statements has generally been based upon tests of difference or inequality when other scenarios and approaches may be more appropriate. Procedures for evaluating two scenarios with interest focused on the similarity in accuracy values, non-inferiority and equivalence, are outlined following a discussion of tests of difference (inequality). It is also suggested that the confidence interval of the difference in classification accuracy may be used as well as or instead of conventional hypothesis testing to reveal more information about the disparity in the classification accuracy values compared.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/ae9235fa1be4a0e1110ae2be57da51c88f5000fb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

1. Mahesh Pal
2. Anting Guo
3. Yingying Dong
4. Wenjiang Huang
5. Yu Jin
6. Yu Ren
7. Huichun Ye
8. Bei Cui

Citations in the corpus

• Feature Selection for Classification of Hyperspectral Data by SVM (2010, methodology): “to the estimated differences in classification accuracy [43].”
• Vegetation classification method with biochemical composition estimated from remote sensing data (2011, background): “Z > 1.96 or Z –1.96 would indicate the difference of the two error matrices being significant at the 5% significance level (Foody 2009).”
• Identification of banana fusarium wilt using supervised classification algorithms with UAV-based multi-spectral imagery (2020, methodology): “accuracy assessment; a confusion matrix was developed and the overall accuracy and the Kappa coefficient were calculated([58,59]).”
• Assessment of an Operational System for Crop Type Map Production Using High Temporal and Spatial Resolution Satellite Optical Imagery (2015, background): “184) and not a normal distribution, as suggested by Foody in [53], since the number of samples is smaller than 30 (10 random draws in our case).”