Harshness in image classification accuracy assessment

By Giles M. Foody. 2008

367 citations recorded by the source corpus.

Thematic mapping via a classification analysis is one of the most common applications of remote sensing. The accuracy of image classifications is, however, often viewed negatively. Here, it is suggested that the approach to the evaluation of image classification accuracy typically adopted in remote sensing may often be unfair, commonly being rather harsh and misleading. It is stressed that the widely used target accuracy of 85% can be inappropriate and that the approach to accuracy assessment adopted commonly in remote sensing is pessimistically biased. Moreover, the maps produced by other communities, which are often used unquestioningly, may have a low accuracy if evaluated from the standard perspective adopted in remote sensing. A greater awareness of the problems encountered in accuracy assessment may help ensure that perceptions of classification accuracy are realistic and reduce unfair criticism of thematic maps derived from remote sensing.

Subfields: Artificial Intelligence, Ecology, Media Technology

View Paper: https://www.semanticscholar.org/paper/36aa54ea49b1315679408bf6aa1de4cd97ac5e8a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Ecology

In Media Technology

Citations in the corpus

Field spectroscopy for weed detection in wheat and chickpea fields (2013, methodology): “The 95% confidence limit (CL) was calculated for overall accuracy as shown by Foody (2008): CL = ±tN ,d−1 √ p (1 − p) N − 1 , (2) where p is the overall accuracy, tN,d–1 is the statistical value of 95% two-tailed testing for d samples, N is the total number of samples, and d is the sum of ground…”
Remotely sensed spectral heterogeneity as a proxy of species diversity: Recent advances and open challenges (2010, background): “Nonetheless, a number of problems arise when trying to create abrupt thresholds (boundaries) for identifying explicit spatial units such as patches (Foody, 2002, 2008b), leading some researchers to ask explicitly if a landscape can actually be viewed as crisp, with abrupt boundaries (Southworth et…”
Measuring and modelling biodiversity from space (2008, background): “…required level of accuracy should be defi ned for an application because in some instances the information provided may be more accurate than suggested in the map’s summary accuracy statement (DeFries and Los, 1999) and some applications may require quite modest levels of accuracy (Foody, 2008).”
Ground-level hyperspectral imagery for detecting weeds in wheat fields (2013, methodology): “The confidence limit (CL) units are percent and were calculated for overall accuracy as shown by Foody (2008) and presented in Eq.”
Assessing the accuracy of land cover change with imperfect ground reference data (2010, methodology): “However, numerous challenges are, or are perceived to be, encountered with the use of remote sensing for the derivation of information on land cover (Foody, 2002, 2008; Rindfuss et al., 2004; Strahler et al., 2006).”
Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment (2011, background): “…be pervasive in spite of harsh criticisms for decades 45 from many authors (Brennan and Prediger 1981, Aickin 1990, Foody 1992, Ma and Redmond 1995, Stehman 1997, Stehman and Czaplewski 1998, Foody 2002, Turk 2002, Jung 2003, Di Eugenio and Glass 2004, Foody 2004, Allouche et al. 2006, Foody 2008).”
Classification accuracy comparison: hypothesis tests and the use of confidence intervals in evaluations of difference, equivalence and non-inferiority (2009, background): “The provision of confidence limits to accompany classification accuracy statements has been encouraged (Stehman, 1997, 2000; Foody, 2008) but they may be particularly useful in comparative analyses.”
Sampling designs for accuracy assessment of land cover (2009, background): “Congalton (1991), Janssen and van der Wel (1994), Stehman and Czaplewski (1998), Congalton and Green (1999), Czaplewski (2003), Foody (2002, 2008), Strahler et al. (2006), Wulder et al. (2006), and Stehman and Foody (2009) provide general overviews of the basic methodology of accuracy assessment.”
Revisiting Kappa to account for change in the accuracy assessment of land-use change models (2011, background): “, 1999), although it depends on the specific case what level of agreement is considered acceptable (Foody, 2008).”