Assessing the accuracy of land cover change with imperfect ground reference data

By Giles M. Foody. 2010

344 citations recorded by the source corpus.

The ground data used as a reference in the validation of land cover change products are often not an ideal gold standard but degraded by error. The effects of ground reference data error on the accuracy of land cover change detection and the accuracy of estimates of the extent of change was evaluated. Twelve data sets were simulated to allow the exploration of the impacts of a spectrum of ground data imperfections on the estimation of the producer’s and user’s accuracy of change as well as of change extent. Simulated data were used since this ensured that the actual properties of the data were known and to exclude effects due to other sources of ground reference data error; although the impacts of simulated reference data on two real confusion matrices is also illustrated. The imperfections evaluated ranged from the inclusion of small amounts of known error into the ground reference data through to the extreme situation in which ground data were absent. The results show that even small amounts of error in the ground reference data can introduce large error into studies of land cover change by remote sensing and reinforce the desire to avoid the expression ground truth as this might imply that the data are a gold standard reference. The effect of reference data imperfections was dependent on the degree of association between the errors in the cross tabulated data sets. For example, in the scenarios investigated, a 10% error in the reference data set introduced an under-estimation of the producer’s accuracy of 18.5% if the errors were independent but an over-estimation of the producer’s accuracy of 12.3% if the errors were correlated. The magnitude of the mis-estimation of the producer’s accuracy was also a function of the amount of change and greatest at low levels of change. The amount of land cover change estimated also varied greatly as a function of ground reference data error. Some possible methods to reduce or even remove the impacts of ground reference data error were illustrated. These ranged from simple algebraic means to estimate the actual values of accuracy and change extent if the imperfections were known through to a latent class analysis that allowed the assessment of classification accuracy and estimation of change extent without the use of ground reference data if the underlying model is defined appropriately.

Subfields: Ecology, Environmental Engineering, Media Technology

View Paper: https://www.semanticscholar.org/paper/d49dc3841a120be02210c7f30a0246dfe42e7f5d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Ecology

In Environmental Engineering

In Media Technology

Citations in the corpus

Advances in Remote Sensing of Agriculture: Context Description, Existing Operational Monitoring Systems and Major Information Needs (2013, background): “Besides these problems, Foody (2010) [135] points out the importance of ground truthing.”
Good practices for estimating area and assessing accuracy of land change (2014, methodology): “Small errors in the reference data set can lead to large biases of the estimators of 674 both classification accuracy and class area (Foody, 2010; 2013).”
Automated cropland mapping of continental Africa using Google Earth Engine cloud computing (2017, background): “However, they are often not an ideal gold standard but degraded by error (Foody, 2010) because of small samples size, sampling bias and inconsistent labeling.”