Good practices for estimating area and assessing accuracy of land change

By Pontus Olofsson, Giles M. Foody, Martin Herold, Stephen V. Stehman, Curtis E. Woodcock, M. Wulder. 2014

2,789 citations recorded by the source corpus.

The remote sensing science and application communities have developed increasingly reliable, consistent, and robust approaches for capturing land dynamics to meet a range of information needs. Statistically robust and transparent approaches for assessing accuracy and estimating area of change are critical to ensure the integrity of land change information. We provide practitioners with a set of “good practice” recommendations for designing and implementing an accuracy assessment of a change map and estimating area based on the reference sample data. The good practice recommendations address the three major components: sampling design, response design and analysis. The primary good practice recommendations for assessing accuracy and estimating area are: (i) implement a probability sampling design that is chosen to achieve the priority objectives of accuracy and area estimation while also satisfying practical constraints such as cost and available sources of reference data; (ii) implement a response design protocol that is based on reference data sources that provide sufficient spatial and temporal representation to accurately label each unit in the sample (i.e., the “reference classification” will be considerably more accurate than the map classification being evaluated); (iii) implement an analysis that is consistent with the sampling design and response design protocols; (iv) summarize the accuracy assessment by reporting the estimated error matrix in terms of proportion of area and estimates of overall accuracy, user's accuracy (or commission error), and producer's accuracy (or omission error); (v) estimate area of classes (e.g., types of change such as wetland loss or types of persistence such as stable forest) based on the reference classification of the sample units; (vi) quantify uncertainty by reporting confidence intervals for accuracy and area parameters; (vii) evaluate variability and potential error in the reference classification; and (viii) document deviations from good practice that may substantially affect the results. An example application is provided to illustrate the recommended process.

Subfields: Ecology, Environmental Engineering, Global and Planetary Change

View Paper: https://www.semanticscholar.org/paper/26c139be2f68d180682ca2266e60e84c5beff03a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Ecology

In Environmental Engineering

In Global and Planetary Change

Citations in the corpus

Global 10 m Land Use Land Cover Datasets: A Comparison of Dynamic World, World Cover and Esri Land Cover (2022, methodology): “Although our analysis provides important information about global model accuracies, it remains to be seen which LULC product provides area change estimates with the least amount of error as measured using design-based inference for area estimation [35,36].”
Reconstructing Three Decades of Land Use and Land Cover Changes in Brazilian Biomes with Landsat Archive and Earth Engine (2020, methodology): “, global user and producer accuracies, quantity and allocation disagreement) were calculated for each year, class and biome, following a standard good practice protocol [51].”
Global Land Cover Mapping: A Review and Uncertainty Analysis (2014, methodology): “Some important work has begun in this area [61].”
Global declines in human‐driven mangrove loss (2020, methodology): “We assessed the accuracy of our land use change maps and estimated the uncertainty of area measurements using recent best practices (Olofsson et al., 2014).”
Rates and drivers of mangrove deforestation in Southeast Asia, 2000–2012 (2015, methodology): “To assess the potential impacts of systematic errors on the conclusions of the study, we calculated error-corrected estimates of the areal coverage of each land use and SEs for these estimates following recommended guidelines (49) for the overall and annual percentages of mangrove converted to each ”
Joint Deep Learning for land cover and land use classification (2019, neutral): “(Olofsson et al., 2014).”
Land-cover classification with high-resolution remote sensing images using transferable deep models (2018, methodology): “Evaluation Metrics: To evaluate our algorithm, we assess the experimental results with Kappa coefficient (Kappa), overall accuracy (OA), and user’s accuracy [102].”
Ongoing primary forest loss in Brazil, Democratic Republic of the Congo, and Indonesia (2018, methodology): “Sample-based analyses result in unbiased estimates of the mapped land cover themes (Stehman 2013, Olofsson et al 2014).”
Land cover and land use classification performance of machine learning algorithms in a boreal landscape using Sentinel-2 data (2019, methodology): “In this way, the reference data can reliably replicate on-ground conditions and meet the good practice guidance for accuracy assessment (Olofsson et al. 2014).”
Remote sensing of forest pest damage: a review and lessons learned from a Canadian perspective* (2016, methodology): “These reference values, however, are themselves subject to uncertainty, resulting in a need to account for potential errors (Olofsson et al. 2014).”
Aboveground carbon loss in natural and managed tropical forests from 2000 to 2012 (2015, methodology): “To this end, we demonstrate a generic and cost-effective approach for estimating forest cover loss activity data that follows good practice guidance (IPCC 2006, GFOI 2014, Olofsson et al 2014).”