Accuracy Assessment

Introduction

The application of GIS in Remote Sensing, particularly the land cover classification has contributed a lot to the scientific community (read: What is the importance of GIS (Geographic Information Science) in Remote Sensing?) and at the same time has led to the need for accurate assessment of the products because most scientists believe that these products are not free of uncertainties. These uncertainties may result from different sources including atmospheric particles, clouds, and cloud shadows, and even the classification approaches we adopt to classify the images. Some of the uncertainties such as atmospheric particles could be somehow rectified using atmospheric correction techniques (read: What is an Atmospheric Correction and why is it necessary?), however, the uncertainties from the classification cannot be rectified and therefore, there is a need of accuracy assessment to examine how accurate our classification is.

Congalton (2004) pointed out the need for accuracy assessment for these three reasons:

1. To perform a self-evaluation and learn from own mistakes
2. To compare methods/algorithms/analysts quantitatively
3. ?To use the resulting maps/spatial information in some decision-making-process

What is accuracy assessment?

"The accuracy of a land cover classification is the degree to which the map land cover agrees with the reference land cover classification (i.e. ground condition).”

Stehman (2009)

Accuracy assessment is the process of evaluating the accuracy of a model or measurement system by comparing the predicted or observed values to the actual or true values. It is a crucial step in evaluating the performance of models or measurement systems, and it is commonly used in fields such as machine learning, statistics, and remote sensing.

In the accuracy assessment, the land cover classification in the map is typically compared to the true land cover condition. This is done using the ‘ground truth’, or ‘reference data’.?Since the ground truth data is not practically attainable, researchers use the reference data such as higher-quality data. Again, it is not possible to obtain the reference land cover classification for the entire region of interest, therefore, a statistical sampling method is used to produce a ‘sample’ i.e. subset or portion of the region mapped, and an accuracy assessment is carried out.

In the accuracy assessment, the accuracy of a model or measurement system is typically measured using various statistical measures such as root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R-squared). These measures are used to compare the predicted or observed values to the true values and to assess the degree of error or deviation between them. The most common method for accuracy assessment at present is the error matrix where a confusion matrix displaying the proportion of area that is correctly classified and misclassified for the different land cover types is produced. It is helpful in estimating overall accuracy, user and producer accuracy, errors of omission and commission, and the Kappa Coefficient.?

What is the need for an Accuracy Assessment?

At present, accuracy assessment has become one of the integral components of land cover classification, why, because, both the researchers and readers have become aware of the uncertainties and fallback of the remotely sensed image classification. The interpretation of the remote sensing images could be misleading if they are not analyzed and presented well. The best way to check this disparity, producing the best thematic maps, and bringing the research to a certain standard, there is a need for accuracy assessment.

There are several other needs for accuracy assessment. Some of them are listed below:

1. To evaluate the performance of the model or measurement system: Accuracy assessment helps to determine how well a model or measurement system is performing. It helps to identify errors or biases and to evaluate the precision and accuracy of the measurements.
2. To identify sources of error: Accuracy assessment helps to identify the sources of error in a model or measurement system. It helps to determine whether errors are due to data collection, processing, or modeling.
3. To validate the model or measurement system: Accuracy assessment is essential for validating a model or measurement system. It helps to determine whether the model or measurement system is fit for purpose and whether it can be used reliably for the intended application.
4. To improve the model or measurement system: Accuracy assessment helps to identify areas where the model or measurement system can be improved. It helps to refine the model or measurement system, reduce errors, and improve overall performance.
5. To compare different models or measurement systems: Accuracy assessment helps to compare the performance of different models or measurement systems. It helps to determine which model or measurement system is more accurate, precise, and reliable.

Overall, accuracy assessment is necessary to ensure the reliability and validity of models and measurement systems, and to provide confidence in the results and predictions generated by them.

References:

• Congalton, R. G. (2004). Putting the map back in map accuracy assessment. Remote sensing and GIS accuracy assessment, 1-11.
• ?Stehman, S. V. (2009). Sampling designs for accuracy assessment of land cover. International Journal of Remote Sensing, 30(20), 5243-5272.