• Home
• Introduction
• Methods
• Maps & Data
• Publications
• Contact Us

Accuracy Assessment

What to expect from the map

When looking at the land cover and impervious maps on this site, it is important to remember that no map is a perfect representation of reality. There are always errors in maps and we need to keep in mind how accurate they are, and whether that level of accuracy is sufficient for the ways we want to use the map information. Based on the 30-meter resolution of the Landsat data used to create these maps, it's important to keep in mind that these maps will be most accurate for viewing geographic patterns over larger areas (e.g., county or city, rather than neighborhood).

How accurate are these maps?

In general, accuracy is assessed by comparing the finished map to a second set of reference data -- not the data that was initially used for classifier training or modeling the relationship. By using a different set of known ground locations, we can check how accurately the model was able to extrapolate the relationship (between the initial set of ground data and its associated pixels) across the entire image.

The result of an accuracy assessment provides us with an overall accuracy of the map based on an average of the accuracies for each class in the map. For example in a land cover map the water class could be very accurate but some of the vegetation classes might be less accurate. Or, in the case of Urban/Developed areas, the heavily developed areas are usually more accurately identified then the lightly developed. Thus, categories of imperviousness (80-100%) are more consistently identified as Urban/Developed than the lightly developed (0-10%).

Using this method, the overall accuracies for the maps found in this website are:

Assessing accuracy can also be affected by how many reference samples are used, how well they align with the map locations, even how correct the reference data is (we assume the reference data is 100% correct when we assess classification accuracy, but recognize that in reality there could be location, as well as thematic errors).

How accurate are the maps depicting change?

Determining the accuracy of change detection maps is even more complex. Change maps are not “predictive” maps, but instead a representation of the change that has occurred during a certain historical period of time. The specific periods are based on when clear satellite imagery is available for creating the maps. We can sense the difference between one map and the next, however, issues such as position and labeling errors can propagate through the multiple dates and show change that did not truly occur. This is especially true when more than two dates are used in the analysis. This often makes it difficult to prove exactly if mapped change is real or an anomaly created by errors.

We have used several methods to determine change detection accuracy. The simplest is multiplying the accuracy of the two maps together to estimate the expected accuracy for the change map. For example, if the accuracy at time 1 was 90% and time 2 was 95%, the expected accuracy would be 85.5%. The theory is that there are errors associated with each of the two maps, and when these are overlaid, the errors are cumulative. As a result, the accuracy of the change map is lower than either of the original maps.

Using this method, the expected accuracies for the change maps found in this website are:

If you are interested in other methods that have been explored for determining change detection accuracy, see our publication on multitemporal change in the TCMA.