Comparing the accuracies of remote sensing global burned area products using stratified random sampling and estimation

The accuracies of six global burned area (BA) products for year 2008 were compared using the same validation methods and reference data to quantify accuracy of each product. The selected products include MCD64, MCD45 and Geoland2, and three products developed within the Fire Disturbance project (fire_cci), which is part of the European Space Agency's (ESA) Climate Change Initiative (CCI) program. The latter three products were derived from MERIS and VEGETATION sensors (one product from each sensor separately, and a third one from the merging of MERIS and VGT products). The reference fire perimeters were mapped from two multi-temporal Landsat TM/ETM. + images at 103 non-overlapping Thiessen scene areas (TSA) selected with a stratified random sampling design. The validation results were based on cross tabulated error matrices from which six accuracy measures were computed following the requirements of end-users of burned area products. While overall accuracy (OA) exceeded 99% for all products, overall accuracy was lower for the burned class. Burned area commission error ratio was above 40% for all products and omission error ratio was above 65% for all products. The statistical significance of differences in accuracy between pairs of products was evaluated based on theory of the stratified combined ratio estimator. Statistical tests identified the MCD64 as the most accurate product, followed by MCD45 and the MERIS product.