Reasoning on the Evaluation of Wildres Risk Using the Receiver Operating Characteristic Curve and MODIS Images

L. Usero1 and M. Xose Rodriguez-Alvarez

1

Departamento de Ciencias de la Computaci on, Universidad de Alcal a 2 Universidad de Santiago de Compostela luis.usero@uah.es, mariajose.rodriguez.alvarez@usc.es

Abstract. This paper presents a method to evaluate the wildres risk using the Receiver Operating Characteristic (ROC) curve and Terra moderate resolution imaging spectroradiometer (MODIS) images. To evaluate the wildres risk fuel moisture content (FMC) was used, the relationship between satellite images and eld collected FMC data was based on two methodologies; empirical relations and statistical models based on simulated reectances derived from radiative transfer models (RTM). Both models were applied to the same validation data set to compare their performance. FMC of grassland and shrublands were estimated using a 5-year time series (2001-2005) of Terra moderate resolution imaging spectroradiometer (MODIS) images. The simulated reectances were based on the leaf level PROSPECT coupled with the canopy level SAILH RTM. The simulated spectra were generated for grasslands and shrublands according to their biophysical parameters traits and FMC range. Both RTM-based models, empirical and statistical, oered similar accuracy with better determination coecients for grasslands. In this work, we have evaluated the accuracy of (MODIS) images to discriminate between situations of high and low re risk based on the FMC, by using the Receiver Operating Characteristic (ROC) curve. Our results show that none of the MODIS bands have a good discriminatory capacity (0.9984) when used separately, but the joint information provided by them oer very small misclassication errors.

Introduction

Fuel moisture content (FMC) is one of the variables that drive re danger. FMC is a quotient of these two variables, EWT and DM, that can be estimated independently. This variable conditions re, since the drier the vegetation the easier res ignite and propagate [2]. Water stress causes changes in the spectral reectance and transmittance of leaves [16]. Direct estimation by eld sampling provides the most accurate method to obtain FMC [15], commonly using gravimetric methods, namely the weight dierence between fresh and dry samples [12]. However, this approach is very costly and the generalization to regional or global scales results unfeasible. The use of meteorological indices is widespread,
J. Mira et al. (Eds.): IWINAC 2009, Part I, LNCS 5601, pp. 476485, 2009. c Springer-Verlag Berlin Heidelberg 2009

Reasoning on the Evaluation of Wildres Risk

477

since they provide an easy spatial and diachronic estimation of FMC [3], but they also present operational diculties since the weather stations are often located far from forested areas and maybe scarce in re prone regions. Furthermore, these estimations are reasonably well suited for dead fuels, because their water content is highly related to atmospheric conditions. However, in live fuels, species physiological characteristics and adaptation to drought imply a great diversity of moisture conditions with the same meteorological inputs [18]. Radiative transfer models are theoretical models with a strong physical basis, generally based on the theory of radiative transfer is its name. To calculate the solar irradiance reaching Earths surface, simulating the processes that suers through the atmosphere. Radiative transfer models show that these spectral measurements are related to equivalent water thickness (EWT), water content per area unit, and dry matter content (DM), matter content per area unit [5]; [10]. The PROSPECT model is a radiative transfer model designed and developed by S. Jacquemoud and F. Baret in 1990 and is one of the models used in many scientic experiments. This model is based on the model proposed by Allen, which represents the optical properties of leaves from 400nm to 2500nm wavelength. The dispersion is described by a spectral index of refraction (n) and a parameter that characterizes the internal structure of the leaf (N). The absorption is modeled using the concentration of pigments (chlorophyll a + b), water content (Cw) and their corresponding spectral absorption coecients. SAILH model is a simulation model of the reectivity at the canopy based on the Kubelka-Munk theory used to describe the interaction of radiation with plant canopies in a fast and under controlled conditions. This model is a variant of the SAIL model developed by Wout Verhoef in 1984 belongs to the type of canopy-level models as models of turbidity. As SAIL, SAILH model is a model of a turbidity level of canopy, so it treats the vegetation as an innitely extended horizontal layer, at and uniform, consisting of elements of vegetation randomized in parallel layers make small reective particles that absorb, depending on their optical properties (reectivity and transmissivity). SAILH can simulate the canopy reectance at given the reectivity and transmissivity of the leaf, the reectivity of the soil, leaf area index (LAI) The Receiver Operating Characteristic (ROC) analysis comes from statistical decision theory [Green and Swets, 1966] and was originally used during World War II for the analysis of radar images. The rst applications of this theory within the medical area occurred during the late 1960s. Today the ROC analysis is a widespread method in the medical eld and many textbooks and articles, for example [Kraemer, 1992, Bamber, 1975,Metz, 1978, DeLong et al., 1988, Hanley, 1989, McClish, 1989, Armitage and Berry, 1994],have descriptions of it. From the computer science point of view, ROC analysis has been increasingly used as a tool to evaluate discriminate eects among dierent methods. The ROC curve relies heavily on notations as sensitivity and specicity and these values depend on the specic data set. Even though the values for sensitivity and specicity in theory lie in the interval [0; 1], in practice the borders are decided by the data set. If the QROC curve is considered instead a better way to compare dierent tests are given since all

478

L. Usero and M.X. Rodriguez-Alvarez

values have the same interval. This is accomplished by using two quality measures that transforms sensitivity and specicity values from dierent data sets to a comparable interval. MODIS (or Moderate Resolution Imaging Spectroradiometer) is a key instrument aboard the Terra (EOS AM) and Aqua (EOS PM) satellites. Terras orbit around the Earth is timed so that it passes from north to south across the equator in the morning, while Aqua passes south to north over the equator in the afternoon. Terra MODIS and Aqua MODIS are viewing the entire Earths surface every 1 to 2 days, acquiring data in 36 spectral bands, or groups of wavelengths (see MODIS Technical Specications). These data will improve our understanding of global dynamics and processes occurring on the land, in the oceans, and in the lower atmosphere. MODIS is playing a vital role in the development of validated, global, interactive Earth system models able to predict global change accurately enough to assist policy makers in making sound decisions concerning the protection of our environment.

Methods

The empirical approach was derived from multivariate linear regression (MLR) analysis between eld collected FMC data and reectance values derived from the moderate resolution imaging spectroradiometer (MODIS). The eld samples were divided in two sets: 60% for calibrating the model and the remaining 40% for the validation. Two dierent models were built for grasslands and shrublands. The simulation approach was derived from RTM that were parameterized using eld data, auxiliary information derived from MODIS products and the knowledge of the type of canopy architecture that dene which RTM is appropriate [7]. Once the simulated reectance values for grasslands and shrublands were obtained for the whole solar spectrum, they were convolved to the MODIS spectral wavelengths and band widths. Finally, separate MLR models between the simulated reectances and the grassland and shrubland FMC values were built, in a similar way to the empirical method. Those equations were applied to the MODIS data for the same validation dataset as the empirical model to compare the performances of both approaches. 2.1 Field Sampling

A eld campaign has been carried out by our research group since 1996 to the present in the Cabaneros National Park (Central Spain; Fig. 1) to collect samples of dierent Mediterranean species for eld FMC estimation. Three plots of grassland and two of shrubland (Cistus ladanifer L., Rosmarinus ocinalis L., Erica arborea L. and Phyllirea angustifolia L.) sized 30m 30 m, were collected in gentle slopes (5%) and homogeneous patches. For this paper, FMC values of C. ladanifer L. were selected as representative for shrubland plots since it is very common in Mediterranean siliceus areas. It appears in a 29.79% of the study area covering a radius of 100 km from the National Park being the dominant species in more than 6% versus less than

Reasoning on the Evaluation of Wildres Risk

479

Fig. 1. Map of Spain showing the location of Cabaeros National Park, as well as a false color composite Landsat image showing the midpoint of the shrubland (S1 and S2) and grassland (G1, G2 and G3) plots used in this analysis. The grey boxes indicate the 3 T 3 MODIS grid (1.5 km T 1.5 km) centered at the plots. Shaded boxes indicate the window adapted to the shrub shape plot.

16% of appearance and 1% of dominance of the other three species together in the same area. In addition to this, it is a typical pioneer species that regenerates easily by seeds after diverse types of handlings and disturbances (Nuez Olivera, 1988), so it is the primary colonizer in areas with recurrent wildres, which are of special interest in this study. The sampling protocol followed standard methods described in [6] and was repeated every 8 days during the spring and summer seasons from 1996 to 2002 and every 16 days from 2003 on. For this paper, FMC measurements taken from 2001 to 2005 have been used to correspond with the temporal series of the MODIS images. Cm2 FMC was computed from the dierence of fresh and dry weight as follows: F M C (%) = Wf Wd 100 Wd (1)

where Wf is fresh weight of leaves and small terminal branches (in the case of shrub species) or the whole plant (in the case of grassland), and Wd is dry weight, after oven drying the samples for 48 h at 60 8C. After 2004, FMC eld sampling incorporated the collection of variables that are critical for running the RTM at leaf level, such as dry matter content (DM), equivalent water content (EWT) and chlorophyll content (Ca + b). DM and EWT were computed as follows: DM = Wd A (2)

480

L. Usero and M.X. Rodriguez-Alvarez

and EW T =

Wf Wd A

(3)

where A is the leaf area. C. ladanifer L. leaf area was measured with an image analysis Delta system (Delta Devices LTD, Cambridge. England). Ca + b was measured by means of destructive sampling and measurement of leaf concentration in laboratory with the dimethyl sulfoxide (DMSO) method and spectrophotometric readings, according to [19]. For grasslands, DM and Ca + b measurements were provided by a eld ecologist working in similar environments (Valladares, personal communication). Spectral soil reectance was also measured with a GER 2600 (GER Corp., Millbrook, NY) radiometer to use as an input at canopy level model. 2.2 MODIS Data

Two standard products of the MODIS program were chosen for this study: the MODIS/Terra surface reectance (MOD09A1) and the MODIS-Terra leaf area index (LAI) and fraction of photosynthetically active radiation (FPAR) (MOD15A2). The rst is an 8-day composite product of atmospherically corrected reectance for the rst seven spectral bands of the MODIS sensor at a spatial resolution of 500m (Fig. 2). This product includes ancillary information, such as sun and sensor angles [17]. The standard MOD15A2 product was selected to take into account the strong eect of LAI variations on reectance as well as to parametrize the RTM. This product is generated daily at 1 km spatial resolution and composited over an 8-day period based on the maximum value of the FPAR for that period (Knyazikhin, 1999). The original products were downloaded from the Land Processes Distributed Active Archive Center (LP DAAC) of the United States Geological Survey (USGS) (http://edcimswww.cr.usgs.gov/pub/imswelcome/) and reprojected from sinusoidal to UTM 30 T Datum European 1950 (ED50) using nearest neighbour interpolation resampling. MOD15A2 data were resampled to 500m to match the resolution of the MOD09A1 product using the same interpolation algorith. The values of a given plot for comparing with the eld data were extracted from each composited image using the median value of a 3 X 3 pixel kernel located at the center of the eld plot. A 3 X 3 window was used in order to reduce the potential noise due to residual atmospheric eects and georeferencing errors. In the case of shrublands, extraction windows were adapted to the shape of shrub patches to avoid including mixed pixels. To verify this approach the coecient of variation (CV) was computed for reectances for a Landsat image (30m X 30m pixel size) within the extraction windows. The CV decreased from 0.052 and 0.255 of the 3 3 windows in the near infrared band (NIR) and the short wave infrared (SWIR) bands, respectively, to 0.050 and 0.195 with the adapted window. The extractions of reectance data of each pixel were derived from the 8-day composite that had a closest selected day to the eld collections. A wide range of vegetation indexes were calculated to be included as independent variables in the empirical MLR model. Only one form of the NDII using band 6

Reasoning on the Evaluation of Wildres Risk

481

(1628-1652 nm) was calculated based on previous studies which show stronger correlations between this band and eld measured FMC values than other MODIS bands in the SWIR region [14];[24]. The rst ve indices in Table 1 measure greenness variations, which are only indirectly related to leaf water content. The other indices included in Table 1 are more directly related to water content, by combining water absorption in the SWIR wavelengths with other bands that are insensitive to water content [9]. Although greenness indices do not include water absorption bands, they can be used as an indirect estimation of water content, since moisture variations aect chlorophyll activity, leaf internal structure and LAI of many Mediterranean plants (Bowyer and Danson, 2004). In this sense, as the plant dries, changes in leaf internal structure cause a decrease in the reectance in the NIR and an increase in the visible region, as a result of reducing photosynthetic activity and LAI values. However, this relation cannot be generalized for all ecosystems because, for example, variations on chlorophyll content can also be caused by plant nutrient deciency, disease, toxicity and phonological stage [4].

Data Analysis and Conclusions

Appropriate statistical techniques for evaluating the accuracy of a given (continuous) classier in distinguishing between two states (S1 and S2) are based on ROC curve analysis [1, 2]. Let Y a continuous variable. Based on the values of Y, the clasication of an observation as belonging to the state S1 or S2, can be made by choosing a threshold value c: if the observation is classied as S1 and if as S2. In this situation, the ROC curve is dened as the plot of the truepositive rate (TPR, the probability of correct classication for S1) versus the false-positive rate (FPR, the probability of misclassication for S2), across all possible threshold values. Related to the ROC curve, several indexes, as the area under the curve (AUC) or the index of Youden, are considered as summaries of the discrimination capability of the classier. The AUC is the most commonly used one, taking values between 0.5 (no discrimination power) and 1 (perfect discrimination power). In many situations, however, the classication rule based on the values of Y that minimize the overall misclassication error, is not necessarily the criterio used in ROC analysis. Moreover, in this context it is known that the best classier based on Y with a threshold as classication decision is that based on the conditional probability of one of the states (e.g. S1) given the values of Y [3, 4]. Therefore, the best classier can be expressed as: = f (Y )P [S 1|Y ] (0, 1) Y (4)

In practice, however, the function f () of 4 is not known, and it is estimation would be required. In this work, we have evaluated the accuracy of several MODIS bands to classify situations of high and low risk of re. From our data set, we classied as high risk of re those observations with values of FMC under

482

L. Usero and M.X. Rodriguez-Alvarez

60% and as re low risk, those with values above this threshold. For this analysis, it was considered as classier, not the values of the MODIS bands, but their values transformed from equation 4. To estimate the function f () generalized additive models (GAM) [20] have been applied. Generalized additive models are exible non parametric regression models that enable much more accurate tting of the real data than the parametric linear models usually used. For each of the MODIS bands considered (from 1 to 7), the following logistic regression model were tted f (M B ) = P [RF = 1|M B ] = g 1 (h(M B )) (0, 1) (5)

where MB denotes de MODIS band variable, RF is a binary variable taking the value 1 for high risk of re and 0 for low risk, g is the logit function (known), and h is a smooth unknown function. The dataset was divided in two sets: 60% for tting the logistic regression model (2) (calibration set), and the remaining 40% to estimate the ROC curve (validation set). Once the model (2) was tted, we have used the estimated probabilities on the validation set to obtain the ROC curve and the AUC. Figures 2 and 3 show, for MODIS bands 1 and 2, (), along with the ROC curve and AUC. As can the estimated transformation f be seen from these gures, none of the MODIS bands have a good accuracy in distinguishing between situations of high and low re risk, showing AUC values close to 0.5 in all but the MODIS 2 band, with a value of 0.7. Therefore, we have conducted another analysis to evaluate if combining dierent MODIS bands to yield a composite classier, we can obtain better classication capability. The methodology in this case has been the same as for the analysis above. First, we have estimated the conditional probability of high re risk given the dierent MODIS bands values, by using exible multivariate logistic regression [20]:
f (M B 1, M B 2, ..., M B 7) = P [RF = 1|M B 1, ..., M B 7] = g 1 (
7

hi (M Bi )) (0, 1)
i=1

(6)

Fig. 2. Left: Estimated probability of high risk of re as a function of MODIS band 1 values, along with 95% condence interval. Right: Estimated ROC curve and AUC.

Reasoning on the Evaluation of Wildres Risk

483

where M Bi denotes the MODIS band i variable, RF is a binary variable taking the value 1 for high risk of re and 0 for low risk, g is the logit function (known), and hi are smooth unknown functions. Not all the MODIS bands were nally included in model (3) . The MODIS bands 3 and 7 were excluded, since they did not provide an improvement in the deviance explained. Once the model (3) was tted, we have used the probabilities estimated on the validation set to obtain the ROC curve. The result is shown in Figure 4. As can be seen, the accuracy of the combination of several MODIS bands in distinguishing between high and

Fig. 3. Left: Estimated probability of high risk of re as a function of MODIS band 2 values, along with 95% condence interval. Right: Estimated ROC curve and AUC.

Fig. 4. Estimated ROC curve and for the combination of several MODIS bands

484

L. Usero and M.X. Rodriguez-Alvarez

low risk of re is almost perfect, with a AUC value near 1. All statistical analysis were carried out in R [8]. Thin plate splines smoothers [22] have been used to estimate the function(s) h[i] in (2) and (3), with optimal smoothing parameters chosen automatically by use of the Un-Biased Risk Estimator criterion (UBRE) [21]. All models have been tted by use of the gam function of the mgcv package [8]. As regard to the estimation of the ROC curve and AUC, the R package ROCR has been used.

References
1. Bowyer, P., Danson, F.M.: Sensitivity of spectral reectance to variation in live fuel moisture content at leaf and canopy level. Remote Sens. Environ. 92, 297308 (2004) 2. Burgan, R.E., Rothermel, R.C.: BEHAVE: Fire Behavior Prediction and Fuel Modeling System. Fuel Subsystem. GTR INT-167, USDA Forest Service, Ogden, Utah (1984) 3. Camia, A., Bovio, G., Aguado, I., Stach, N.: Meteorological re danger indices and remote sensing. In: Chuvieco, E. (ed.) Remote Sensing of Large Wildres in the European Mediterranean Basin, pp. 3959. Springer, Berlin (1999) 4. Ceccato, P., Flasse, S., Tarantola, S., Jacquemoud, S., Gr egoire, J.M.: Detecting vegetation leaf water content using reectance in the optical domain. Remote Sens. Environ. 77, 2233 (2001) 5. Ceccato, P., Gobron, N., Flasse, S., Pinty, B., Tarantola, S.: Designing a spectral index to estimate vegetation water content from remote sensing data. Part 1, Theoretical approach. Remote Sens. Environ. 82, 188197 (2002) 6. Chuvieco, E., Aguado, I., Cocero, D., Ria no, D.: Design of an empirical index to estimate fuel moisture content from NOAA-AVHRR analysis in forest re danger studies. Int. J. Remote Sens. 24(8), 16211637 (2003a) 7. Combal, B., Baret, F., Weiss, M., Trubuil, A., Mace, D., Pragnere, A., Myneni, R., Knyazikhin, Y., Wang, L.: Retrieval of canopy biophysical variables from bidirectional reectance Using prior information to solve the ill-posed inverse problem. Remote Sens. Environ. 84, 115 (2002) 8. R Development Core Team: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2008) ISBN 3-90005107-0, http://www.R-project.org 9. Fourty, T., Baret, F.: Vegetation water and dry matter contents estimated from topof-the atmosphere reectance data: a simulation study. Remote Sens. Environ. 61, 3445 (1997) 10. Jacquemoud, S., Baret, F.: PROSPECT: a model of leaf optical properties spectra. Remote Sens. Environ. 34, 7591 (1990) 11. Knyazikhin, Y., et al.: MODIS leaf area index (LAI) and fraction of photosynthetically active radiation absorbed by vegetation (FPAR) product (MOD15). Algorithm Theoretical Basis Document (1999), http://eospso.gsfc.nasa.gov/atbd/modistables.html 12. Lawson, B.D., Hawkes, B.C.: Field evaluation of moisture content model for medium-sized logging slash. In: Proceedings of the 10th Conference on Fire and Forest Meteotology, Ottawa, Canada, pp. 247257 (1989) 13. Nu nez Olivera, E.: Ecolog a del jaral de Cistus ladanifer, Universidad de Extremadura (1988)

Reasoning on the Evaluation of Wildres Risk

485

14. Roberts, D.A., Peterson, S., Dennison, P.E., Sweeney, S., Rechel, J.: Evaluation of airborne visible/infrared imaging spectrometer (AVIRIS) and moderate resolution imaging spectrometer (MODIS) measures of live fuel moisture and fuel condition in a shrubland ecosystem in southern California. J. Geophy. Res. 111, G04S02 (2006), doi:10.1029/2005JG000113 15. Usero, R., Zarco, U.: Estimaci on del contenido de agua del combustible mediante el uso de redes neuronales articiales CEDI Granada (2005) 16. Ustin, S.L., Jacquemoud, S., Zarco-Tejada, P.J., Asner, G.: Remote Sensing of Environmental Processes: State of the Science and New Directions. In: Ustin, S.L. (ed.) Manual of Remote Sensing. Remote Sensing for Natural Resource Management and Environmental Monitoring, New York, vol. 4, pp. 679730 (2004) 17. Vermote, E.F., Vermeulen, A.: Atmospheric correction algorithm: spectral reectances (MOD09). NASA (1999) 18. Viegas, D.X., Pi nol, J., Viegas, M.T., Ogaya, R.: Estimating ive ne fuels moisture content using meteorologically-based indices. Int. J. Wildland Fire 10, 223240 (2001) 19. Wellburn, A.R.: The spectral determination of chlorophylls a and b, as well as total carotenoids, using various solvents with spectrophotometers of dierent resolution. J. Plant Physiol. 144, 307313 (1994) 20. Wood, S.: Generalized Additive Models. Chapman and Hall/CRC, Boca Raton (2006) 21. Wood, S.: Stable and ecient multiple smoothing parameter estimation for generalized additive models. Journal of the American Statististical Association 99, 549673 (2004) 22. Wood, S.: Thin plate regression splines. Journal of the Royal Statistical Society: Series B 65, 95114 (2003) 23. Yebra, M., Chuvieco, R.: Estimation of live fuel moisture content from MODIS images for re risk assessment agricultural and forest meteorology, vol. 148, pp. 523536. Elsevier, Amsterdam (2008) 24. Yebra, M., de Santis, A., Chuvieco, E.: Estimaci on del peligro de incendios a partir de teledetecci on y variables meteorol ogicas: variaci on temporal del contenido de humedad del combustible. Recursos Rurais 1, 919 (2005)