A new image formation model for the segmentation of the snow cover in mountainous areas

Transcription

1 A new image formation model for the segmentation of the snow cover in mountainous areas Mihai Datcu +, Dragos Luca ++ and Klaus Seidel ++ + German Aerospace Research Establishment (DLR) German Remote Sensing Data Center (DFD) D Oberfaffenhofen Mihai.Datcu@dlr.de ++ Image Science Division, Communication Technology Laboratory, ETHZ, CH-8092 Zürich {dluca, seidel}@vision.ee.ethz.ch ABSTRACT The paper describes a new model for the correction of topographic effects in satellite images of snow covered rough terrain. The model simulates a synthetic image of the scene using a computer graphics approach which combines ray-tracing techniques with radiosity methods to render accurately both diffuse and specular reflections. Computation is structured on three levels: a macro level in which the image is described by the Digital Elevation Model, the position and physical properties of the light source and global atmospheric effects, a meso-scale in which the model simulates the integration effect of the imaging sensor and a micro-scale which is characterized by the reflectance of the snow cover (specular and diffuse reflections). After setting up the model, its parameters are tuned with a gradient search to fit real images acquired from the Landsat-TM sensor. The results show a good coincidence between synthetic and real images and prove the validity of the suggested image formation model. KEY WORDS: snow segmentation, modeling, ray-tracing/radiosity, diffuse/specular reflection, gradient search, Digital Elevation Model (DEM), Landsat-TM 1. INTRODUCTION The radiometry of remote sensing images in rough mountainous areas is severly affected by topographic artifacts. Shadow areas, diffuse and indirect solar illumination due to slopes covered by highly reflective snow as well as perturbant atmospheric effects cause ambiguities that appear in the interpretation of these images. The artifacts pose severe problems especially if the land-cover classification and/or image segmentation is to be performed automatically. On the other hand, manual interpretation of these images is slow and affected by human subjectivity. The new generation of satellites will even increase the problems connected with automated image interpretation by sending down to the Earth much larger amounts of data of higher geometric and spectral accuracy. Remote sensing applications in rough mountainous areas have increased in the last years and will continue to increase. The need for algorithms that master more accurately the correction of artifacts has become obvious and will certainly increase in the near future. Three approaches have mainly been used for the topographic correction of images in rough terrain areas. The first one uses multispectral image analysis: Image correction is then performed using band ratios or some statistical transformations based mainly on regression techniques [Naugle and Lashlee, 1992; Civco, 1989; Colby, 1991; Dozier, 1989]. This method is rather simple to implement but it contains some heuristic assumptions and has often a poor numeric performance. The second approach uses a radiative transfer code to obtain a deterministic description of the correction of topographic effects [Goel et al., 1991; Conese et al., 1993; Hill et al., 1995; Oren and Nayar, 1995; Richter, 1997]. While this approach avoids empirical techniques, it is more difficult to implement and encounters problems in the correct description of radiances and other illumination parameters of the scene. The approach we suggest is the third one and represents a model based Bayesian solution [Datcu and Holecz, 1993]. Our algorithm uses computer graphics techniques (ray-tracing and radiosity) to synthesize an image of the observed scene starting from the Digital Elevation Model (DEM), the position of the sun and the characteristics of the imaging sensor. But unlike other approaches, critical parameters related to the description of scene radiances are not postulated or deduced in advance EARSeL Advances in Remote Sensing

2 but estimated on-line in an optimization loop. This loop performs a search in the parameter space to find the set of parameters that minimizes the error between the synthesized and the observed remote sensing image. In this way we avoid problems related to the off-line estimation of radiance values; we make only basic assumptions on reflectances and irradiance and let the algorithm extract from the measured data the appropriate values. The parameter selection and optimization is only the first step in the proposed method, but this paper concentrates on it since we feel that it is a crucial point in the whole concept. Although we will not detail the rest of the method here, we want to point the main steps that follow to give a global overview of the algorithm: the set of parameters is estimated locally in an image window to support segmentation in multidimensional parameter space, each image window is represented by a point a clustering procedure is applied that looks for compact sets of points in the parameter space. The clusters characterize the different land-cover types (Fig. 1). It is easy to recognize in this approach the general paradigm of a classical unsupervised pattern recognition procedure. p2 cover type A * * * ** * * * * * * * * * * * * cover type B * * * * * * * * * * * The aim of our research is to provide an automatic method with improved accuracy for the segmentation of different land-cover types in remote sensing images from different sensors. Nevertheless, for the moment we will restrict the discussion to the segmentation of the snow cover on Landsat-TM images and will attempt to separate snow from non-snow areas. We believe that this example provides a good starting point for the development and refinement of the algorithm due to the large experience p1 Fig. 1: Clustering of parameters p1 and p2 in parameter space. The parameters are obtained from the optimization loop. acquired in the Remote Sensing community in the interpretation of Landsat-TM images and in modeling of snow reflectance. The paper is organized as follows. In section 2 we present some theoretical background and describe the structure of our model. Section 3 shows the results that are obtained for a Landsat-TM image from the region of Davos, Switzerland: we compare the mean square error as well as the local differences between the synthetic image and the real image for several models. Our model achieves the lowest error. This result is discussed in the conclusion section. 2. THE IMAGE FORMATION MODEL 2.1 A global description of the model The image synthesis is based on the knowledge of the DEM, on the position and physical properties of the light source and on the scattering process for different cover types. To derive an accurate model a multiresolution approach is used that includes three scales [Datcu and Holecz, 1993]: At macro-scale the scene is described by the the DEM and the position of the light source. These parameters are input to a computer graphics program that combines ray-tracing and radiosity techniques [Ashdown, 1994] and can render with accuracy both specular and diffuse reflections. The simulation is performed at a resolution higher than the sensor s spatial resolution. At meso-scale a spatial integration is performed that models the image formation process of the sensor. The resolution of the simulated image is reduced to match the observed data. Due to the nonlinear description of the light scattering process, the result of the operation does not equal the reflection value obtained without integration. Even in a simple Lambertian assumption (Fig. 2a), for each face of the low resolution DEM we have an intensity proportional to the cosine of the incidence angle: I low cos( nl, ) (1) while for the high resolution DEM the intensity is the sum of the cosines of all the incidence angles corresponding to the same area: I high cos( n k, l) k s (2) Obviously, I low is not equal to I high. Additionally, the interreflections between slopes that occur in snow- EARSeL Advances in Remote Sensing

3 covered terrain cannot be neglected and influence the result of the integration of the small facettes contribution (Fig. 2b). with m depending on the surface smoothness and h j the vector in the direction of the bisector line between the observer and the j-th light source (Fig. 3). low n s n k l l a. light source l j n high light source l j n h j view direction a. b. light source Fig. 3: a. Lambertian reflection, b. quasi-specular reflection b. Fig. 2: a. Reflected intensity from a low resolution DEM and from a high resolution DEM. Due to the nonlinear light scattering process, the values are not equal. b. Interreflections between slopes of the high resolution DEM. At micro-scale the facettes of the DEM surface are characterized by reflectance functions. We model the reflectance function as a superposition of Lambertian and specular behaviour. In order to catch the interreflections between slopes we suppose that each facette may be illuminated by several light sources. The Lambertian component will thus be: N I diffuse = k d C s I j cos( nl, j ) (3) j = 1 with k d the diffuse reflection coefficient, C s a coefficient related to the surface spectral characteristics, n the local surface normal, I j the intensity of the j-th source and l j the incidence vectors. Similarly, the specular component will be: N m I specular = k s C s I j cos( nh, j ) (4) j = 1 As described before the model contains some parameters that have to be set like e.g. the percentage of specular reflection and of lambertian reflection, the level of the ambient light, etc. In our approach, these parameters are not preset, but are determined on-line in an optimization loop with the gradient method. The mean square error between the synthetic and the observed image is used as an objective function of the minimization. In this way we avoid the common difficulties that appear in estimating the radiances for each image pixel and let the algorithm adapt the parameters to the available measurements. The block diagram of the model is shown in Fig The proposed model in the framework of Bayesian inference theory The set of fixed and variable parameters controling the image synthesis characterizes a model of the scene. Our task is to select one model (one set of parameters) from a multitude of possible models (many possible sets of parameters). The way this selection is done is irrelevant for this discussion: it will usually involve some optimization techniques but it could also be done by Monte-Carlo or any other methods. Important is the derivation of a goodness measure that depends on the information we have about the image formation process and also on some a priori information we have about the data and the parameters that describe it. This goodness measure allows the selection of an appropriate model. Most of the information we have is not complete, both about the data and the parameters. Even the information EARSeL Advances in Remote Sensing

4 fixed parameters DEM position of the sun snow albedo other parameters High resolution image synthesis Spatial integration MSE TM image error variable parameters amount of specular reflectance amount of ambient light Gradient search Fig. 4: The block diagram of the described model about the image formation process may not be complete due to errors and noise. It makes sense to represent this incomplete information as probabilities. If we denote the set of parameters and D the data, then the a priori information available on and D will be p( ) and p(d). The image formation process represents the generation of the observed data D conditioned by a controling set of parameters. The information available on the image formation process can thus be written as p(d ). For each set of parameters, there exists a classical formula to obtain a goodness measure p( D) of the set describing D. This formula takes into account all the (incomplete) information we have about, D and the process (D ) and is given by Bayes relation [Papoulis, 1984]: p( D) pd ( )p( ) = pd ( ) (5) The Bayesian inference method consists thus in interpreting probabilities as incomplete information rather than randomness and gives an alternative understanding of the information hidden in the measured data [MacKay, 1991; Freeman, 1993]. In our case, D represents the observed data and p(d) is the sensitivity of our observations to the geometric structure of the scene, i.e. to the DEM. p(d) is called the evidence. represents the set of parameters that describes the reflectance function of snow and p( ) is called the prior. The last term in eq. (5), p( D), represents our incertitude on the image formation process and is called the likelihood. The optimal reflectance parameters are selected by evaluating the function p(d ) (the posterior) for several sets. This reformulates the topographic correction problem in the general frame of illposed inverse problems. The Bayesian approach is of great importance in remote sensing where the information available is very incomplete and many ambiguities may occur. E.g. for high complexity scenes or for scenes in mountainous areas the application of classical image analysis methods is a weak approach. Significant radiometric uncertainties remain unsolved and as a consequence the signature analysis, structural interpretation and classification are affected by errors. Alternatively, in this paper the scene understanding approach is proposed. Scene understanding is the attempt to extract knowledge about the physical characterization, the structure and geometry of the threedimensional scene from the two-dimensional image. The major task of the scene understanding process is thus to find the scene which best explains the observed data [Datcu, 1996]. The solutions to this challenge are usually model based in the general frame of the Bayesian inference. 3. VALIDATION OF THE MODEL: RESULTS We have been testing our method on a Landsat-5 TM image from April 26th 1992 of the Davos region in Switzerland. The data is an Earthnet standard product, i.e. system corrected for both radiometry and geometry. It was resampled to both 25 meters and 10 meters using nearest neighbour interpolation to fit the available DEMs. The 10 meter data is used for the high resolution image synthesis, the 25 meter data for the computation of the error. Fig. 5a EARSeL Advances in Remote Sensing

5 a. b. Fig. 5: a. Geocorrected Landsat-TM image from April 26th of the Davos region, Switzerland. b. Test area (zoom of the selected area in a.) shows the geocorrected image, from which a small rectangular area which contains several shadows and interreflections was selected as test data (Fig. 5b). The test data has been selected in order to allow the tuning of model parameters for snow in an optimization loop. This will be the kernel for a subsequent snow segmentation which will be performed in a second step for the whole image. For the computer graphics simulation we used the RADI- ANCE 2.5 public domain software from the Lawrence Berkeley Laboratory, Berkeley, California [Ward, 1994; Larson, 1991]. This software combines ray-tracing and radiosity techniques and allows an accurate representation of both specular and diffuse reflections. It offers a large range of parameters to control the quality of the simulated image and several built-in functions for surface generation, sun position computation, ambient light control, filtering, enhancement, etc. We selected as variable parameters the amount of specular reflection and the ambient light in the image. These parameters were modified by the gradient algorithm to minimize the mean square error between the simulated and the observed image. The aim of the experiments was to validate the proposed model. In this respect we tried to answer the following three questions: what is the error obtained after successful minimization of the objective function? How close can our model simulate the real image? what is the difference between the simulation performed on a high resolution DEM followed by integration and the direct simulation on the low resolution DEM? Does the simulation on high resolution data improve the results? what is the performance of our model compared to other known models? The answer to the first question is given by the images in Fig. 6. Fig. 6a represents the simulated image after the minimization of the mean square error to the original image. The mean square error obtained is 233, i.e. in the mean the pixels are affected by an error of 15.2 grey levels. Fig. 6b represents the error image computed as the module of the difference between the simulated and the observed image. The error image was inverted for a better visualization, i.e. white corresponds to a zero error. Note that the simulated image renders in general with good accuracy the structure of the real image. Errors appear mostly in specular and shadow areas. We will see that these errors are much smaller than those obtained with other simulation methods.. The second question can be answered by performing the simulation directly on the low resolution DEM and comparing the results with those in Fig. 6. Just like in the high resolution case, the parameters of the simulation are optimized by minimizing the mean square error in a gradient loop. The results are shown in Fig. 7. The mean square error is 334, i.e. in the mean the pixels are affected by an error of 18.3 gray levels. This higher error is visible especially in the shadow area in the central-eastern part of the image. Although the simulation of a higher resolution image is more costly in terms of computer time and EARSeL Advances in Remote Sensing

6 Fig. 6: a. Simulated image after minimization of error. The simulation was performed on a high resolution DEM and the resolution was subsequently reduced to account for the image formation process of the sensor. b. Error image between the simulated image in figure a. and the original TM image. The error image is inverted, white represents zero error. Fig. 7: a. Simulated image after minimization of error. The simulation was performed on a low resolution DEM. b. Error image between the simulated image in figure a. and the original TM image. The error image is inverted, white represents zero error. Fig. 8: a. Error image between an image simulated with the Lambertian model and the original TM image. b. Error image between an image simulated with the Minnaert model (k=0.5) and the original TM image. The error images are inverted, white represents zero error. EARSeL Advances in Remote Sensing

7 power, it leads to a lower rendering error and is justified. For our last question we performed simulations of the same area with the simple Lambertian model and the Minnaert model Sandmeier and Itten, 1997 and compared the results. The mean square error of the Lambertian model is 1569, the Minnaert model achieves lower errors in the range depending on the value of the Minnaert constant. Both these mean square errors are substantially higher than that of 233 achieved with our algorithm. The error images for the Lambertian model and the Minnaert model with constant k = 0.5 are given in Fig. 8. Especially shadow areas show a high level of error, the models do not manage to render accurately the pixel intensity in these zones. An overview of results of our experiments is presented in Table 1. This overview shows a comparison of the mean square errors (MSE) obtained with several models for our area. The smallest error is obtained with our model using simulation on high resolution DEMs.. Table 1: Overview of results Model: Bayes High Res. Bayes Low Res. Lambertian Minnaert k = 0.8 Minnaert k = 0.5 Minnaert k = 0.2 MSE IMPORTANCE OF THE PROPOSED METHOD The method presented in this paper demonstrates the importance of an accurate modeling of physical phenomena for the precise estimation of snow parameters. The main difference between our method and other common algorithms used for the alleviation of topographic effects relies in the fact that we incorporate all the effects that control the image formation process, like multiple reflections, diffuse atmospheric effects, special reflectance functions, etc. Other methods use approximated or heuristic models that take into account only part of the reality. They achieve higher simulation errors, especially in areas where the second order effects are predominant: in areas with deep shadows where diffuse atmospheric light plays an important role, on facing slopes covered with snow where interreflections tend to amplify the sensed radiation or in the case of the special reflections shown by different snow types. To evaluate qualitatively the snow Fig. 9: Map of the selected test area (reproduced by permission of the Swiss Federal Office of Topography, 10. Nov. 1997) EARSeL Advances in Remote Sensing

8 simulation for the test area we present an extract of the map of the region (original scale 1:25 000). Note that the errors correspond either to man-made objects (houses, cable car buildings, etc.) or to steep rocks that are not covered by snow. As mentioned in the introduction, the simulation of the snow covered rough relief is only the first step in a snow segmentation algorithm of high accuracy that may be used for satellite but also for aerial images. In several applications this segmentation at pixel level is crucial. A typical example is target detection where the emphasis is on detecting objects in snow covered rough relief. Another example is the monitoring of interventions in disaster areas where single pixels give insight on the changes in the scene and may guide intervention groups: in this case the simulation may be used complementary to the visual inspection to highlight unexpected objects. It has equally been shown that for the derivation of the snow line on glaciers, the segmentation of the snow cover has to be made with high precision Seidel et al., A low error for snow segmentation is also important in areas where ground truth is not available or hard to obtain and the operator has to rely on the accuracy of the model since there is no practical verification possibility. In this case even for visual inspection a model may be needed: without solid knowledge of the observed area it is difficult for a human observer to say if a dark pixel cluster is due to a deep shadow or to snow-free ground. In our approach, the high accuracy simulation of snowcovered rough relief was used as a first step in an automatic snow segmentation. But the simulation itself may be a valuable tool for enviromental research as well: after having established a physical meaning for the parameters which control the graphic rendering process, the simulation allows the cost-effective evaluation of different environmental scenarios and their effect on the snow cover extend and type. The parameters of the rendered area are in this case tuned by the operator such as to fit the different climatic conditions under research. 5. CONCLUSIONS This paper describes a new model for the correction of topographic effects in satellite images of snow covered rough terrain. The topography influence alleviation is obtained as a parameter identification process and can be interpreted in the frame of Bayesian inference. The model simulates a syntethic image using computer graphic techniques that rely on a multiresolution approach. Its parameters are estimated in an optimization loop which minimizes the mean square error between the synthetic and the real image. The results of the performed experiments confirm the validity of the approach. The error obtained with the suggested method is much smaller than the error obtained with other usual models and is also much smaller than the error obtained if no multiresolution approach is used. This encourages us to apply this technique as a first step in the development of improved algorithms for the automatic segmentation of snow in high relief areas. ACKNOWLEDGEMENT We would like to acknowledge the Remote Sensing Laboratories of the University of Zürich for providing us the data on which this investigation is based. REFERENCES: Ashdown, I. (1994). Radiosity A Programmer s Perspective. John Wiley & Sons, Inc. Civco, D. L. (1989). Topographic normalization of Landsat Thematic Mapper digital imagery. Photogrammetric Engineering and Remote Sensing, 55(9): Colby, J. D. (1991). Topographic normalization in rugged terrain. Photogrametric Engineering and Remote Sensing, 57(5): Conese, C., Gilabert, M. A., Maselli, F., and Bottai, L. (1993). Topographic normalization of TM scenes through the use of an atmospheric correction method and digital terrain models. Photogrammetric Engineering and Remote Sensing, 59: Datcu, M. (1996). Scene understanding from SAR images. In Stein, T. I., editor, Remote Sensing for a Sustainable Future, IGARSS 96, volume 1, pages Datcu, M. and Holecz, F. (1993). Generation of synthetic images for radiometric topographic correction of optical imagery. In Chavez, P. S. and Schowengerdt, R. A., editors, SPIE Proceedings on Recent Advances in Sensors, Radiometric Calibration and Processing of Remotely Sensed Data, Orlando, USA, volume 1938, pages SPIE OE/Aerospace Sensing. Dozier, J. (1989). Spectral signature of alpine snow cover from the Landsat Thematic Mapper. Remote Sensing Environment, 28:9 22. Freeman, W. T. (1993). Exploiting the generic view assumption to estimate scene parameters. In Fourth International Conference on Computer Vision, May 11-14, Berlin, Germany, pages IEEE Computer Society Press. Goel, N. S., Rozehnal, I., and Thompson, R. L. (1991). A computer graphics based model for scattering from objects of arbitrary shapes in the optical region. Remote Sensing Environment, 36: EARSeL Advances in Remote Sensing

9 Hill, J., Mehl, W., and Radeloff, V. (1995). Improved forrest mapping by combining corrections of atmospheric and topographic effects in Landsat TM imagery. In Askne, J., editor, Sensors and Environmental Applications of Remote Sensing, 14th EARSeL Symposium 1994 in Göteborg, Sweden, pages A. A. Balkema Rotterdam/Brookfield. Larson, C. (1991). RADIANCE: User s Manual. Lawrence Berkeley Laboratory, Berkeley, California. MacKay, D. J. C. (1991). Bayesian Interpolation. In Smith, C. R., Erickson, G. J., and Neudorfer, P. O., editors, Maximum Entropy and Bayesian Methods, pages Kluwer Academic Publishers. Naugle, B. I. and Lashlee, J. D. (1992). Alleviating topographic influences on land-cover classifications for mobility and combat modeling. Photogrametric Engineering and Remote Sensing, 58(8): Oren, M. and Nayar, S. K. (1995). Generalization of the Lambertian model and implications for machine vision. International Journal of Computer Vision, 14: Papoulis, A. (1984). Probability, Random Variables, and Stochastic Processes. McGraw-Hill, 2nd edition. Richter, R. (1997). Correction of atmospheric and topographic effects for high spatial resolution satellite imagery. International Journal of Remote Sensing, 18(5): Sandmeier, S. and Itten, K. I. (1997). A physically-based model to correct atmospheric and illumination effects in optical satellite data of rugged terrain. IEEE TRans. on Geoscience and Remote Sensing, 35(3): Seidel, K., Ehrler, C., Martinec, J., and Turpin, O. (1997). Derivation of statistical snowline from high resolution snow cover mapping. In Wunderle, S., editor, EARSeL Workshop: Remote Sensing of Land Ice and Snow, April 1997, Freiburg, Germany, pages Ward, G. J. (1994). The RADIANCE lighting simulation and rendering system. Computer Graphics, pages SIGGRAPH 94, Orlando, Florida. /home/seidel/mac-server/texte/earsel/freiburg/ FREIBURG_dragos_final.frm EARSeL Advances in Remote Sensing