IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 6, JUNE

Transcription

1 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 6, JUNE Adaptive Shadow Detetion Using a Blakbody Radiator Model Aliaksei Makarau, Rudolf Rihter, Rupert Müller, and Peter Reinartz Abstrat The appliation potential of remotely sensed optial imagery is boosted through the inrease in spatial resolution, and new analysis, interpretation, lassifiation, and hange detetion methods are developed. Together with all the advantages, shadows are more present in suh images, partiularly in urban areas. This may lead to errors during data proessing. The task of automati shadow detetion is still a urrent researh topi. Sine image aquisition is influened by many fators suh as sensor type, sun elevation and aquisition time, geographial oordinates of the sene, onditions and ontents of the atmosphere, et., the aquired imagery has highly varying intensity and spetral harateristis. The variane of these harateristis often leads to errors, using standard shadow detetion methods. Moreover, for some senes, these methods are inappliable. In this paper, we present an alternative robust method for shadow detetion. The method is based on the physial properties of a blakbody radiator. Instead of stati methods, this method adaptively alulates the parameters for a partiular sene and allows one to work with many different sensors and images obtained with different illumination onditions. Experimental assessment illustrates signifiant improvement for shadow detetion on typial multispetral sensors in omparison to other shadow detetion methods. Examples, as well as quantitative assessment of the results, are presented for Landsat-7 Enhaned Themati Mapper Plus, IKONOS, World- View-2, and the German Aerospae Center (DLR) 3K Camera airborne system. Index Terms Blakbody radiator, multispetral image, Plank equation, shadow detetion. I. INTRODUCTION SHADOWING is one of the main and inevitable aquisition artifats in high-resolution optial data. The quality of data proessing may be signifiantly degraded by the appearane of shadows, partiularly in urban areas. Sine images are obtained in different areas of the Earth and with different onditions of the atmosphere, the intensity and the spetral harateristis of the imagery have high variations, and the task of an automati shadow detetion beomes very omplex. Prevention of errors aused by this kind of artifats is still a urrent topi and disussed widely in remote sensing literature. Manusript reeived August 23, 2010; revised Otober 27, 2010; aepted November 16, Date of publiation January 12, 2011; date of urrent version May 20, This work was supported by the German Aerospae Center and German Aademi Exhange Servie (DLR-DAAD) postdotoral fellowship under Award A/09/ The authors are with the German Aerospae Center (DLR), Wessling, Germany ( aliaksei.makarau@dlr.de; rudolf.rihter@dlr.de; rupert.mueller@dlr.de; Peter.Reinartz@dlr.de). Digital Objet Identifier /TGRS The applied methods on shadow detetion may be divided into several broad groups of methods based on the following: 1) physial properties of light propagation/geometry [1]; 2) olor-invariant models (olor spae transformation) [2] [4]; and 3) omputational and statistial models [5] [9]. Sine the methods based on the physis of light propagation and the sun objet sensor geometry are designed for speifi appliations, they are diffiult to apply to real senes [10]. Therefore, the methods based on the properties of shadowed areas are preferred. Tappen et al. [7] proposed a method that uses multiple ues to reover shading from a single image. A lassifier is trained to reognize gray-sale patterns on an image derivative. Eah image derivative is lassified as being aused by shading or a hange in the surfae s refletane. The lassifier gathers loal evidene about the surfae form and olor, whih is then propagated using the generalized belief propagation algorithm. In the paper of Wu and Tang [9], the shadow extration problem is analyzed and formulated using a Bayesian framework. The method requires user s input, and the supplied hints are employed to effetively impose useful onstraints for solving the diffiult and ill-posed shadow extration problem from one image. The hierarhial algorithm proposed by Yao and Zhang [6] onsists of two levels of proessing: the pixel level lassifiation (ahieved through modeling an image as a reliable graph) and on maximization of the graph reliability using the expetation maximization algorithm. The region verifiation is ahieved through minimizing the Bayesian error by further exploiting the domain knowledge. Tsai [2] assesses the transformation of red, green, and blue (RGB) olor image into different invariant olor spaes to deouple hroma and luma omponents. Otsu method is used to segment shadow. Chung et al. [3] presented a modifiation of this method with loal shadow thresholding. Transformation into different olor spaes an modify olor invariane properties; therefore, suh approahes an fail on omplex senes. Salvador et al. [4] proposed another method of ast shadow detetion for still and moving images. This method exploits the spetral and geometrial properties of the shadowing proess. A hypothesis is applied on the fat that ast shadows darken the surfae whih they are ast upon, and olor invariane with geometri properties of shadows is used to verify deteted regions. The information integration stage onfirms or rejets the initial hypothesis. Polidorio et al. [11] proposed a robust tehnique by thresholding the differene image of the saturation and the intensity omponent in a normalized hue, saturation, and intensity olor /$ IEEE

2 2050 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 6, JUNE 2011 spae. Considering the atmospheri Rayleigh sattering effet, this tehnique allows one to segment shadowed areas in olor images obtained by airborne and orbital sensors. Tian et al. proposed a triolor attenuation model for shadow detetion [10]. The parameters of the model are fixed by using the spetral power distributions of daylight and skylight, whih are prealulated and fixed. Based on the model, a multistep shadow detetion algorithm is proposed to extrat shadows. The authors report that the weakness of this method is that it will fail on deteting shadows in sunrise and sunset (the orrelated olor temperatures during sunrise and sunset are very different from the orrelated olor temperatures used in the paper [10]). Sine this method has stati parameters, it an fail on remotely sensed data obtained at different daytimes with varying latitude/longitude (Lat/Lon) oordinates and atmospheri onditions. In this paper, we propose an alternative method for shadow detetion using a blakbody radiator model. This approah is fully motivated by the physial proess of shadow formation. Sine all the parameters of the method are alulated diretly from the input data, this method is adaptive and performs on all remotely sensed optial data (medium-resolution, highresolution, and very high resolution airborne and spaeborne sensors). The numerial assessment using ground truth data with several types of shadow situations illustrates the high performane of the approah. This paper is organized as follows: In Setion II, the physial model of shadow formation is presented, together with an appliation of the blakbody radiator model for shadow detetion. An automati parameter set alulation by solution of the equations is derived in Setion III. A step-by-step algorithm is presented in Setion IV. Experiments on different data, numerial evaluation using ground truth, and omparison with other methods are shown in Setion V, and the assessment of the results and disussion are presented in Setion VI. II. MODEL OF SHADOW FORMATION Illumination of an outdoor area is haraterized by two main light omponents: diret sunlight and the atmospheri sattered light (diffuse skylight). The diffusion of sunlight in the atmosphere is aused by Rayleigh and aerosol satterings. Obsuring objets from the diret sunlight auses the appearane of shadows, and the objets in this area are illuminated by the sattered light. We take the ommon assumption that the nonshadowed parts of the sene are illuminated by diret sunlight, but shadowed regions are illuminated only by the sattered light [12] (Fig. 1). Objets of the sene an have varying refletion harateristis and an be illuminated by different types of illumination (diret sunlight and/or sattered light). Therefore, besides the different refletion of objets, the reorded intensities in the image may have a very high variation. Instead of the reorded image intensities to be used for shadow detetion, the properties of the diret sunlight and the sattered-light illumination soures are expeted to provide a more stable and robust way for shadow detetion. Calulation of the illumination soure properties instead of using image intensities allows the separa- Fig. 1. Shadowed area formation. The area under sunlight illumination is haraterized by daylight sun illumination spetra, while the shadowed area is haraterized primarily by the spetra of the sattered sunlight. tion of two areas in the sene: the area illuminated by diret sunlight and the shadowed area (i.e., illuminated by sattered light). A. Image Aquisition We assume a Lambertian surfae refletion model for the image-forming proess. The intensity values reorded by a digital amera (three spetral hannels) an be desribed as p k = τ [T (λ)s(λ)e(λ)/π + L p (λ)] Q k (λ) dλ (1) ω where p k is the reorded intensity (digital number) in a olor hannel k = R, G, B (RGB hannel); τ is the gain fator that is dependent on the amera aperture, eletronis, and the integration time; T (λ) is the total ground-to-sensor transmittane; S(λ) is the surfae spetral refletane; E(λ) is the spetral irradiane at ground level; Q k (λ) is the sensitivity of the olor sensor; and L p (λ) is the atmospheri path radiane. For atmospherially orreted data, p k is already evaluated at the ground p atm k = τ ω [S(λ)E(λ)Q k (λ)/π] dλ. (2) Then, several assumptions and approximations are taken. A narrow amera model is assumed (narrow bandwidths) with enter wavelength λ k for eah hannel [13], [14]. Under the assumption of alibrated amera, S beomes onstant for eah hannel [13]. We an reasonably approximate (2) into I k = τs k E k, (S k = S(λ k ),E k = E(λ k )) (3) where I k is the intensity in the image (approximated p atm k ) and E k inludes the fator π for brevity. The intensities in the aquired image an have a very high variation. To represent olor in an invariant way (instead of olor triplets), many researhers [13], [14] proposed to use olor hromatiities. Color

3 MAKARAU et al.: ADAPTIVE SHADOW DETECTION USING A BLACKBODY RADIATOR MODEL 2051 hromatiities are widely used in the literature on olor siene, olor onstany, and shadow detetion and ompensation. The formation of reorded olor hromatiity i k is represented by multipliation of surfae and illumination hromatiities s k and e k i k = s k e k, (k = r, g). (4) Moreover, olor hromatiities in an image may be obtained by the ratios of the R, G, orb values to the sum of the R, G, and B values [15] or by the ratios of the R and G values to the B value [13]. We use hromatiities as a ratio of the R and G values to the B value i r = I R /I B i g = I G /I B. (5) The dependene of a material hromatiity on the reorded intensity an be shown by substituting (3) into (5) (note that τ is aneled out) s r = S R /S B = I RE B E R I B s g = S G /S B = I GE B E G I B. (6) B. Illuminant Approximation by the Blakbody Radiator Model The next assumption is that illumination spetra an be approximated by the model of a blakbody radiator [13]. Blakbody radiator model is found very useful for diret-sunlight spetra modeling in the appliations on olor onstany [13] [15]. Plank s formula for a blakbody emitting spetral radiane is defined in the following way: M(λ, T )= 1 λ 5 [exp( 2 /T λ) ] 1 (7) where M(λ, T ) is the spetral power of the blakbody radiation, 1 and 2 are onstants ( 1 = W m 2, and 2 = mk), λ is the wavelength m, and T is the temperature in Kelvin [14]. The blakbody radiator model an be used to represent the illuminant s hromatiity e r (T )= M(λ R,T) M(λ B,T) e g (T )= M(λ G,T) (8) M(λ B,T) or using (7) e r (T )= e g (T )= λ 5 B λ 5 R λ 5 B λ 5 G exp 2 Tλ B exp 2 Tλ R exp 2 Tλ B. (9) exp 2 Tλ G Fig. 2. Uniform material is illuminated by different soures of downwelling light: Diret sunlight and sattered light. The onstant hromatiity of the material allows one to reover the hromatiities of the two illuminants and to alulate the temperatures of blakbody radiators. The harateristis of diret sunlight and sattered skylight are dependent on the temperature of the blakbody radiator (the temperatures for diret sunlight and sattered skylight). Let us denote the temperature for diret sunlight as T light and the temperature for the sattered light (shadowed area) as T shadow. It should be noted that T shadow >T light, sine the spetra of sattered light are more bluish than the spetra of diret sunlight. Reovery of the temperatures allows one to identify the regions under sunlight illumination and sattered light illumination (shadowed region) independently of the sene. III. SHADOW DETECTION USING THE BLACKBODY RADIATOR MODEL To alulate the temperatures T shadow and T light, we an use the fat that the same objet in a sene an be under diret sunlight and an be shadowed. This means that the same material of the objet is illuminated by the two different illuminants (Fig. 2). To find the temperatures, we an use the fat that the image hromatiities divided by the illuminant s hromatiity are idential to the surfae hromatiity [14] (the surfae hromatiity is the same under both diret-sunlight and sattered-light illuminants). A. Illuminant Temperature Calulation Using (4), (5), and (9), a set of equations an be established, and the temperatures of the blakbody illuminants an be alulated i r,shadow e r,shadow i r,light e r,light =0 (10) where i r, and i g, are alulated using (5) or i g,shadow e g,shadow i g,light e g,light =0 (11) where the indies shadow and light denote the hromatiities of the shadowed and illuminated areas of the material. The assumption for (10) and (11) is that the hromatiity of the

4 2052 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 6, JUNE 2011 material is onstant. Components e r,shadow, e r,light, e g,shadow, and e g,light are alulated by e r,shadow = λ5 exp 2 B T shadow λ B λ 5 R exp 2 T shadow λ R e r,light = λ5 exp 2 B T light λ B λ 5 (12) R exp 2 T light λ R e g,shadow = λ5 exp 2 B T shadow λ B λ 5 G exp 2 T shadow λ G e g,light = λ5 exp 2 B T light λ B λ 5. (13) G exp 2 T light λ G Sine the hromatiities of the diret-sunlight and satteredlight illuminants are different, it is possible to perform a detetion of shadow areas (the detetion is dependent on the temperatures of blakbody radiators T light and T shadow ) or (i r /e r,shadow i r /e r,light ) < thresh r (14) (i g /e g,shadow i g /e g,light ) < thresh g (15) where thresh r and thresh g are the threshold parameters for shadow segmentation. Only one equation is neessary for the shadowed-region detetion. The detetion of the shadowed regions an be made using the e r hromatiity (12) or using e g hromatiity (13). The threshold value thresh r (or thresh g ) an be alulated automatially by the Otsu method [16], from a funtional dependene on two variables T light and T shadow,or manually. The nonparametri and unsupervised Otsu method allows automati threshold seletion for image segmentation. An optimal threshold is alulated aording to the shape of image histogram to separate the resultant lasses in gray levels with the between-lass variane maximization and the intralass variane minimization. Manual parameter seletion an be used in an offline method to reah the best possible quality of shadow detetion. B. Equation Solution Sine the funtions of (10) and (11) desribe an exponential dependene, the diret minimization of the squared sum is diffiult to perform [14]. The authors in [14] propose to use braketing for the solution. In this paper, numerial Brent s method was used (IDL funtion zbrent.pro with modifiations) [17]. Brent s method ombines root braketing, bisetion, and inverse quadrati interpolation to onverge from the neighborhood of a zero rossing. Using this method, onvergene is reahed as long as the funtion is evaluated within the interval known to ontain a root. The initial values are set to onstrain the ranges to the expeted solution aording to the prior knowledge of the temperature range for a partiular illuminant. The approximate Fig. 3. Example of onstant material (asphalt) partly shadowed and illuminated by diret sunlight. Regions under different illuminants (sunlight and sattered light) are outlined by retangles. temperature for the sattered light (blakbody radiator model) is in the range K, and that for diret sunlight is from 5500 to 6500 K [18]. For example, the temperature of sunrise/sunset illumination is in the range K, while the temperature of lear blue sky is between and K. The variation of the temperatures is possible due to varying Sun elevation, geographial oordinates, et. We set extended ranges for the temperatures to inlude a wider range of possible values: T light is in the range [5500, 7000], and T shadow is in the range [7000, 8500]. To alulate image hromatiities i r,shadow and i r,light (or i g,shadow and i g,light ), the R, G, and B pixel triplet values in two regions (illuminated and shadowed) of the same material have to be taken. An example of an objet of uniform material (asphalt) illuminated by diret sunlight and partly shadowed is shown in Fig. 3. The two regions are seleted near a shadow border, at the shadowed and illuminated sides. The shadow border an be found in different ways (depending on the appliation of the method): Automati approahes are desribed in [19] [21] or manually. To make the alulation of the parameter set more robust, several pairs of pixel triplets are randomly seleted from the two retangular regions where the temperatures are alulated and the mean values of the T light and T shadow temperatures are used for further alulations. IV. SHADOW DETECTION ALGORITHM The step-by-step shadow detetion algorithm is exeuted as follows. 1) Find the border of shadow irrespetive of the loation in the sene or shadow type. This an be done manually or automatially [19] [21]. 2) Loate shadowed and illuminated regions (see example in Fig. 3). To make sure that the shadowed and illuminated pixels are from the same objet or material, pixels should be taken near the shadow border (in the shadow and illuminated parts). 3) Calulate hromatiities i r,shadow and i r,light (or i g,shadow and i g,light ) for the shadowed and illuminated regions using (5).

5 MAKARAU et al.: ADAPTIVE SHADOW DETECTION USING A BLACKBODY RADIATOR MODEL 2053 Fig. 4. Shadow detetion on Landsat-7 ETM+ data (topographi shadows, mountain area, winter). (a) RGB band omposition. (b) Tsai method [2]. () Shorter method [24]. (d) Proposed method. Shadow masks are represented in white. Fig. 5. Shadow detetion on IKONOS data (loud shadows, urban area, 4-m spatial resolution). (a) RGB band omposition. (b) Tsai method [2]. () Shorter method [24]. (d) Proposed method. Shadow masks are represented in white. 4) Calulate the parameters of the method T light and T shadow using (10) and (11) aording to the temperature ranges for diret-sunlight and sattered-light illuminants. 5) Calulate the hromatiities i r (or i g ) for the image and the values of e r,shadow and e r,light (or e g,shadow and e g,light ) using (12) and (13), respetively.

6 2054 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 6, JUNE ) Calulate the mask for the shadowed regions using (14) or (15). The adaptive part of the algorithm onsists in the alulation of the temperatures T light and T shadow at step 4) (parameters of the method) for the partiular multispetral image. TABLE I LANDSAT-7 ETM+ BAND SPECTRAL AND SPATIAL RESOLUTIONS V. P ERFORMANCE EVALUATION This setion presents a desription of the used multispetral data, the metris, and the results of numerial assessment and omparison of the developed method. Medium, high, and very high spatial resolution remotely sensed optial imageries were used for the numerial assessment of the proposed method. Medium-resolution data sets inluded Landsat-7 Enhaned Themati Mapper Plus (ETM+) senes, high-resolution data sets omprised IKONOS senes, and the very high resolution data sets were omposed from the senes aquired by WorldView-2 and the German Aerospae Center (DLR) 3K Camera system [22]. The spetral sensitivity ranges for the hannels are presented in tables. The hannels employed for shadow detetion in the developed method are marked in itali. In order to illustrate the robustness of the shadow detetion method, different and ommonly enountered shadowing situations and shadow types were used: 1) shadow aused by loudiness (medium- and high-resolution data: Landsat-7 ETM+ and IKONOS); 2) shadow aused by buildings (very high resolution data: WorldView-2 and DLR 3K Camera system); and 3) topographi shadows (mountainous area and medium-resolution data: Landsat-7 ETM+). High-resolution and very high resolution data allow one to assess the method with inreased auray. To enable numerial evaluations of the developed method, test regions with different shadow types were ropped from the senes. Manual interpretation of shadowed regions was made in order to reate ground truth of shadow regions (true shadow masks). The true shadow masks were used in the evaluation of the preision of the shadow detetion methods. The performane of shadow detetion methods is usually assessed by several metris, among them the detetion rate (DR) and false-alarm rate (FAR). These metris are appropriate for an assessment of still shadow detetion methods and depend on the true positives (TPs; the number of shadow points that are lassified orretly), false negatives (FN; the shadow points lassified as bakground), and false positives (FP; the bakground points are deteted as shadows) [23] (TP rate, or reall), and DR = TP TP + FN (16) FP FAR = (17) TP + FP (orresponds to 1 p, where p is the so-alled preision in the lassifiation theory). The DR is expeted to inrease up to a value of one, while the FAR is expeted to derease to zero, respetively. In the following setions, we present several examples of widely used spaeborne and airborne imageries with ommon shadow situations, together with detetion results. The senes were aquired during different seasons and times, as well as in different geographial areas. For omparison, two other reent and ompetitive methods were seleted: the Vitor Tsai method [2] and the Niholas Shorter method [24]. Both methods show robust DRs for a broad range of appliations. The threshold parameters in both methods were seleted manually in suh a way that the best possible quality of shadow detetion an be reahed. The i r hromatiity and thresh r threshold parameter were employed for shadow detetion in the developed method. In omparison to the i g hromatiity, the i r -hromatiity-based shadow detetion allows one to use wider value range of the thresh r parameter and to alulate the thresh r value more preisely. The numerial assessment rates for all the detetion methods are presented in Table IV. A. Landsat-7 ETM+ This example ontains topographi shadows in a mountain region [Switzerland, Fig. 4(a)]. The aquisition parameters are as follows: Lat/Lon: N46.43 /E11.13, date: January 26, 2000, and time: UTC 09:45:00. The threshold values are 135 and 0.35 for the Tsai and Shorter methods, respetively. The alulated values for the proposed method parameter set are T shadow = 8228 K, T light = 5519 K(λ B = μm, λ G =0.561 μm, and λ R = μm), and thresh r =0.55 (Table I). B. IKONOS For IKONOS imagery, we present results on loud shadow detetion [urban area, Athens, Greee, Figs. 5(a) and 6(a)]. The aquisition parameters are as follows: Lat/Lon: N38.04 /E23.67, date: July 27, 2004, and time: UTC 08:46:44. The threshold values for the Tsai method are 45 [Fig. 5(b)] and 135 [Fig. 6(b)]. The threshold values for the Shorter method are 0.5 [Fig. 5()] and 0.37 [Fig. 6()]. The alulated values for the developed method parameter set are T shadow = 7765 K, T light = 5984 K(λ B = μm, λ G = μm, and λ R =0.665 μm), and thresh r =0.45 (Table II). C. WorldView-2 For WorldView-2 multispetral image, we present results on the detetion of the shadows from buildings (urban area,

7 MAKARAU et al.: ADAPTIVE SHADOW DETECTION USING A BLACKBODY RADIATOR MODEL 2055 TABLE II IKONOS BAND SPECTRAL AND SPATIAL RESOLUTIONS Fig. 7. Shadow detetion on WorldView-2 data (urban area, 2-m spatial resolution). (a) RGB band omposition. (b) Tsai method [2]. () Shorter method [24]. (d) Proposed method. Shadow masks are represented in white. TABLE III WORLDVIEW-2 BAND SPECTRAL AND SPATIAL RESOLUTIONS Fig. 6. Shadow detetion on IKONOS data (loud shadows, urban area, 4-m spatial resolution). (a) RGB band omposition. (b) Tsai method [2]. () Shorter method [24]. (d) Proposed method. Shadow masks are represented in white. Munih, Germany, Fig. 7). The aquisition parameters are as follows: Lat/Lon: N48.13 /E11.57, date: February 25, 2010, and time: UTC 10:22:06. The blue, green, and red bands were used for shadow detetion. The threshold values are 130 and 0.6 for the Tsai and Shorter methods, respetively. The alulated values for the proposed method parameter set are T shadow = 7962 K, T light = 6035 K(λ B = μm, λ G = μm, and λ R =0.665 μm), and thresh r =0.5 (Table III). D. DLR 3K Camera For the DLR 3K Camera system, we evaluated the detetion of shadows from buildings. Two types of test senes were seleted: shadows from low-rise buildings (residential area, sene 1) and shadows from high-rise buildings (sene 2). For sene 1 (low-rise buildings, residential area) Munih, Germany [Fig. 8(a)], the aquisition parameters of the DLR 3K Camera airplane system are as follows: Lat/Lon: N48.12 /E11.48, date: 30 April 2007, time: UTC 12:52:28, aperture: F9.11, exposure time: 1/2 048 seonds, altitude: m, spatial resolution: m. In Fig. 8, the threshold values are 135 and 0.37 for the Tsai and Shorter methods, respetively. In Fig. 9, the threshold values are 135 and 0.35 for the Tsai and Shorter methods, respetively.

8 2056 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 6, JUNE 2011 Fig. 8. Shadow detetion on DLR 3K Camera data (shadows from low-rise buildings, residential urban area). (a) RGB band omposition. (b) Tsai method [2]. () Shorter method [24]. (d) Proposed method. (e) Manually interpreted shadow mask. Shadow masks are represented in white. The alulated values for the proposed method parameter set are T shadow = 8260 K, T light = 5993 K, (λ B = μm, λ G =0.561 μm, and λ R = μm were taken as for Landsat-7 ETM+; see Setion VI), and thresh r =0.55. For sene 2 (high-rise buildings, residential area), Munih, Germany [Fig. 9(a)], the aquisition parameters and the temperatures are the same as those for sene 1. VI. RESULTS ASSESSMENT AND DISCUSSION The numerial evaluation of the method for senes with omplex objets provides high preision for shadowed-region detetion (see Table IV). Examples of IKONOS test images (Figs. 5 and 6) ontain roads in shadowed region. Despite the muh higher intensity of the pixels of the roads, the shadowed area was deteted with the best quality, whih is one of the examples where other olor transformation and segmentation methods may fail (e.g., Tsai method). Another example on ast shadow detetion is shown in Fig. 6. In this experiment, the illuminated and shadowed regions were taken at the border of the shadow from a loud, and the values for the method parameter set were alulated. As a result, all ast shadows were deteted, and the self-shadow (an objet is shadowed by itself) of the mountain was also deteted [see Fig. 6(d)]. This pratial example illustrates that the shadowed regions an be deteted irrespetive of the shadow type (i.e., all types of shadows are deteted in the sene), and the detetion depends on the alulated temperatures of diret sunlight and the sattered-light soures. The Tsai and Shorter methods did not provide aeptable results and did not detet self-shadows. Fig. 7 shows an RGB image omposed from the WorldView- 2 spetral hannels (visible range), together with the orresponding shadow masks. In this experiment, the illuminated region of a roof and the shadowed part (i.e., self-shadow) at the same roof were taken for the temperature alulations. Cast shadows and self-shadows were deteted in the sene [Fig. 7(d)]. Shorter s method results in an inreased value of the FARsore, aused by the wrong interpretation of buildings as shadows. For the DLR 3K Camera system, the entral wavelengths were taken as for Landsat-7 ETM+ (λ B = μm, λ G = μm, and λ R = μm). It was found that a small hange of the entral wavelengths does not degrade the auray of detetion. All the methods provided little degradation of the DR and FAR (aused by moving ars, Figs. 8 and 9). The DLR 3K Camera test sites illustrate the main drawbaks of the Tsai and Shorter methods. The Tsai method detets roofs of buildings as shadowed regions and is unable to deal with bright objets under shadow (Fig. 8, road surfae marking). Shorter s method mistakes vegetation (trees in Figs. 8 and 9) for shadows and is unable to properly detet shadow over bright objets (Fig. 8, onrete pavement). Table IV illustrates the advantage of the proposed method for all the examples having the highest DR and the lowest false aeptane rate FAR. In Fig. 7(a), the Shorter method provided

9 MAKARAU et al.: ADAPTIVE SHADOW DETECTION USING A BLACKBODY RADIATOR MODEL 2057 Fig. 9. Shadow detetion on DLR 3K Camera data (shadows from medium-rise buildings, residential urban area). (a) RGB band omposition. (b) Tsai method [2]. () Shorter method [24]. (d) Proposed method. Shadow masks are represented in white. TABLE IV SHADOW DETECTION ACCURACY NUMERICAL ASSESSMENT (BEST SCORES ARE MARKED IN BOLD) better detetion of shadows but together with the very high FAR value (FAR =0.48); therefore, the presented method is preferable. Another problem of shadow detetion in very high resolution data is moving objets. A high FAR sore is partly aused by a wrong assignment of moving objets as shadow. Many fators influening image aquisition parameters also influene the auray of shadow detetion. The main fators are as follows. 1) Daily and seasonal hanges of solar illumination. The hange of the illumination spetra leads to a hange of the refleted spetra and, therefore, to a hange of the hromatiity of the objets. The detetion of shadows aording to the aquired spetral or olor properties of the sene and omparison with prior expetations may provide errors in the detetion. 2) Geographi position, atmospheri omposition (aerosol optial thikness, water vapor olumn, et.), and insolation. Change of geographi position, as well as elevation, leads to hanges in diret-sunlight spetral distribution. The Sun illumination spetra are influened by the atmospheri omposition. Moreover, the sunlight passing through the atmosphere is attenuated.

10 2058 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 49, NO. 6, JUNE ) Content of the sene. For example, highly refleting objets in a shadowed area an be interpreted as nonshadow by the detetion methods based on invariant olor spaes. Therefore, many of the methods with straightforward employment of invariant olor spaes may fail beause of high variability of the image intensity in shadowed and illuminated regions. The properties of the illumination soures, as well as the properties of the illumination in shadow area, together with the adaptive alulation of the method s parameter set, an only allow robust and stable detetion of shadows. In omparison to the triolor attenuation model [10], our method adaptively alulates all the parameter set values from the input image and does not have restritions on the lighting onditions, i.e., there is no need to realulate parameters or extend to other lighting environments. This method is easy for deployment as a proedure for a system on remotely sensed image proessing and interpretation. The omplexity of the method is dependent primarily on the algorithm for shadow border searh and on the method for the temperature alulation. The seletion of the proper method for shadow border searh, as well as automati seletion of the initial parameters for the Brent s method, allows one to run this method with very low exeution time. Further appliation of this method has been performed for redution of errors in ity digital elevation model update (wrong boundaries of buildings an be deteted due to the building ast shadow and self-shadow), hange detetion (shadows from louds and buildings), and for lassifiation of optial data (speifi lasses among whih a shadow lass is outlined). VII. CONCLUSION An alternative method for shadow detetion for remotely sensed optial data has been developed. This method is based on the physial properties of illumination soure and employs the blakbody radiator model for the desription of the illumination proess. Robustness and high auray of shadow detetion are reahed by the adaptive nature of the method. The adaptiveness of the method onsists in the alulation of the parameter set values for a partiular input image. This allows one to extrat all types of shadows from a single image. High sores of statistial assessment rates were obtained on different remotely sensed imageries influened by the variations of illumination, aquisition time, Lat/Lon oordinates, the Sun elevation, atmospheri onditions, et. High DRs were obtained on the proessed data sets of multispetral visible and nearinfrared images. It should be noted that the appliation of the method is not limited to remotely sensed data. The method an easily be applied to other imagery from different soures and in the areas of image reognition (objet traking, robot navigation, et.). ACKNOWLEDGMENT We would like to thank European Spae Imaging (EUSI) for the olletion and provision of Digitalglobe WorldView-2 and IKONOS data over Munih ity. REFERENCES [1] D. C. Knill, P. Mamassian, and D. Kersten, Geometry of shadows, J. Opt. So. Amer. A, Opt. Image Si., vol. 14, no. 12, pp , De [2] V. J. D. Tsai, A omparative study on shadow ompensation of olor aerial images in invariant olor models, IEEE Trans. Geosi. Remote Sens., vol. 44, no. 6, pp , Jun [3] K. L. Chung, Y. R. Lin, and Y. H. Huang, Effiient shadow detetion of olor aerial images based on suessive thresholding sheme, IEEE Trans. Geosi. Remote Sens., vol. 47, no. 2, pp , Feb [4] E. Salvador, A. Cavallaro, and T. Ebrahimi, Cast shadow segmentation using invariant olor features, Comput. Vis. Image Understand., vol. 95, no. 2, pp , Aug [5] Y. Wang and S. Wang, Shadow detetion of urban aerial images based on partial differential equations, in Pro. ISPRS Congr., Comm. II, Jul. 3 11, 2008, vol. XXXVII, pp , Part B2. [6] J. Yao and Z. M. Zhang, Hierarhial shadow detetion for olor aerial images, Comput. Vis. Image Understand., vol. 102, no. 1, pp , Apr [7] M. F. Tappen, W. T. Freeman, and E. H. Adelson, Reovering intrinsi images from a single image, IEEE Trans. Pattern Anal. Mah. Intell., vol. 27, no. 9, pp , Sep [8] H. Jiang and M. S. Drew, Traking objets with shadows, in Pro. SPIE Image and Video Communiations and Proessing 2003, May7, 2003, pp [9] T.-P. Wu and C.-K. Tang, A Bayesian approah for shadow extration from a single image, in Pro. IEEE Int. Conf. Comput. Vis., Ot , 2005, vol. 1, pp [10] J. Tian, J. Sun, and Y. Tang, Triolor attenuation model for shadow detetion, IEEE Trans. Image Proess., vol. 18, no. 10, pp , Ot [11] A. M. Polidorio, F. C. Flores, N. N. Imai, A. M. G. Tommaselli, and C. Frano, Automati shadow segmentation in aerial olor images, in Pro. Brazilian Symp. Comput. Graph. Image Proess., Los Alamitos, CA, Ot , 2003, pp [12] S. Chakraborti, Verifiation of the Rayleigh sattering ross setion, Amer. J. Phys., vol. 75, no. 9, pp , Sep [13] J. A. Marhant and C. M. Onyango, Spetral invariane under daylight illumination hanges, J. Opt. So. Amer. A, Opt. Image Si., vol.19,no.5, pp , May [14] R. Kawakami, J. Takamatsu, and K. Ikeuhi, Color onstany from blakbody illumination, J. Opt. So. Amer. A, Opt. Image Si., vol. 24, no. 7, pp , Jul [15] G. D. Finlayson and S. D. Hordley, Color onstany at a pixel, J. Opt. So. Amer. A, Opt. Image Si., vol. 18, no. 2, pp , Feb [16] N. Otsu, A threshold seletion method from gray level histograms, IEEE Trans. Syst., Man, Cybern., vol. SMC-9, no. 1, pp , Jan [17] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerial Reipes in C: The Art of Sientifi Computing. New York: Cambridge Univ. Press, [18] I. Priest, The olorimetry and photometry of daylight and inandesent illuminants by the method of rotatory dispersion, J. Opt. So. Amer., vol. 7, no. 12, pp , De [19] G. D. Finlayson, S. D. Hordley, and M. S. Drew, Removing shadows from images, in Pro. 7th Eur. Conf. Comput. Vis., May 28 31, 2002, vol. 2353, pp [20] G. D. Finlayson, S. D. Hordley, and M. S. Drew, Removing shadows from images using Retinex, in Pro. IS&T/SID 10th Color Image Conf.: Color Si. Eng. Syst. Tehnol. (So. Inf. Display), Sottsdale, AZ, Nov. 2002, pp [21] T. Gevers and H. Stokman, Classifying olor edges in video into shadowgeometry, highlight, or material transitions, IEEE Trans. Multimedia, vol. 5, no. 2, pp , Jun [22] F. Kurz, D. Rosenbaum, U. Thomas, J. Leitloff, G. Palubinskas, K. Zeller, and P. Reinartz, Near real time airborne monitoring system for disaster and traffi appliations, in Pro. ISPRS Workshop, Hannover, Germany, Jun. 2 5, [23] A. Prati, I. Miki, M. Trivedi, and R. Cuhiara, Deteting moving shadows: Algorithms and evaluation, IEEE Trans. Pattern Anal. Mah. Intell., vol. 25, no. 7, pp , Jul [24] N. Shorter and T. Kasparis, Automati vegetation identifiation and building detetion from a single nadir aerial image, Remote Sens.,vol.1, no. 4, pp , Ot

11 MAKARAU et al.: ADAPTIVE SHADOW DETECTION USING A BLACKBODY RADIATOR MODEL 2059 Aliaksei Makarau reeived the Diploma (Dipl.Ing.) degree in omputer siene from the Belarusian State University of Informatis and Radioeletronis, Minsk, Belarus, in 2003 and the Ph.D. degree in tehnial sienes from the United Institute of Informatis Problems, Minsk, in His dissertation was on fast methods for multispetral image fusion and proessing. He is urrently a Postdotoral Fellow with the Department of Photogrammetry and Image Analysis, Remote Sensing Tehnology Institute (IMF), German Aerospae Center (DLR), Wessling, Germany. His researh interests inlude multimodal data fusion, pattern reognition, and automati lassifiation. Rudolf Rihter reeived the M.S. degree in physis from the Tehnial University of Munih, Munih, Germany, in 1973 and the Ph.D. (Dr. Ing.) degree in engineering from the Tehnial University of Dresden, Dresden, Germany, in He is urrently a Senior Sientist with the German Aerospae Center (DLR), Wessling, Germany, onduting onept development, modeling, and simulation of airborne/spaeborne hyperspetral instruments. It involves advaned tehniques and interation with sientists assoiated with theory and remote sensing experiments. He developed the ATCOR model, one of the standard odes for atmospheri and topographi orretions of multi/hyperspetral imagery used at universities and researh laboratories. His urrent work fouses on the design of fully automati proessing hains for the evaluation of remotely sensed optial data from the visible to the thermal spetral region. Rupert Müller reeived the Dipl.-Phys. degree from the Ludwig Maximilians University of Munih, Munih, Germany, in He is urrently a Team Leader of the Proessors and Traffi Monitoring Group, Remote Sensing Tehnology Institute, German Aerospae Center (DLR), Wessling, Germany, and is responsible for the Environmental Mapping and Analysis Program proessing, alibration, and validation part within the ground segment, as well as several European Spae Ageny projets like Prototype Proessor for ALOS Optial Data. His main researh interests inlude photogrammetri evaluation of spaeborne satellite data and digital image proessing. Peter Reinartz reeived the Diploma (Dipl.-Phys.) degree in theoretial physis from the University of Munih, Munih, Germany, in 1983 and the Ph.D. (Dr.-Ing) degree in ivil engineering from the University of Hannover, Hannover, Germany, in His dissertation was on the statistial optimization of lassifiation methods for multispetral image data. He is urrently the Head of the Department of Photogrammetry and Image Analysis, Remote Sensing Tehnology Institute (IMF), German Aerospae Center (DLR), Wessling, Germany, and holds a professorship on geoinformatis at the University of Osnabruek, Osnabruek, Germany. He has more then 20 years of experiene in image proessing and remote sensing and over 150 publiations in these fields. He is also engaged in using remote sensing data for disaster management and using high-frequeny time series of airborne image data for real time appliation in the ase of disasters, as well as for traffi monitoring. His main interests are in diret georeferening, stereo photogrammetry and data fusion, generation of digital elevation models, and interpretation of VHR image data from sensors like WorldView, GeoEye a.o.