Proceedngs of the 009 IEEE Internatonal Conference on Systems, Man, and Cybernetcs San Antono, TX, USA - October 009 A Shadow Detecton Method for Remote Sensng Images Usng Affnty Propagaton Algorthm Huayng Xa 1, Xnyu Chen 1 1 Image Processng and Pattern Recognton Laboratory Bejng Normal Unversty Bejng, 100875, Chna xa_huayng@163.com, xychen@bnu.edu.cn Abstract Shadow detecton n hgh spatal resoluton remote sensng mage s very crtcal for locatng geographcal targets. In ths paper, we proposed a new shadow detecton method usng Affnty Propagaton (AP) algorthm n the Hue-Saturaton Intensty (HSI) color space. Because the pxel matrx s a largescale matrx, f we apply AP algorthm drectly on the raw pxel space, t wll be computaton ntensve to calculate the smlarty matrx. To solve ths problem, we propose to dvde the matrx nto several blocs and then applyng AP to detect shadows n H, S and I components respectvely. Then, three detected mages are fused to obtan a fnal shadow detecton result. Comparatve experments are performed for K-means and threshold segmentaton methods. The expermental results show that hgher detecton accuracy of the proposed approach s obtaned, and t can solve the problems of false dsmssals of K-means and threshold segmentaton method. Keywords Affnty Propagaton, Shadow Detecton, Remote Sensng Image, HSI Color Space, K-means I. INTRODUCTION In hgh spatal resoluton remote sensng mages, shadows are usually cast by elevated objects such as buldngs, brdges, and towers, whch s especally obvous n urban regons. Shadows can provde geometrc and semantc nformaton contaned n mages, ncludng cues about shapes and relatve postons of objects, as well as the characterstcs of surfaces and lght sources. However, shadows may also cause loss of feature nformaton, false color tone, and objects shape dstortons, whch affect the qualty of mages serously, and nfluence the effects of mage processng drectly, such as object recognton, change detecton and scene matchng [1] [] [3]. Hence, t s mportant to segment shadow regons and restore ther nformaton for mage nterpretaton. To process shadows or to use the nformaton provded by shadows, we must detect shadows accurately frst. It s not dffcult for humans to dentfy shadow n remote sensng mages, because shadow tself s one of the fundamental elements n vsual photo nterpretaton for obtanng nformaton about the shape, relatve poston, and surface characterstcs of objects n the scene. However, dentfyng shadows n dgtal mages by computers nvolves developng effectve algorthms n solvng many dffcult problems. Many technques for shadow detecton are developed for vdeo mages. However, n remote sensng mages feld, few technques are developed, whch manly based on model and property []. The approach based on model employs a pror nowledge of the llumnaton and the 3D geometry of the Png Guo 1, School of Computer Scence and Technology Bejng Insttute of Technology Bejng, 100081, Chna pguo@eee.org scene beng maged to calculate postons of shadows. However, ths method has rarely been used because the pror nowledge s not always avalable. The method based on property s generally used by analyss of lghtness, geometry structure and color character [3]. Recent exstng shadow detecton methods are commonly based on property, whch can be summarzed nto three types: threshold segmentaton method, color space ratos and photometrc color nvarants, and homomorphc flterng [1] [3]. Those methods get some achevements. However, none of them alone acheves common acceptablty to all nds of mages. The man reason s the complexty of mage shadow formaton mechansm. Shadow areas are beleved to be of lower ntensty than surroundng areas. For gray mages, shadows could be detected by approprate ntensty threshold. However, for color mages, there exst no-shadowed areas wth low ntensty (e.g. areas wth darer colors), threshold technques, such as hstogram threshold method, Otsu s method, maxmum entropy threshold, may msclassfy these areas as shadows [4] [5]. To overcome ths drawbac, we propose a novel method based on the Affnty Propagaton (AP) clusterng algorthm n the HSI color space for shadow detecton. Comparatve experments are performed for K-means [6] and hstogram threshold methods [7]. These methods are comparable because they are actually mage segmentaton method. Expermental results show that the proposed method s better than K-means and hstogram threshold n terms of accuracy, stablty and robustness. Ths paper s organzed as follows: In Secton II, the AP clusterng method and HSI color space are revewed. Secton III presents the developed shadow detecton approach. In Secton IV, expermental results are presented and analyzed. And fnally, some dscussons and conclusons are gven n Secton V. II. RELATED WORK A. Affnty Propagaton The AP clusterng technque was presented by Brendan J. Frey and Delbert Duec [8]. AP clusterng begns wth a collecton of real-valued smlartes between pars of data ponts, and teratvely exchanges the real-valued messages between data ponts so as to produce a hgh-qualty set of centers and correspondng clusters. There are several ndcators n AP. The smlarty s(, ) ndcates how well the data pont wth ndex s sutable to be the center for data pont. The 978-1-444-794-9/09/$5.00 009 IEEE 305
smlarty s the negatve squared error for mnmzng the squared error, thus for ponts x and x, s (, ) = x x. (1) When the value of s(, ) s nfnte (INF), messages wll not be exchanged between ponts x and x. In partcular, a real number s(, ) for each data pont s assgned a common value so that data ponts wth larger values of s(, ) are more lely to be chosen as centers after the message-passng procedure. These values are referred to as preferences, whch drectly decdes the number of clusters. The bgger number of clusters s, the larger values of the nput preferences are. There are two nds of message exchanged between data ponts, the frst one s the responsblty r(, ), whch s sent from data pont to pont, reflectng the accumulated evdence for how well-suted pont s to serve as the center for pont, consderng other potental centers for pont. In partcular, the self-responsblty reflects accumulated evdence that pont s a center, based on ts nput preference tempered by how ll-suted t s to be assgned to another center. The second one s the avalablty a(, ), whch s sent from canddate center pont to pont, reflectng the accumulated evdence for how approprate t would be for pont to choose pont as ts center, consderng the support from other pont that pont should be a center. In partcular, the self-avalablty a(, ) reflects accumulated evdence that pont s a center, based on the postve responsbltes sent to canddate center from other ponts. The whole clusterng procedure can be dvded nto the followng steps: 1) The avalabltes are ntalzed to zero: a(, ) = 0. The smlarty s computed and s(, ) s assgned a common value based on the dataset s value range. In our experment, we ntalze s(, ) to be the mnmum of nput smlartes. ) The responsbltes are computed usng the rule (), r (, ) s (, ) max { a (, ') + s (, ')}. () ': ' 3) The avalabltes are computed usng the rule (3), a (, ) mn{0, r (, ) + max{0, r ( ', )}}. (3) ': ' {, } 4) The self-avalablty a(, ) s updated dfferently: a (, ) max{0, r ( ', )}. (4) ': ' 5) If any of the followng stuatons occurs: after a fxed number of teratons, after changes n the messages fall below a threshold, or after the local decsons stay constant for some number of teratons, the clusterng procedure s termnated, else go to step. 6) Determne the center and ther ponts. For pont, the value of that maxmzes a(, ) + r(, ) ether dentfes pont as a center f =, or dentfes the data pont that s the center for pont. In order to avod numercal oscllatons n the procedure of exchangng message, a dampng factor λ s ntroduced, whch s between 0 and 1. Each message s set to λ tmes ts value from the prevous teraton plus 1-λ tmes ts prescrbed updated value. For example, f the value of a(, ) by the message-passng procedure s a(, ) cur, the old value of a(, ) s a(, ) old, then the new value of a(, ) new s (1-λ) a(, ) cur + λ a(, ) old. In our experments, we set dumpng parameter λ as 0.5, ntalze the s(, ) to be the mnmum of the nput smlarty matrx and fx the cluster number to be. Unle other methods for whch the number of clusters must be specfed, and whch are qute senstve to the ntal selecton of centers, AP taes as nput a real number s(, ) for each pont. Experment done by B. J. Frey and D. Duec has shown that AP fnds clusters wth much lower error. More detals and proof about the AP algorthm can be found n [8]. B. K-means The K-means [9] algorthm wors as follows. Frstly, t randomly selects of the objects, each of whch ntally represents a cluster mean. For each of the remanng objects, an object s assgned to the cluster to whch s the most smlar based on the dstance between the object and the cluster mean. Then t computes the new mean of each cluster. Ths procedure terates untl the crteron functon converges. The squarederror crteron s defned as expresson (5): E = p m. (5) = 1 p C Where E s the sum of square-error for all objects n the database, p s the pont n the space representng a gven object, and m s the mean of cluster C. The algorthm attempts to determne parttons that mnmze the squared-error functon. From these two clusterng algorthms, we can see that the classcal K-means clusterng result s related to ts ntal cluster means value whle that of AP s not. C. HSI Color Space The dffculty of usng the RGB color space s that t does not model the psychologcal understandng of color closely. The HSI model, whch follows the human vsual percepton closely, s a better color model. Ths color model separates the color components n terms of chromatc and achromatc nformaton [10]. The HSI model manpulates color mages wth the followng transformaton from the RGB model [9]: 1 1 1 3 3 3 I R 6 6 6 V 1 G = 6 6 6 V B 1 0 6 6, (6) 306
S ( V V ) 1 = +, (7) 1 V H = tan ( )( f V 0). (8) V 1 1 Note that hue and saturaton taen together are called chromatcty and the brghtness of a chromatc lght embodes the achromatc noton of ntensty [11]. It has been observed that shadowed regons n HSI space hold the followng propertes [14] [15]: 1) Lower ntensty: because the electro-magnetc radance from the Sun s obstructed. ) Hgher saturaton wth short blue-volet wavelength: due to the atmospherc Raylegh scatterng effect. 3) Increased hue values: because the change of ntensty of an area when shadowed and no-shadowed s postve proportonal to the wavelength. Once the mage s converted from RGB color space to HSI color space, the aforementoned propertes can be used to detect shadows. III. METHODOLOGY A. Clusterng n HSI Color Space In a sense, shadow detecton can be treated as a problem of mage segmentaton, whch s to separate an mage nto two regons. However, mage segmentaton can be vewed as a clusterng process [6] [1] [16]. The goal of clusterng s to group pxels together that exhbts some type of smlarty such as color, texture, or brghtness to form shadowed regon and no-shadowed regon. The clusterng algorthm s based upon pxel smlarty, and the locaton of boundares between regons comes very naturally to the human observer [6]. We now that shadows hold large hue values; however, n colored remote sensng mages, there are objects not shadowed wth large hue values, such as blue buldngs, rvers, grasslands, and so on, whch should be dstngushed out from shadows. For blush objects, whch always have hgh ntensty values whle shadows have low ones. Therefore, shadows could be descrbed as regons wth large hue values, low ntensty values. However, there stll exst objects not shadowed n colored remote sensng mages wth low ntensty values, for example, the presence of water and hgh reflectve regons, etc., we should remove them from shadows by usng other characterstcs of shadows, such as ncreased hue values and hgher saturaton. As a result, usng a sngle band of data usually does not gve us enough nformaton to dstngush between shadows and other dar objects. Therefore, we employ a set of color nvarant ndces n order to dscrmnate between shadow regons and other dar areas n remote sensng mages. Followng the analyss from above, we use the AP clusterng algorthm to detect shadows by fusng the features of ncreased hue, hgh saturaton, and lower lumnance of the shadow regon n the HSI color space. When usng AP, we should set smlarty matrx frst. In the hue space, we redefne the smlarty s(, ) for ponts H and H as expresson (9): H H H H π π H H H H > π sh (, H ) =. (9) In the saturaton and ntensty space, smlarty s(, ) for ponts x and x s defned as expresson (10): s (, ) = x x. (10) AP clusterng method consders all data ponts as potental exemplars smultaneously at frst. As we nown, raw pxel matrx for a gven mage usually s a large-scale matrx. If we apply AP algorthm drectly on t, t wll consume a large amount of system resources and spend a lot of tme when calculate the smlarty matrx. Therefore, before clusterng, we can dvde the mage matrx nto several blocs. We can dvde matrx by row, column or blocs accordng to the requrements. Thus, the number of the sample ponts may be reduced and the effcency of the algorthm wll be mproved a lot. B. Fuson Frstly, we calculate the hstogram of the H component. By applyng AP clusterng, the centers of two classes H c1, H c and the membershp functons of the two classes are obtaned [15]. Each pxel s grades of membershp for each class can be gven as expresson (11): μ ( H( xy, )) ( Hxy (, )) Hc1 μ Hc, (11) H(x, y) s the H component of the pxel at (x, y). These two values usually add up to 1. Secondly, we deal wth the S and I components usng the same method as the H component respectvely. Each pxel s grades of membershp for each class can be expressed as expresson (1) and (13): μ ( Sxy (, )) ( Sxy (, )) Sc1 μ Sc, (1) μ ( I( xy, )) ( Ixy (, )) Ic1 μ Ic. (13) Now we obtan sx grades of membershp for each pxel n the mage. Snce the pars add up to 1. Hc1 (H(x, y)), Sc1 (S(x, y)) and Ic1 (I(x, y)) are chosen to represent the overall color characterstcs of the pxel. Fusng these three grades of membershp to form a three-dmensonal feature vector as expresson (14): C = ( μ H ( (, )), ( (, )), ( (, ))) 1 H x y μ S x y μ c Sc Ic I x y. (14) Fnally, by applyng the AP algorthm on the fused feature space, we obtan the fnal detecton result for remote sensng mages. C. Algorthm Flow of the Proposed Method The proposed method ncludes fve major steps as follow: 1) Input a remote sensng mage F(x, y). ) Space converson from RGB color space nto HSI space, and F(x, y) s converted nto G(x, y). 307
3) The AP clusterng algorthm s utlzed to detect shadows n the H, S, and I components, respectvely. Frstly, we dvde the orgnal mages by column, then use AP to each column and obtan representatve ponts of every column, and we can get the fnal clusterng results by usng AP to these representatve ponts of every column. AP can group mage pxels nto any number of clusters. In our experments, we choose the cluster number to be two, whch are shadow regon and no-shadowed regon. 4) Three detected result mages are fused to obtan a fnal shadow detecton mage R 0 (x, y). Concrete procedure of fuson has been dscrped n subsecton B of Secton III. 5) Enhance outlne of the shadow area by applyng operatons n mathematcal morphology [0]. The purpose of ths step s to elmnate any outler and sland pxel wthn the boundary of shadow. The operaton steps as shown n expresson (15): R( xy, ) = fc( fo( R0( xy, ), g), g) (15) where f c denotes close operaton, f o denotes open operaton, g denotes structural elements. The shape sze of the structural elements s the ey to remove solated ponts to fll the empty area. For dfferent mages we need to utlze dfferent structural elements. In order to observe the effect of detecton and compensate the nformaton of shadow regon later, we adopt a bnary mage whch has the same sze wth the orgnal mage to dentfy the detecton results. The pxel value whch s zero on the bnary mage represents that the pxel belongs to shadow regon; and the pxel value whch s 55 on the bnary mage represents that the pxel belongs to no-shadowed regon. The algorthm flow s shown n Fg. 1. IV. EXPERIMENTAL RESULTS In ths experment, we also compared K-means and hstogram threshold method, whch were mplemented n MATLAB R007a under Mcrosoft Wndows XP envronment. In the proposed method, n order to use the ndex value of the whole remote sensng mage easly, we dvde the mage by column and apply AP for each column. Thus, we get the representatve ponts of every column, than apply AP to these representatve ponts. Through teratons, we get the fnal (a) Fgure. Shadow detecton results for mage one. (a) Orgnal Image, The proposed method, K-means, and Orgnal Image Space Converson RGB to HSI Clusterng of Hue Clusterng of Saturaton Clusterng of Intensty (a) Three Components Fuson Mathematcal Morphology Shadow Detected Images Fgure 1. Flowchart of the proposed method Fgure 3. Shadow detecton results for mage two. (a) Orgnal Image, The proposed method, K-means, and 308
(a) Fgure 4. Shadow detecton results for mage three. (a) Orgnal Image, The proposed method, K-means, and (a) Fgure 5. Shadow detecton results for mage four. (a) Orgnal Image, The proposed method, K-means, and detecton result. Expermental mages wth heavy shadows were downloaded from Google Earth. Four comparatve experment results for shadow detecton are shown n Fg., Fg. 3, Fg. 4 and Fg. 5 respectvely. Snce mages used n the experments are real remote sensng mages downloaded from Google Earth, we cannot evaluate the algorthm performance by objectve numercal values because lac of crteron. Here the only way to evaluate the algorthm performance s by comparng the vsual effects. In these fgures, the orgnal color remote sensng mages are shown n subfgure (a). Subfgure s the result of applyng the AP clusterng n the HSI color space; Subfgure shows the result usng K-means; and subfgure s the detecton result usng hstogram threshold method [5]. Fg. s a remote sensng mage wth heavy shadowed buldng, hgh reflectve regons, and dar green lawn. We observe that the proposed method solves the problems of false dsmssals, and t mproves the accuracy of shadow detecton. Fg. 3 demonstrates a qute dfferent example wth dar road and blue water whch also exhbt hgher hue and lower lumnance as the same as shadow regons. It s easy to see that the proposed method can obtan the shadow regons more precsely. Fg. 4 and Fg. 5 are typcal examples wth the presence of water area, dar green lawn and hgh reflectve regons, whch have the smlar hue wth shadow regon. We also observe that the proposed method can exclude water area from shadows as well. The comparson results show that the proposed method can dstngush dar objects from shadows, and the shape of the segmented shadows s preserved well, t has a hgher detecton precson than that of K-means and hstogram threshold method. For K-means, the clusterng result s related to ts ntal cluster mean value, whle that of AP s not. Threshold method s manly sutable for gray mages. Meanwhle, there s no unversal method to determne threshold for all mages. Besdes, repeated experments show that the proposed method has better stablty than K-means. As for the executon speed of the proposed method, t s not optmzed n ths wor. The computatonal complexty of AP s O(N ), whle those of K-means and threshold method both are O(N). To speed up the algorthm, one of the solutons s to tae the strategy of dvde and conquer. In the experments, when we dvde the mage feature matrx nto several blocs, the number of the sample ponts may be reduced n each bloc and the effcency of the algorthm wll be mproved a lot. In partcular, we dvde the orgnal mages by column, and the sze of the mages used n our experment s 18 18. In AP algorthm, we set dumpng parameter λ to be 0.5, the cluster number to be, and fxed the number of teratons s 3. Table I shows runtme of dfferent methods for mage F1, F, F3, and F4, tme unt s n Second. The tme shown n table refers to the average runtme after mplementng four tmes clusterng. Although the executon speed of AP s not optmal, t has hgher detecton precson than other clusterng algorthms, such as K-means or threshold method. The hgher detecton accuracy and better stablty proved that the proposed method can be well appled to shadow detecton for remote sensng mages when there s lac of pror nowledge. Besdes, detecton procedure does not requre manual nterventon. TABLE I. RUN TIME OF DIFFERENT ALGORITHMS AP K-means Threshold Image F1 10.08 45.1 1.19 Image F 16.8 48.37 1.38 Image F3 117.97 39.88 0.97 Image F4 119.61 4.46 1.03 309
V. CONCLUSIONS AND FUTURE WORKS In ths paper, we analyzed the shadow propertes n HSI color space, and presented an effcent method to detect shadows for remote sensng mages, whch usng AP clusterng algorthm and fusng the detected results n the H, S and I components. Test mages contan the areas whch are manly heavy shadowed buldngs. The proposed method to detect shadow of remote sensng mages as presented here manly concerned about the smlarty between colors. Further study s to fnd a better smlarty measure when usng AP clusterng algorthm. Two aspects of spatal nformaton and color components wll be taen nto consderaton. It s the fact that AP can automatcally choose the clusters number f we do not manually fx t. However, whether the two clusters are sutable to the applcaton of color mage segmentaton s an nterestng problem. For example, f there are varous regons n remote sensng mages wth heavy shadows, whch are smlar to shadowed regon, how to group them nto shadowed regon and no-shadowed regon automatcally wthout fxng the cluster number beforehand should be nvestgated. ACKNOWLEDGMENT The research wor descrbed n ths paper was fully supported by the grants from the Natonal Natural Scence Foundaton of Chna (Project No. 60675011, 9080010). Prof. Png Guo s the author to whom the correspondence should be addressed, hs e-mal address s pguo@eee.org. REFERENCES [1] E. Salvador, A. Cavallaro, and T. Ebrahm, Cast shadow segmentaton usng nvarant color features, Computer Vson and Image Understandng, vol. 95, no., pp. 38-59, 004. [] E. Salvador, A. Cavallaro and T. Ebrahm, Shadow dentfcaton and cassfcaton usng nvarant color models, Proceedngs of the IEEE Internatonal Conference on Acoustcs, Speech and Sgnal Processng, vol. 3, pp. 1545-1548, 001. [3] G. D. Fnlayson, S. D. Hordley, C. Lu, and M. S. Drew, On the removal of shadows from mages, IEEE Trans. on Pattern Analyss and Machne Intellgence, vol. 8, no. 1, pp. 59-68, 006. [4] J. L. Wang, The research of shadow detecton algorthms based on color aeral mages, M.S. thess, Department of Remote Sensng Informaton Engneerng, Wuhan Unversty, Wuhan, Chna, 005 (n Chnese). [5] F. C. Cheng, Y. K. Chen, and K. T. Lu, An effcent gray-level clusterng algorthm for mage segmentaton, Proceedngs of the IEEE Internatonal Conference on Informatcs n Control, Automaton and Robotcs, vol., no. 6, pp. 59-6, 009. [6] A. Wees and G. Erc Hague, Color segmentaton n the HSI color space usng the K-means algorthm, Proceedngs of SPIE Symposum on Electronc Imagng, vol. 306, pp. 143-154, 1997. [7] S. U. Lee, S. Y. Chung and R. H. Par, A comparatve performance study of several global thresholdng technques for segmentaton, Computer Vson, Graphcs and Image Processng, vol. 5, no., pp. 171-190, 1990. [8] B. J. Frey and D. Duec, Clusterng by Passng Messages between data ponts, Scence, vol. 315, no. 5814, pp. 97-976, 007. [9] T. Kanungo, D. M. Mount, and N. S. Netanyahu, An effcent K-means clusterng algorthm: analyss and mplementaton, IEEE Trans. on Pattern Analyss and Machne Intellqence, vol. 4, no. 7, pp. 881-89, 00. [10] Vctor J. D. Tsa, A comparatve study on shadow compensaton of color aeral mages n nvarant color models, IEEE Trans. on Geoscence and Remote Sensng, vol. 44, no. 6, pp.1661-1671, 006. [11] J. Huang, W. Xe and L. Tang, Detecton of and compensaton for shadows n colored urban aeral mages, Proceedngs of the 5th Word Congr. Intellgent Control and Automaton, vol. 4, pp. 3098-3100, 004. [1] P. Guo and Lyu MR, A study on space selecton for determnng mage segmentaton regon number, Proceedngs of Internatonal Conference on Artfcal Intelgence, vol. 1, no. 3, pp.117-113, 000. [13] J. Yao and Z. F. Zhang. Herarchcal shadow detecton for color aeral mages, Computer Vson and Image Understandng, vol. 10, no. 1, pp. 60-69, 006. [14] V. Arévalo, J. González, and G. Ambroso, Shadow detecton n colour hgh-resoluton satellte mages, Internatonal Journal of Remote Sensng, vol. 9, no. 7, pp. 1945-1963, 008. [15] J. Yang and Z. M. Zhao, A shadow removal method for hgh resoluton Remote Sensng mage, Journal of Wuhan Unversty (Informaton Scence). vol. 33, no. 1, pp.17-0, 008(n Chnese). [16] B. Rao, G. Zheng, T. M. Chen, J. Huang, and X. Shao, An effcent herarchcal method for mage shadow detecton, Proceedngs of nd Internatonal Worshop on Knowledge Dscovery Date Mnng, vol., no., pp. 6-67, 009. [17] W. Zhang, F. X. Zang, and X. K. Yang, Movng cast shadows detecton usng rato edge, IEEE Trans. on Multmeda, vol. 9, no. 6, pp. 10-114, 007. [18] P. Guo and H. Q. Lu, A study on Bayesan Probablstc mage automatc segmentaton, Acta Optca Snca, vol., no. 1, pp. 1479-1483, 00 (n Chnese). [19] K. L. Chung, Y. R. Ln, and Y. H. Huang, Effcent shadow detecton of color aeral mages based on successve thresholdng scheme, IEEE Trans. On Geoscence and Remote Sensng, vol. 47, no., pp. 671-68, 009. [0] S. Nomura, K. Yamanaa, T. Shose, H. Kawaam, and O. Kata, Morphologcal preprocessng method to thresholdng degraded word mage, Pattern Recognton Letters, vol. 30, no. 8, pp. 79-744, 009. [1] J. H. Guo, P. Gong, and L. Lang, Removng shadows from Google Earth satellte mages, Proceedngs of SPIE, vol. 675, no., pp. 1-1, 007. 310