OBJECT DETECTION FROM HS/MS AND MULTI-PLATFORM REMOTE- SENSING IMAGERY BY THE INTEGRATION OF BIOLOGICALLY AND GEOMETRICALLY INSPIRED APPROACHES

Transcription

1 OBJECT DETECTION FROM HS/MS AND MULTI-PLATFORM REMOTE- SENSING IMAGERY BY THE INTEGRATION OF BIOLOGICALLY AND GEOMETRICALLY INSPIRED APPROACHES Bo Wu 1, Yuan Zhou 1, Ln Yan 1, Jangye Yuan 2, Ron L 1, and DeLang Wang 2 1 Mappng and GIS Laboratory, Department of Cvl and Envronmental Engneerng and Geodetc Scence 2 Department of Computer Scence and Engneerng and Center for Cogntve Scence The Oho State Unversty, Columbus, OH wu.573@osu.edu ABSTRACT Ths paper presents a system that ntegrates bologcally and geometrcally nspred approaches to detectng objects from hyperspectral and/or multspectral (HS/MS), multscale, multplatform magery. Frst, dmensonalty reducton methods are studed and used for hyperspectral dmensonalty reducton. Then, a bologcally nspred method, S- LEGION (Spatal - Locally Exctatory Globally Inhbtory Oscllator Network), s developed to perform object detecton on the multspectral and dmenson-reduced hyperspectral data, whch provdes rough object shapes. Thereafter, a geometrcally nspred method, GAC (geometrc actve contour), s employed for refnng object boundary detecton on the hgh-resoluton magery based upon the ntal object shapes provded by S-LEGION. A geospatal database s compled and used for expermental analyss that ncludes data from a selected test ste at Slver Lake n the Mojave Desert, Calforna. Multspectral (Landsat TM 4-5) and hyperspectral (EO-1) satellte magery, hgh-resoluton satellte magery (IKONOS), and descent mages and ground stereo mages are ncluded n ths database. Ths paper presents the frst year results of a two-year research project. INTRODUCTION Over the last decades, there has been a remarkable ncrease n the number of remote-sensng sensors onboard varous satellte-, arcraft-, and land vehcle-based platforms. Large volumes of panchromatc, multspectral, and hyperspectral data have been collected perodcally. Fuson of these mult-platform remote-sensng data along wth n stu observatons from multple sensors can help us to derve more nformaton than s possble from a sngle sensor alone. Examples nclude detecton of roads and buldngs, determnaton of the composton of ground vegetaton, and localzaton of mneral resources, as well as other applcaton areas. However, the most detaled nformaton, such as shape and spectral attrbutes, often cannot be derved precsely. Recent advances n bologcally nspred methods nvolve segmentng patterns, materals, and objects, among other capabltes. Terman and Wang (1995) proposed locally exctatory globally nhbtory oscllator networks (LEGION) as a computatonal framework for mage segmentaton and object recognton. It has been shown analytcally that LEGION networks can rapdly acheve both synchronzaton n a locally coupled oscllator group and desynchronzaton among dfferent oscllator groups. LEGION has been successfully appled to segmentng grayscale mages, medcal mages, and aeral mages (Wang and Terman, 1997; Wang, 2005). On the other hand, geometrcally nspred methods, such as level set theory and GAC (geometrc actve contour) models, have also been wdely used n mage segmentaton and object detecton (Osher and Sethan, 1988; Caselles et al., 1997; Nu, 2006). If the bologcally nspred object detecton methods can be combned wth advanced geometry-based object detecton technques, a varety of object detecton and recognton tasks n cvlan, mltary, and ntellgent applcatons can be sgnfcantly mproved and speeded up. Ths paper presents a system that ntegrates bologcally and geometrcally nspred approaches to detect objects from hyperspectral and/or multspectral (HS/MS), multscale, multplatform magery. Dmensonalty reducton methods are studed and used for hyperspectral dmensonalty reducton. A bologcally nspred method, S- LEGION (Spatal - LEGION), s developed to perform object detecton on the multspectral and dmenson-reduced hyperspectral data, whch provdes rough object shapes. Then, a geometrcally nspred method, GAC (geometrc actve contour), s employed for refnng object boundary detecton on the hgh-resoluton magery based on the ntal object shapes provded by S-LEGION. Ths research s funded by a NGA Unversty Research Intatves project. Ths paper presents the results of the frst year of the project, manly summarzng the archtecture of the ntegrated system for object detecton and hyperspectral dmensonalty reducton.

2 AN INTEGRATED SYSTEM FOR OBJECT DETECTION The archtecture and concept of the system ntegratng bologcally and geometrcally nspred object detecton methods are llustrated n Fgure 1. Frst, satellte and arborne HS/MS data wll be processed usng an S-LEGION algorthm for mage segmentaton n order to study regonal and contextual nformaton about the entre ste. Dmensonalty reducton methods wll be employed for hyperspectral dmensonalty reducton. Spectral compostons of HS/MS mages wll help to quckly dentfy canddate regons for detectng objects of nterest. Supported by hgh-resoluton satellte mages, multscale descent mages, and ground mages, a GAC model wll then be appled to each small regon of nterest detected by S-LEGION for mproved boundary extracton and shape reconstructon. The extracted nformaton wll serve as nput to a fnal object recognton method usng shape-based and spectral-based technques. Durng the entre process, a multplatform sensor modeler wll be appled to support precson multsensor and multplatform mage analyss. S-LEGION Search for undentfed objects and spectral nformaton from HS/MS magery (or dmensonal reduced magery) GAC Boundary extracton, refnement and shape reconstructon from hghresoluton magery Object Recognton Shape- and spectral-based object recognton Multplatform sensor modeler (Satellte, arborne, descent, ground) Modelng of satellte, arborne, descent and ground sensors for cross-platform object detecton Detected Objects Fgure 1. Integrated system for bologcally/geometrcally-nspred object detecton. Interested area GAC Landsat MS Image (30-m resoluton) EO1 HS Image (30-m resoluton) Dmensonalty Reducton Object Recognton S-LEGION IKONOS Image (1-m resoluton) Descent Image from a Helcopter (0.3-m resoluton) Fgure 2. Conceptualzaton of the ntegrated system for object detecton usng the mult-platform data collected at Slver Lake n the Mojave Desert, CA. In ths nvestgaton, we wll apply the proposed approach to data compled nto a geospatal database usng the data collected from feld tests at Slver Lake n the Mojave Desert. The geospatal database ncludes mult-scale mages collected from satellte, arborne, descent (helcopter), and n stu (land vehcle and feld robot) mages as

3 well as GPS control ponts. It has been collected and mantaned snce The descent mages represent a sequence of multscale mages that can be used to test the capablty of scale nvarant object recognton. The Slver Lake test ste conssts of desert terran wth scattered bushes, a dry lake, hghways, paved and unpaved roads, utlty lnes, a helcopter port, and small housng complexes. Fgure 2 llustrates the concept of the ntegrated system for road detecton usng the Slver Lake data. Ths geospatal database wll contrbute sgnfcantly to system development and valdaton. HYPERSPECTRAL DIMENSIONALITY REDUCTION Object detecton methods (such as LEGION and GAC mentoned above) have been successfully appled to process mages ncludng medcal mages, close-range mages, and arborne/satellte mages. However, problems wll arse from transplantng these algorthms to HS/MS remote-sensng magery. Compared wth the sngle-band mage, the new data form (especally the hyperspectral mage) has a much hgher number of dmensons n spectral space. Ths ncrease n dmensons results n a rapd ncrease n computatonal costs and a reducton n classfer performance. Ths s called the Hughes Phenomenon (Hughes, 1968). Moreover, there s nformaton redundancy n the hgh-dmensonal space, such as the hgh correlaton between those spectral bands close to each other and the nose from atmospherc absorpton. Ths nformaton redundancy wastes computaton tme and depresses accuracy n the mage processng. Therefore, dmensonalty reducton for hyperspectral magery s necessary before t can be used for the subsequent object detecton. Dmensonalty Reducton Methods Dmensonalty reducton maps a hgh-dmensonal space onto a space wth fewer dmensons, whle the data n the orgnal space can stll be fully represented. It reduces the mpact of the Hughes Phenomenon and also reduces redundant nformaton, thus rasng the effcency of the data processng. Many dmensonalty reducton methods have been presented n the past; Prncpal Component Analyss (Hotellng, 1933), Lnear Dscrmnant Analyss (Fsher, 1936), and Maxmum Nose Fracton (Green et al., 1988) have been effcently appled n compressng the HS/MS mages. These methods share the common employment of egenvalue-based lnear approxmaton to retreve the orgnal spectral space. However, from a spectral pont of vew, spectra characterstcs are treated as lnear features n these approaches, whle ther nonlnear features are gnored. However, n certan crcumstances, the nonlneartes of hyperspectral data can be the major propertes n spectral space (Bachmann et al., 2004). L et al. (2005) proved that the nonlnear spectral nverse model s more accurate than the lnear method. In recent years, Manfold Learnng technques have been ntroduced to model the nonlnear features (manfold) of hgh-dmensonal data and to project the manfold onto low-dmensonal space whereby the nonlnear propertes of the data could be well preserved durng the data compresson. Bascally, these technques could be dvded nto two groups: the frst one preservng the global propertes of the data and the second one focusng on the local propertes of the data (van der Maaten et al., 2007). The representatve methods for the frst group nclude ISOmaps (Tenenbaum et al., 2000), Kernel PCA (Scholkopf et al., 1998), and Dffuson Maps (Lafon and Lee, 2006). The second group ncludes Local Lnear Embeddng (Rowes, 2000; Han and Goodenough, 2005), Laplacan Egenmap (Belkn, 2003), and Local Tangent Space Algnment (Zhang et al., 2002). Compared wth the global propertes preserved n Manfold Learnng methods, local nonlnear technques for dmensonalty reducton are based on solely preservng the propertes of small neghborhoods around the data ponts of nterest. Ths satsfes the prncples of object detecton wthn a local neghborhood. Ths research manly examnes the followng three local nonlnear technques. Local Lnear Embeddng (LLE) s the frst local Manfold Learnng technque ntroduced for dmensonalty reducton (Rowes, 2000; Han and Goodenough, 2005). In LLE, the ponts neghborng the data ponts of nterest are found and saved n a neghborhood graph. The local geometry of the data ponts s estmated by the reconstructon weghts usng a cost functon. In ths functon, the reconstructon weghts are subject to the constrant that the dfference between the lnear combnaton of the weghted neghbor ponts and the center pont should be mnmzed. The data ponts wll be projected nto a new low-dmensonal space based on mnmzng the lnear combnaton of the pont and ts neghbor ponts n the new space, weghted by the reconstructon weghts. Laplacan Egenmap (LE) calculates the weght between data ponts and ther neghborhood usng the nearest neghborhood methods (Belkn, 2003). The nearest neghbor of the data pont wll contrbute the most. The weghts are used as edges connectng each pont wth ts neghbors n a graph. By usng spectral graph theory, the projecton

4 from a hgh-dmensonal space to a low-dmensonal space s defned as an Egen problem. A Laplacan matrx s derved from the connected weght edges of the graph. The data ponts n the low-dmensonal space are generated by the lnear combnaton of the largest egenvectors of the Laplacan matrx. The number of egenvector s the same as the number of the dmensons n the low-dmensonal space. Local Tangent Space Algnment (LTSA) uses the local tangent space of each data pont to descrbe the local propertes of the hgh-dmensonal space (Zhang et al., 2002). In LTSA, t s assumed that f there s local lnearty n the manfold surface, the data ponts n the hgh-dmensonal space and the correspondng low-dmensonal space could be mapped to the same local tangent space. In other words, LTSA smultaneously searches for the local tangent space of the hgh-dmensonal data and the low-dmensonal data representatons, and the mappng relatonshps between the low-dmensonal data ponts and the local tangent space of the hgh-dmensonal data. Expermental Results Ths paper compared the above-mentoned three Manfold Learnng methods (LLE, LE, and LTSA) usng hyperspectral data (Fgure 3) collected at Slver Lake. Ths hyperspectral data was selected from a 30-m resoluton EO1 Hyperon Hyperspectral Level 1GST product that was acqured n October The mage sze for the selected expermental area s 200 x 200 pxels (Fgure 3a). It has 242 bands (from nm to nm) n spectral space. Pre-processng of spectral and geometrc correcton was performed usng the methods proposed by Beck (2003) and Datt et al. (2003). After removal of atmospherc absorpton effects, 156 bands were left for later dmensonalty reducton experments. Usng the Maxmum Lkelhood Estmaton method (Levna and Bckel, 2004), the ntrnsc dmensons of the nput data were estmated as 15. In our experments, 40,000 data ponts (200 x 200) wth 156 dmensons are used as nput data for the dmensonalty reducton process usng the LLE, LE, and LTSA methods; the output s the reduced data wth 15 dmensons. The tme-consumpton rato for these three methods s 15(LLE):2(LE):10(LTSA), showng that the LE method s the most effcent. a) b) Fgure 3. Expermental data for dmensonalty reducton: a) a pseudocolor EO1 hyperspectral mage of 200 X 200 pxels, and b) manually dgtzed objects as ground truth (brown denotes hlls, whte lakebed, black asphalt roads, blue buldngs, and green vegetaton). To evaluate the results of these three dmensonalty reducton methods, fve categores of objects were manually dgtzed and labeled on the pseudocolor mage. These objects nclude hlls, lakebed, asphalt road, buldngs and vegetaton as llustrated n Fgure 3b. The coverages for these objects are 4781 pxels for the hlls (brown n Fgure 3b), 2353 pxels for the lakebed (whte), 950 pxels for asphalt roads (black), 964 pxels for buldngs (blue), and 39 pxels for vegetaton (green). These objects were used as ground truth n our experment. After dmensonalty reducton usng the LLE, LE, and LTSA methods, 15 dmensons data were obtaned for each of these methods. Then an unsupervsed classfer, the K-nearest neghborhood (KNN) method, was used to perform classfcaton for the dmenson-reduced data. Each dataset was separated nto eght clusters. Those clusters correspondng to the manually dgtzed ground-truth objects were assgned ther same color; the remanng clusters were all colored black and treated as background. These classfcaton results are llustrated n Fgure 4.

5 a) b) c) Fgure 4. Classfcaton results of the dmenson-reduced data from the a) LLE, b) LE, and c) LTSA methods. As shown n Fgure 4, we found that the lakebed could be separated n the classfcaton results from all three dmensonalty reducton methods. However, the asphalt roads and hlls are mxed up n all three cases. In a feld spectral survey performed at Slver Lake n October 2008, we found that these two types of objects are nseparable from the spectral pont of vew due to ther hgh correlaton of spectrum features (ths could be the consequence of usng the materals collected from the hlls to construct the roads). Therefore, we merged the hlls and asphalt roads n the followng analyss of the classfcaton results. To analyze the classfcaton results, we compared the numbers of pxels classfed n each of the clusters based on the three dfferent dmensonalty reducton methods wth the numbers of pxels from the correspondng manually dgtzed ground truth. The results are lsted n Table 1. It can be seen that the LTSA provdes the best average classfcaton rate (80.02%), whle the LE has the lowest average classfcaton rate (73.47%). Buldngs can be solated from other objects n the results based on LE, whle vegetaton can be solated by the classfer based on LTSA. In general, t s beleved that LTSA performs better than the other two dmensonalty reducton methods, LE and LLE. Methods Objects Table 1. Classfcaton results based on the LLE, LE, and LTSA methods. Pxels from ground truth LLE LE LTSA Classfed pxels Rate Pxels from ground truth Classfed pxels Rate Pxels from ground truth Classfed pxels Hlls and Road % % % Lakebed % % % Buldngs N/A N/A N/A % N/A N/A N/A Vegetaton N/A N/A N/A N/A N/A N/A % Average Rate 78.01% 73.47% 80.02% Rate Neghborhood Dstorton Index for Performance Evaluaton of Dmensonalty Reducton The above expermental analyss manly uses statstcs to evaluate the performance of the dfferent dmensonalty reducton methods. However, these statstcs are largely dependent on the performance of the classfer adopted n the processng. Consequently, ths statstc analyss may not be capable of fully studyng the capabltes of these varous methods. Ths paper proposes a crteron, the Neghborhood Dstorton Index (NDI), for evaluatng the performance of the dmensonalty reducton methods. We assume that the deal dmensonalty reducton method should fully preserve the topologcal relatonshps between the data ponts durng the reducton n spectral space. Ths means that the same clusterng results could be obtaned from ether the orgnal hgh-dmensonal dataset or the dmenson-reduced dataset. If there are some nconsstences, then we consder them to be dstortons due to the dmensonalty reducton. The NDI has been developed to evaluate the type of dstorton caused by the dmensonalty reducton methods. For any pxel a n a hyperspectral mage data set, there wll be a vector V a n the spectral space. The n neghborng pxels around a wll also have vectors V b ( = 1,2,...,n) n the spectral space (Fgure 5). The NDI D a for a can be calculated usng the followng equaton:

6 D a 1 = n n = 1 α α ( ) α where α s the ntersecton angle between vector V a and V b n the orgnal spectral space and α s the correspondng angle n the reduced low-dmenson space. We use the ntersecton angles of the vectors to descrbe ther topologcal relatonshp. After the NDIs for all the pxels n the hyperspectral mage have been calculated, an NDI map can be generated to llustrate the overall dstorton of the results from the dmensonalty reducton. (1) Fgure 5. The topologcal relatonshps between the hgh-dmensonal data and the dmenson-reduced data. Fgure 6 shows the NDI maps of the dmensonalty reducton results from LLE, LE and LTSA, respectvely. In these maps, the pxels wth larger NDIs have darker gray values. The average NDI for the LLE, LE, and LTSA NDI maps are 0.91, 1.43 and 0.32, respectvely. Ths means that LTSA performs best for preservng topologcal relatonshps for dmensonalty reducton. These results are consstent wth those results obtaned from the prevous analyss by comparng wth the ground truth. a) b) c) Fgure 6. NDI maps of the results of dmensonalty reducton from: a) LLE, b) LE, and c) LTSA. DISCUSSION AND CONCLUSIONS Ths paper nvestgates a system that ntegrates bologcally and geometrcally nspred approaches to detect objects from HS/MS, multscale and multplatform mages. The archtecture and concept of the ntegrated system have been studed and llustrated. As the frst step of the proposed system, ths paper studes the dmensonalty reducton methods for hyperspectral dmensonalty reducton. Usng the hyperspectral data collected at Slver Lake, three dmensonalty reducton methods ncludng LLE, LE, and LTSA, were used for dmensonalty reducton. To compare the performance of these methods, a Neghborhood Dstorton Index s developed. By expermental statstc comparson and analyss usng NDI maps, LTSA has the best capablty to preserve the features n hgh-dmensonal dataset reducton. Ths paper summarzes the prelmnary results of our research. Our future works wll focus on the further development of the S-LEGION and GAC to fully mplement the proposed ntegrated system for bologcally/geometrcally-nspred object detecton.

7 ACKNOWLEDGEMENT Ths work was supported by an NGA Unversty Research Intatves grant (HM ). REFERENCES Bachmann, C.M., T.L. Answorth, and R.A. Fusna, Modelng data manfold geometry n hyperspectral magery, Proceedngs of 2004 IEEE Geoscence and Remote Sensng Symposum, 5: Beck, R., EO-1 user gude v.2.3, Avalable at Belkn, M., and P. Nyog, Laplacan egenmaps for dmensonalty reducton and data representaton, Neural Computaton, 15(6): Caselles, V., R. Kmmel, and G. Sapro, Geodesc actve contours, Internatonal Journal of Computer Vson, 22(1): Datt, B., T.R. McVcar, T.G. van Nel, D.L.B. Jupp, J.S.Pearlman, Preprocessng EO-1 Hyperon hyperspectral data to support the applcaton of agrcultural ndexes, IEEE Transactons on Geoscence and Remote Sensng, 41(6, Part 1): Fsher, R.A., The use of multple measurements n taxonomc problems, Annals of Eugencs, 7: Green, A.A., M. Berman, P. Swtzer, and M.D. Crag, A transformaton for orderng multspectral data n terms of mage qualty wth mplcatons for nose removal, IEEE Transactons on Geoscence and Remote Sensng, 26: Han, T., and D.G. Goodenough, Nonlnear feature extracton of hyperspectral data based on locally lnear embeddng (LLE), Proceedngs of 2005 Geoscence and Remote Sensng Symposum, 2: Hotellng, H., Analyss of a complex of statstcal varables nto prncpal components, Journal of Educatonal Psychology, 24: Hughes, G.F., On the mean accuracy of statstcal pattern recognzers, IEEE Transactons on Informaton Theory, IT-14: Lafon, S., and A.B. Lee, Dffuson maps and coarse-granng: A unfed framework for dmensonalty reducton, graph parttonng, and data set parameterzaton. IEEE Transactons on Pattern Analyss and Machne Intellgence, 28(9): Levna, E., and P.J. Bckel, Maxmum Lkelhood Estmaton of Intrnsc Dmenson, Neural Informaton Processng Systems: NIPS, Vancouver, CA, December, L, D., S. Fang, and Y. Zhou, A technque for nverson of water qualty parameters n the East Lake based on spectral reflectance measurement, Proceedngs of SPIE, the Internatonal Socety for Optcal Engneerng, 6043(2):60430M M.7. Nu, X., A sem-automatc framework for hghway extracton and vehcle detecton based on a geometrc deformable model, ISPRS Journal of Photogrammetry and Remote Sensng, 61(3-4): Osher, S., and J. Sethan, Fronts propagatng wth curvature-dependent speed: algorthms based on the Hamlton-Jacob Formulaton, Journal of Computer Physcs, 79(1): Rowes, S.T., Nonlnear dmensonalty reducton by locally lnear embeddng, Scence, 290(5500): Scholkopf, B., A.J. Smola, and K.-R. Muller, Nonlnear component analyss as a kernel egenvalue problem, Neural Computaton, 10(5): Tenenbaum, J.B., V. de Slva, and J.C. Langford, A global geometrc framework for nonlnear dmensonalty reducton, Scence, 290(5500): Terman, D., and D.L. Wang, Global competton and local cooperaton n a network of neural oscllators, Physca D. 81: van der Maaten, L.J.P., E.O. Postma, and H.J. van den Herk, Dmensonalty reducton: A comparatve revew, Avalable at Wang, D.L., and D. Terman, Image segmentaton based on oscllatory correlaton, Neural Computaton, 9: Wang, D.L., The tme dmenson for scene analyss, IEEE Transactons on Neural Networks, 16: Zhang, Z., and H. Zha, Prncpal manfolds and nonlnear dmensonalty reducton va local tangent space algnment, SIAM journal of Scentfc Computng, 26(1):