3D TERRAIN SEGMENTATION IN THE SWIR SPECTRUM

Size: px

Start display at page:

Download "3D TERRAIN SEGMENTATION IN THE SWIR SPECTRUM"

Lionel Jacobs
5 years ago
Views:

1 3D TERRAIN SEGMENTATION IN THE SWIR SPECTRUM Dalton Rosario 1, Anthony Ortiz 2, an Olac Fuentes 2 U.S. Ary Research Laboratory 1, University of Texas at El Paso 2 ABSTRACT We focus on the autoatic 3D terrain segentation proble using hyperspectral shortwave IR (HS-SWIR) iagery an 3D Digital Elevation Moels (DEM). The atasets were inepenently collecte, an etaata for the HS-SWIR ataset are unavailable. We explore an overall slope of the SWIR spectru that correlates with the presence of oisture in soil to propose a ban ratio test to be use as a proxy for soil oisture content to istinguish two broa classes of obects: live vegetation fro ipereable anae surface. We show that iage base localization techniques cobine with the Optial Ranoize RANo Saple Consensus (RANSAC) algorith achieve precise spatial atches between HS-SWIR ata of a portion of owntown Los Angeles (LA (USA)) an the Visible iage of a georegistere 3D DEM, covering a wier-area of LA. Our spectral-elevation rule base approach yiels an overall accuracy of 97.7%, segenting the obect classes into builings, houses, trees, grass, an roas/parking lots. Inex Ters 3D terrain segentation, hyperspectral, Liar, shortwave infrare, igital elevation oel, fusion 1. INTRODUCTION There are any challenges associate with the fusion of ifferent sensing oalities, to inclue the uncertainty of feature corresponence aong the oalities ue to ifferences in easure phenoena, spatial resolutions, an viewing perspectives [1]-[2]. We focus on the proble of enabling a achine to autoatically segent a igitize terrain by first fusing aerial Shortwave infrare (SWIR: µ) hyperspectral ata with 3D Liar-erive Digital Elevation Moels (DEMs) that overlap the target scene, then eploying a spectral-elevation rule base approach to segent the igitize terrain into finer aterial classes (e.g., high builings, houses, tree clusters, grassy areas, roas/parking lots). Reliable perforance of such autoatic tasks through software is consiere useful for various types of coercial an ilitary applications, to inclue ission planning for rescue operations [3]. We further constrain the proble in twofol: (i) by using unregistere atasets fro inepenent ata acquisitions an (ii) by not relying on the geo-registration of ata fro one of the sensing oalities. The proble aresse in this paper eparts fro prior works on 3D terrain segentation, which exclusively use hyperspectral ata in the visible spectru (Vis:.4-.7 µ) that are alreay spatially registere to 3D DEMs (see past IGARSS challenges [4]), where variations to the state of the art achine learning ethos are applie to segent the igitize scene via obect classification. This paper also explores an overall slope observe in SWIR spectra that correlates with the presence of oisture in soil to propose a ban ratio test to be use as a proxy for soil oisture content that can istinguish two broa classes of aterial types: live healthy vegetation fro ipereable anae surface. 2. SWIR SPECTRAL ANALYSIS FOR MATERIAL SEGMENTATION The proucts erive fro both spectral an spatial analyses that are helpful to ission planning consist of three proceures: Deterination of soil oisture content; segentation of the iage into vegetation, soil, an water; an erivation of lines of counication (roas, rivers). The aount of oisture in soil can ipact a nuber of phenoena iportant for ission planning, incluing the aount of ust in the air an the firness of the groun. It is intuitive that shoul the soil excee a certain level of oisture content, groun obility of tracke an wheele vehicles will be aversely affecte. Reotely sense spectral iagery can be use to eterine the relative aounts of oisture in surface soils by exaining the reflectance in the SWIR region of the spectru. The atosphere is opaque to SWIR raiation near 1.4 µ an 1.9 µ ue to the presence of gaseous water an carbon ioxie. As a result, surface properties at these wavelengths are not accessible fro reotely sense iages. However, ieiately aacent to these regions, the spectru is influence by the presence of absorbe water (see Fig 1). This is true for an absorption feature near 2.2 µ as well. As a result of the analysis of laboratory spectra of three types of soil characterize as loa, clay, an san it was observe that the overall slope of the spectru in the SWIR also correlates with the presence of oisture in the soil. Specifically, in ters of the hyperspectral SWIR bans use in for this paper, ry soils exhibite an increase in reflectance in bans between 2.9 µ an 2.15 µ over bans between 1.55 µ an 1.75 µ, see Fig. 1. Wet soils were either flat or ecrease in reflectance over that span.

Material spectra courtesy of Spectral Inforation Technology Application Center Library.

2 Fig. 1. Laboratory reflectance spectra for three types of soils loa (green), san (re), an clay (blue) uner two oisture conitions. Areas ost sensitive to the presence of liqui water are highlighte by light blue an light yellow vertical bans. Note the contrast between ry an wet saples. Material spectra courtesy of Spectral Inforation Technology Application Center Library. This suggests that a ban sub-range ratio test can be use as a proxy for soil oisture content in the HS SWIR iage use in our work. We explore this phenoenon by proposing the following wetness inex in orer to istinguish two broa classes of obects in a given terrain (live healthy vegetation fro ipereable anae surface): where 1.75µ = 1.55µ WET = 2.35 Ratio µ = 2.9µ, (1) is the raiance or reflectance value at ban. Notice fro Fig. 1 that the inverse of (1), i.e., 1/Ratio, is expecte to yiel real values between an unity ([,1]) for wet aterials. The behavior of such a rano variable ay be oele by a flexible istribution efine on the interval [,1], such as the beta istribution function, see Fig. 2. The beta istribution is a faily of continuous probability istributions efine on the interval [, 1] paraetrize by two positive shape paraeters, enote by α an β, that appear as exponents of the rano variable an control the shape of the istribution. Moeling (1) as a rano variable an restricting its behavior by the beta istribution faily to eterine the aterial wetness concentration woul allow a likelihoo ratio test to be forulate. It is beyon the scope of this paper to forulate this hypothesis test, although we o have interest in pursuing this approach in the future. In the eantie, we anticipate that exploring the wetness inex in (1) with an aaptively estiate threshol can broaly segent the iage between live vegetation an anae structures in iagery, as it will be shown later. Obect class finer istinction (e.g., between trees an grass) can further be segente by aligning elevation ata to the segente ap pixels. Results using (1) an elevation easureents are also iscusse later. 3. DATASETS We use a ataset collecte by Heawall Photonics over a ulti-block urban area of owntown Los Angeles (LA), California, USA, using Heawall s Hyperspec SWIR hyperspectral iaging sensor [5] onboar a sall anne airplane. Key sensor specifications: 384 spatial bans, 26 spectral bans, wavelength range µ, axiu frae rate 45 Hz, Stirling-coole MCT etectors. The anne airplane flew the pushbroo hyperspectral iaging syste over the target area in Los Angeles, collecting 829 saples by 26 SWIR bans (efine in this case as a frae) an using the airplane s oveent to obtain 1,163 lines. A representation of the atacube consisting of 1,163 lines by 829 saples by 26 bans of the target area in Los Angeles is shown as the ban average (less than 1 pixel resolution) in Fig. 3. The area represente in Fig. 3 inclues Fig. 2. Beta probability ensity function. Fig. 3. SWIR hyperspectral ban average of a ulti-block urban region of Los Angeles, California, USA. live vegetation (trees, grass) an anae structures (high builings, houses, roas, an parking lots, an other unknown anae obects). Notice in Fig. 3 that the ata acquisition occurre closer to sunset; explaining the prolonge shaows observe near the high builings. We conecture that scale invariant features of terrain lanarks between SWIR iages an spatially corresponing Vis iages will be highly correlate; thus, we ecie to

3 explicitly explore this intuition as part of our iage registration approach. For elevation easureents (Fig. 4), we use a 3D DEM representing a significantly wier area of Southern California, containing a large portion of LA County, to inclue ountains an the entire owntown LA. DEM SWIR Fig. 4. 3D DEM (1 pixel resolution) of a portion of Los Angeles County, California, USA, which inclues in relative scale the ulti-block owntown portion of Los Angeles where the SWIR hyperspectral ata were collecte. The DEM is geo-registere for Latitue an Longitue worl coorinates per pixel of equal size (1 spatial resolution), an inclues both elevation easureents fro a Liar syste an a corresponing geo-registere orthophoto (nair Vis iage) to put scene context in the DEM. Fig. 4 epicts the DEM an shows in relative scale the spatial area where within the DEM spatial area the SWIR ata acquisition occurre; but of course this spatial location was unknown prior to the execution of our iage registration etho using these two inepenent regions of the electroagnetic spectru: SWIR an Vis. The provie orthophoto with the DEM also covers an area that is any orers of agnitue larger than the area covere by the SWIR atacube. In the scope of this paper, successful registration by a achine using these atasets constitutes a successful autoatic association of elevation ata with spectra, given that for each aligne pixel between these two iages there is a corresponing spectru available in the hyperspectral atacube. Our iage registration approach is DE iscusse next. 4. SWIR-VISIBLE SPATIAL MATCHING Inspire by iage-base localization approaches siilar to the one introuce by Sattler et al. in [6], an the Optial Ranoize RANSAC algorith [7], we succeee in registering the 2D spatial area of the SWIR hyperspectral atacube onto the 3D DEM using a variation of our prior work [8], resulting in the association of elevation easureents with spectra; as follows. The fusion approach starts by representing each 3D point in the DEM by all the Scale Invariant Feature Transfor (SIFT) escriptors [9] contributing to its spatial locations in the Vis iage. All the escriptors of a single 3D point are assigne to a visual wor. The K-eans clustering algorith is applie to cluster all the 3D point clous into k clusters. We use 1, clusters in this experient. To iprove coputational efficiency, we avoi coparing all of the points in the point clous by assigning the centrois obtaine fro the k-eans to be the visual wors. SIFT is inepenently copute on the SWIR an Vis reference iages an results are represente as visual wors for coparison with visual wors fro the previous step. The visual wor coparison strategy can then be reuce by fining through a sequential search the two nearest visual wors to the visual wor (hyperspectral-base SIFT escriptor). The sequential search yiels a set of corresponences between each hyperspectral-base SIFT escriptor an two 3D oel visual wors. A corresponence is accepte if the two nearest neighbors pass the SIFT ratio test using a threshol. This is base on the iea that the probability that a atch is correct can be eterine by taking the ratio of istance fro the closest neighbor to the istance of the secon closest [9]. The etho reects all atches in which the istance ratio is greater than.7. This threshol works epirically well for this ataset an any others. If ore than one 2D feature atches exist in association with the sae 3D point, the escriptor with sallest Eucliean istance is selecte. The linear search continues until a userspecifie nuber of corresponences is satisfie (in this experient this nuber was set to 1), or the search is exhauste fro the hyperspectral ata perspective. The resulting set of corresponences oes not necessarily guarantee a geoetric alignent of the hyperspectral iages onto the 3D DEM that woul atch the quality that coul have been achieve by huan intervention. In orer to iprove the alignent process, we applie the Optial Ranoize RANSAC algorith to the resulting set of corresponence vectors obtaine up to this stage. The geoetric alignent is eclare as acceptable using the criterion that ore than n corresponences (n = 5 works well epirically in this ataset) ust be inliers to accept a atch. 5. RESULTS AND DISCUSSION The fusion approach escribe in Section 4 was applie to the hyperspectral SWIR ataset an Vis orthophoto of corresponing DEM using the following etails. We chose ban 14 (near 1.2 µ) to represent the spatial area of the entire SWIR hyperspectral atacube because of the high reflectance property of ost aterial types in an urban scene, relative to other bans. In orer to aress ore efficiently the coputational loa an associate tie, given the vast spatial coverage of the DEM relative to the saller hyperspectral SWIR spatial coverage (see Fig. 4), we split in

half the spatial area of the SWIR ban-14 an allowe each half to copute their spatial atches inepenently of each other, such that each half estiate its spatial feature atches within a neighborhoo

4 half the spatial area of the SWIR ban-14 an allowe each half to copute their spatial atches inepenently of each other, such that each half estiate its spatial feature atches within a neighborhoo centere at a rano (without replaceent) spatial location within the wie-area Vis orthophoto, thus eliinating once for all the unesirable locations in the wie-area igitize scene an retaining the better prospects. This strategy is very effective because it allows the coputational loa, which is in the orer of O 2 relative to the spatial area, to be processe in parallel using two saller areas rather than siultaneously using a single twice-as-large spatial osaic, cutting in half the atchingsearch coputational tie. This atching search strategy pai off an yiele the result epicte in Fig. 5, where the Fig. 5. Material segente ap separating aterial types in the scene featuring large concentration of water (trees, bushes, an grass), they are represente by white-yellow pixels in the center iage. Re to arker colore pixels inicate the presence of ipereable anae surfaces in the scene (builings, houses, roas, an other unknown anae obects. Top iage is a corresponing Google Map RGB iage. segente (white, yellow, re, ark) ap represents the SWIR spatial area overlai onto the larger local neighborhoo in the Vis orthophoto that prouce the best atching result. This segente ap was the result fro applying the wetness spectral ratio test in (1), to be iscusse shortly. Notice in Fig. 5 (segente ap) that the split atch is slightly isaligne, since they were perfore inepenently fro each other. We coul rectify this isalignent by applying a final atching search as an aitional last step between the entire SWIR osaic an the neighborhoo where this split conversion occurre in the Vis orthophoto; this optiization, however, was not one for this paper. The real avantage fro this spatial atch is that each voxel in the SWIR atacube not only features 27 bans but also has a corresponing elevation value fro the DEM. Using this ata association, it will be possible to refine the terrain segente aps base on spectral an elevation properties as iscusse earlier. The top iage in Fig. 5 is the corresponing Google Map Vis photo of the target LA area, which was taken in July 217 this is about 24 onths after the HS SWIR ata acquisition of the sae area occurre, an is being use in here for visual reference for the reaer to qualitatively check the aterial types present in the scene. Ieiately below the Google Map Vis photo is the overlai segente ap, inicating the aterials in the scene containing large concentrations of water, where white an yellow pixels epict the wettest aterials an re an arker pixels epict the riest aterials. A visual inspection of Fig. 5 suggests that the aterial types in the scene featuring large concentration of water are those aterials copose of live vegetation (i.e., trees, bushes, an grass), which are represente by white-yellow pixels in the segente ap. Pixels showing re to arker colors see to belong to anae surfaces in the scene, i.e., builings, houses, roas, parking lots, etc. Notice also fro the Google Map Vis photo that the level of wetness observe in the segente ap sees agnostic to the actual color of the obects consisting of live vegetation in the scene, e.g., both brownish grassy regions an green tree clusters observe in the Google Map Vis photo are inistinguishable in the segentation ap. This result suggests that the wetness spectral ratio in (1) is an effective eans to separate live vegetation fro anae structure in the scene, a crucial capability for obtaining ission planning geospatial proucts. Fig. 6 inclues results with elevation. Fig. 6 epicts the result fro our spectral-elevation rule base segentation etho, where by erely using the spectral property istinction in (1) between live vegetation an anae structure in the scene; an taking into account the voxels corresponing elevation ata, which was possible because of the successful registration between the HS SWIR atacube an DEM, a visual inspection between the segente ap in Fig. 6 an the Google Map color iage in Fig. 5 suggests that re obects in the segente ap represent builings (yiele low wetness inex) greater than a typical two floor house (groun level inclue) in the Unite State of Aerica (USA), white obects represent neighborhoo houses (also having low wetness inex) of lower height than three floors, green obects are the grass regions an other lower canopy (e.g., bushes), yellow obects represent the trees (having both high wetness inex an an elevation greater than shorter canopy), an finally black regions represent the roas (having low wetness inex an low elevation), parking lots, an other unknown anae obects having a lower elevation than a typical single-floor house in the USA (e.g., otor vehicles). It is worth noting that the results epicte in Fig. 6 were obtaine without the

shortfalls associate with supervise achine learning, where a vastly large atabase of labele saples is require an often unavailable, an with statistical clustering, where prior inforation about the

5 shortfalls associate with supervise achine learning, where a vastly large atabase of labele saples is require an often unavailable, an with statistical clustering, where prior inforation about the total nuber of iniviual clusters are often require. In our approach, the wetness inex in (1) was copute for the entire igitize scene, a wetness threshol was applie (aaptively estiate to be halfway between the lowest an highest values fro the scene), an finally using the corresponing elevation ata an two elevation threshols (lower than a single floor house an higher than a two-floor house), the resulting terrain segente ap is obtaine as an useful ission planning prouct. Fig. 6. Segente ap of key obects in the scene prouce by our spectral-elevation rule base etho, highlighting tall builings (re), houses (white), trees (yellow), an roas/parking lots/other unknown anae obects (black). To quantify the approach s perforance, we withrew fro the SWIR atacube an store a saple library of key aterial classes (i.e., tree clusters, grassy regions, builings, houses, roas/parking lots), such that each class consiste of 5 ranoly selecte spectral saples, without replaceent, with associate elevation atu per spectral saple. We applie the wetness inex in (1) an the spectral an elevation threshols use to generate the segente ap epicte in Fig. 6 in orer to prouce the perforance results shown in Table 1. Table 1 suarizes perforance in ters of Precision, Recall, an Accuracy, which are borrowe fro atheatical statistics an have becoe popular in the coputer vision, ocuent retrieval, an eical test counities [6]. Precision only takes into account the inforation containe within the preicte class coluns, yieling a noralize correct etection in ters of false alars per preicte class that is not constraine by the true saple size per true class. Recall takes only into account the inforation containe within the true class row, also yieling a noralize correct etection in ters of issing the target class; notice that each row su is constraine by the saple size of each true class, in this case Preicte Class True Class Tree Grass Builing House Roa/ Parking Lot Table 1. Quantifie Perforance. Tree Grass Builing House saples per class. Finally, Accuracy oes give a truly coprehensive perforance easure taking into account all of the key counts in a confusion atrix (i.e., a easure of everything that went right noralize by everything that went both right an wrong in the test trial). The spectralelevation rule base approach i perfor well for this set of etrics, an very well for the overall accuracy at 97.7%. This overall perforance easure was copute by taking into account all of true/false positives an true/false negatives fro all classes. Since elevation ata were use to istinguish the classes, in aition to the wetness inex in (1), we woul not expect to observe false alars between classes that fall outsie expecte elevation ranges (e.g., Builing an Roa/Parking Lot), although the Builing false alar count of 41 ue to House saples (see Table 1) suggests that in soe of the builings portions in the scene fall within the elevation range of the house class. 6. REFERENCES Roa/ Parking Lot [1] P. Ghaisi, M. S. Couceiro, F. M. Martins, an J. A. Beneiktsson, Multilevel iage segentation approach for reote sensing iages base on fractional-orer Darwinian particle swar optiization, IEEE Trans. Geosci. Reote Sens., (214). [2] Q. Weng, Reote sensing of ipervious surfaces in the urban areas: Requireents, ethos, an trens, Reote Sensing of Environent, (212). [3] M. Dalponte, L. Bruzzone, an D. Gianelle, Fusion of hyperspectral an LIDAR reote sensing ata for classification of coplex forest areas, IEEE Trans. Geosci. Reote Sens., (28). [4] Xiang Xu, Jun Li, Antonio Plaza, "Fusion of hyperspectral an LiDAR ata using orphological coponent analysis", Proc. IEEE Int. Geoscience an Reote Sensing Syposiu (IGARSS), (216) [5] Heawall Photonics: Website (as of April 4 th, 218): [6] Torsten Sattler, Bastian Leibe, an Leif Kobbelt, Fast iage-base localization using irect 2-to-3 atching, Proc. International Conference on Coputer Vision, (211). [7] Onre Chu, Jiri Matas, Optial Ranoize RANSAC, IEEE Trans. on Pattern Analysis an Machine Intelligence, (28). [8] A. Ortiz, D. Rosario, O. Fuentes, B. Sion, Iage-Base 3D Moel an Hyperspectral Data Fusion for Iprove Scene Unerstaning, Proc. IEEE IGARSS, (217). [9] Davi G Lowe, Distinctive iage features fro scale invariant keypoints, International Journal of Coputer Vision, (24) Precision (%) Recall (%) Accuracy (%)

AN ATTRIBUTE DRIVEN APPROACH FOR IMAGE REGISTRATION USING ROAD NETWORKS

AN ATTRIBUTE DRIVEN APPROACH FOR IMAGE REGISTRATION USING ROAD NETWORKS Caixia Wang, Peggy Agouris, Anthony Stefaniis Center for Earth Observing an Space Research Departent of Earth Systes an Geoinforation