DECISION LEVEL FUSION OF LIDAR DATA AND AERIAL COLOR IMAGERY BASED ON BAYESIAN THEORY FOR URBAN AREA CLASSIFICATION

Transcription

1 DECISION LEVEL FUSION OF LIDAR DATA AND AERIAL COLOR IMAGERY BASED ON BAYESIAN THEORY FOR URBAN AREA CLASSIFICATION H. Rstiveis* School of Surveying nd Geosptil Engineering, Fculty of Engineering, University of Tehrn, Tehrn, Irn, KEY WORDS: High Resolution LiDAR Dt, Clssifier, Decision Level Fusion, Clssifiction ABSTRACT: Airborne Light Detection nd Rnging (LiDAR) genertes high-density 3D point clouds to provide comprehensive informtion from obect surfces. Combining this dt with eril/stellite imgery is quite promising for improving lnd cover clssifiction. In this study, fusion of LiDAR dt nd eril imgery bsed on Byesin theory in three-level fusion lgorithm is presented. In the first level, pixel-level fusion, the proper descriptors for both LiDAR nd imge dt re extrcted. In the next level of fusion, feture-level, using extrcted fetures the re re clssified into six clsses of Buildings, Trees, Asphlt Rods, Concrete rods, Grss nd Crs using clssifiction lgorithm. This clssifiction is performed in three different strtegies: (1) using merely LiDAR dt, (2) using merely imge dt, nd (3) using ll extrcted fetures from LiDAR nd imge. The results of three clssifiers re integrted in the lst phse, decision level fusion, bsed on lgorithm. To evlute the proposed lgorithm, high resolution color orthophoto nd LiDAR dt over the urbn res of Zeebruges, Belgium were pplied. Obtined results from the decision level fusion phse reveled n improvement in overll ccurcy nd kpp coefficient. 1. INTRODUCTION Airborne Light Detection nd Rnging (LiDAR) genertes highdensity 3D point clouds to provide comprehensive informtion of obect surfces. Recently, the use of LiDAR dt hs incresed in mny pplictions, such s 3D city modeling nd urbn plnning. Although the sptil resolution of this dt hs intensely improved, however, the lck of spectrl nd texturl informtion is still big problem of LiDAR technology. On the other hnd, high resolution eril/stellite imgeries offer very detiled spectrl nd texturl informtion but poor structurl informtion. Therefore, LiDAR dt nd eril/stellite imgery re complementry to ech other nd, combining them is quite promising for improving lnd cover clssifiction (Lee et l., 2008; Li et l., 2007; Pedergnn et l., 2012; Rottensteiner et l., 2005; Schenk nd CsthA, 2002). Mny methods for fusion of LiDAR dt nd multispectrl eril/stellite imge hve been proposed by reserchers, in recent yers(li et l., 2013; Mlpic et l., 2013; Schenk nd CsthA, 2002; Sohn nd Dowmn, 2007; Trinder nd Slh, 2012; Yousef nd Iftekhruddin, 2014). The mority of these studies hve pplied eril imge insted of stellite imge s complementry of LiDAR dt. Moreover, there re number of reserches fused hyper-spectrl imge nd LiDAR dt for different pplictions(bigdeli et l., 2014; Dlponte et l., 2008). Here, few number of these studies re briefly discussed. Bigdeli et l. (2014) ddressed the use of decision fusion methodology for the combintion of hyperspectrl nd LIDAR dt in lnd cover clssifiction. The proposed method pplied support vector mchine (SVM)-bsed clssifier fusion system for fusion of hyperspectrl nd LIDAR dt in the decision level. First, feture spces re extrcted from LIDAR nd hyperspectrl dt. Then, SVM clssifiers re pplied on ech feture dt. After producing severl of clssifiers, Nive Byes s clssifier fusion method combines the results of SVM clssifiers from two dt sets. The results discovered tht the overll ccurcies of SVM clssifiction on hyperspectrl nd LIDAR dt seprtely were 88% nd 58% while the decision fusion methodology receive the ccurcy up to 91%(Bigdeli et l., 2014). An nlysis on the oint effect of hyperspectrl nd light detection nd rnging (LIDAR) dt for the clssifiction of complex forest re bsed on SVM lgorithm is proposed in (Dlponte et l., 2008). Hong et l. (2009) proposed fusion method by fusing the LiDAR points with the extrcted points from imge mtching through three steps: (1) registrtion of the imge nd LiDAR dt using the LiDAR dt s control informtion; (2) imge mtching using the LiDAR dt s the initil ground pproximtion nd (3) robust interpoltion of the LiDAR points nd the obect points resulted from imge mtching into grid(hong, 2009). Zbuwl et l. (2009) proposed n utomted nd ccurte method for building footprint extrction bsed on the fusion of eril imges nd LiDAR. In the proposed lgorithm, first initil building footprint ws extrcted from LiDAR point cloud bsed on n itertive morphologicl filtering. This initil segmenttion result ws refined by fusing LiDAR dt nd the corresponding colour eril imges, nd then pplying the wtershed lgorithm initilised by the LiDAR segmenttion ridge lines on the surfce were founded(zbuwl et l., 2009). The fusion of eril imgery nd LiDAR dt hs been proposed to improve the geometricl qulity of the building outlines (Rottensteiner et l., 2005). They re lso pplied to improve plnr segmenttion due to the complementry of these dt sources (Khoshelhm et l., 2008). In this pper, multi-level fusion technique is proposed for lnd cover clssifiction using LiDAR dt nd eril imgery. This method is performed through four consecutive phses: preprocessing, pixel-level fusion, feture-level fusion nd decision level fusion. * Corresponding uthor doi: /isprsrchives-xl-1-w

2 2. NAÏVE BAYES FUSION METHOD The Byesin lgorithm combines trining dt with priori informtion to clculte posteriori probbility of hypothesis. So, the most probble hypothesis ccording to the trining dt is possible to figure out. The bsis for ll Byesin Lerning Algorithms is the Byes Rule which is Eqution 1. Where, P ( D h) P ( h) P ( h D ) (1) P( D) Aeril Orthophoto Histogrm Equliztion - Red Bnd - Green Bnd - Blue Bnd - RGVI Trining Dt LiDAR Point Cloud Grid Dt Genertion - FP DSM-LP DSM - FP DSM-DTM - FP Rnge - FP Intensity Input Dt Pre- Processing Pixel Level Fusion P(h) nd P(D) re prior probbilities of hypothesis h nd D, nd P(h D) nd P(D h) re probbility of h given D nd D given h, respectively. Clssifiction Clssifiction Clssifiction Feture Level Fusion Here the conditionl independence of the ttributes of the instnces is required for the use of in Clssifiers. To brought it into formul, let X be set of instnces xi = (x1, x2,, xn) nd w be set of clssifictions w w mx P ( w x, x..., x ) w w P ( x 1, x 2..., x n w ) P ( w ) mx w w P ( x, x...,x ) mx P ( x, x..., x w ) P ( w ) v w n n n (2) Where P ( x, x..., x w ) P ( x w ) (3) And ( ) Pw n i i is priori probbility of clss w. These formule cn be used in both feture nd decision level dt fusion. In the feture level cse, extrcted descriptors/fetures from trining dt set re pplied to clculte P (x w ). A priori probbility my lso be clculted i bsed on the size of the trining dt in ech clss. Estimted posterior probbility in ech clssifiction my lso be pplied in decision level fusion. In this cse, one cn clculte priori informtion of ech clssifiction using its resulted confusion mtrix. 3. HIGH RESOLUTION LIDAR AND IMAGE FUSION The proposed method for fusion of high resolution LiDAR dt nd eril orthophoto is bsed on the flowchrt shown in Fig. 1. First, in pre-processing step, conversion of LiDAR point cloud into grid form nd contrst enhncement of the orthophoto re performed. Then, clssifiction of the re is executed in three sequentil fusion levels: pixel level fusion, feture level fusion, nd decision level fusion. As seen in Fig. 1, in pixel level fusion useful descriptors re clculted from both LiDAR nd imge. After tht, extrcted fetures re pplied for clssifiction of the re in three different strtegies: (1) using merely LiDAR dt, (2) using merely imge dt, (3) using ll extrcted fetures from LiDAR nd imge. Finlly, ll the clssifiction results from these implementtions re used in decision level fusion for mking lst decision bout the pixels. Further detils of the proposed method re described in the following sections. Figure 1. Flowchrt of the proposed method 3.1 Pre-Processing In the pre-processing phse, to simplify the process nd, bility to del with the LiDAR dt s n imge, irregulr 3D point cloud is converted into regulr form using interpoltion techniques. Although, the interpoltion process my cuse the loss of informtion, however, it is negligible in this pper. Note, it is ssumed tht LiDAR dt nd eril imge re ccurtely registered. Moreover, histogrm equliztion of the color orthophoto is performed due to its effectiveness on contrst enhncement. 3.2 Pixel Level Fusion The im of pixel-level phse is to generte the proper descriptors for both dt. In this step, eight descriptors re extrcted on LiDAR dt (four fetures) nd eril imge (four fetures). These fetures re selected bsed on the previous litertures of LiDAR or eril imge clssifiction (Bigdeli et l., 2014; Li et l., 2007). For exmple, the height differences between the first pulse rnge nd DTM to distinguish buildings nd trees from other obects nd, lso, the height differences between the first pulse nd the lst pulse to distinguish tree clss from other clsses cn be seen in severl studies (Bigdeli et l., 2014; Li et l., 2007; Rottensteiner et l., 2005). These descriptors cn be clculted using equtions 4 nd 5. First pulse rnge nd First pulse Intensity re the other descriptors which re extrcted from LiDAR dt. NDSI Decision Fusion Obect s ndsm Lst pulse rnge- DTM First Pulse Rnge- Lst First Pulse Rnge pulse Rnge Lst pulse Rnge Decision Level Fusion Obect s From the orthoimge four descriptors of Red bnd, Green bnd, Blue bnd nd Green-Red Vegettion Index re considered for clssifiction. Here, therefore, only GRVI feture is clculted through pixel level fusion of Red nd Green chnnels. Sme s NDVI in remote sensing dt nlyses, GRVI )4( (5) doi: /isprsrchives-xl-1-w

3 my help to distinguish vegettion re from other obects. Eqution 6 shows the fusion formul for clculting this feture. G - R GRVI G R (6) 3.3 Feture Level Fusion In the feture-level fusion phse, the re re clssified into six clsses of Buildings, Trees, Asphlt Rods, Concrete rods, Grss nd Crs using forementioned extrcted fetures from LiDAR nd imge dt. Trining nd check dt set of ech clss re mnully selected for the clssifiction. Three different clssifiers re implemented in this level. (1) clssifier which merely used LiDAR dt, (2) clssifier which merely used imge dt, (3) clssifier tht pplied ll extrcted fetures from LiDAR nd imge dt. Although in clssifier s soft clssifier membership degrees of ech pixel to the clsses re clculted, here, only one clss with higher degree of membership (mximum probbility), which is shown in Eqution 2, is selected. However, the degree of membership to the clsses re kept to be used in the next phse of the lgorithm, which will be described in the following section. 3.4 Decision Level Fusion In this step, three previous clssifiction results re integrted for mking finl decision bout pixel. There re different decision fusion techniques in pttern recognition literture such s Simple Voting, Weighted Voting, Rule Bsed Fuzzy System, Dempster- Shfer nd. Voting nd Weighted Voting lgorithms cn be pplied for fusing crisp clssifiction results while other methods would be ble to integrte soft clssifiction results. In this pper, the decision level fusion is implemented bsed on Byesin theorem. For this purpose, provides method for computing the posteriori degree of membership, bsed on previous estimted degrees. In the resulted posteriori degrees of membership of ech pixel to the clsses, the mximum degree cn be considered s the finl clss lbel. Here, resulted confusion mtrix for ech clssifier cn be pplied to estimte priori probbility of ech clss. b Figure 2. Study re.. High resolution eril orthophoto. b. High resolution LiDAR point cloud. 4.2 Results After generting regulr LiDAR dt with 5 cm sptil resolution from point cloud, nd contrst enhncing of the orthophoto, in the pre-processing step, the fetures for both LiDAR nd orthophoto were extrcted. These fetures from LiDAR dt nd orthoimge re displyed in Figure 3 nd 4, respectively. 4.1 Dtset 4. EXPERIMENT AND RESULTS To evlute the proposed lgorithm, high resolution color orthophoto nd LiDAR dt over the urbn res of Zeebruges, Belgium were pplied. The point density for the LiDAR sensor is pproximtely 65 points/m² nd the color orthophoto were tken t ndir nd hve sptil resolution of pproximtely 5 cm. From this dt set building block which included 1.03 million points ws cropped s smple dt. Selected re s test dt is depicted in Figure 2. b c d Figure 3. Extrcted Fetures for LiDAR dt.. difference between lst pulse nd DTM. b. difference between first nd lst pulse rnge. c. First pulse intensity. d. First pulse rnge. doi: /isprsrchives-xl-1-w

4 b c d Figure 4. Extrcted Fetures of orthophoto.. Red chnnel. b. Green chnnel. c. Blue chnnel. d. NDGI. Smple dt collection is the next step fter feture extrction. In this cse, s ll the evlution prmeters re computed bsed on these smples, here, huge number of smple dt were collected. The collected smple dt were divided into two groups of trining nd check dt for clculting the probbility density functions nd confusion mtrix, respectively. In this study, smple pixels were collected nd nd smple pixels were collected s trining nd check dt, respectively. Selected smples re shown in Figure 5. Figure 6. Obtined results from Byesin clssifiers bsed on merely orthophoto. Figure 5. Mnully observed smple dt. After collecting smple dt, clssifiers were designed nd executed on the smple dt set in three different strtegies. clssifier is soft clssifiction technique nd results degree of memberships for ech pixel in different clsses. In this cse, the clss with higher degree of membership (mximum probbility) is selected for pixel. The obtined clssifiction results nd corresponding confusion mtrix cn be seen in Figures 6-8. However, the degree of membership to the clsses were kept to be used for decision level fusion. Figure 7. Obtined results from Byesin clssifiers bsed on merely LiDAR feture spce. doi: /isprsrchives-xl-1-w

5 4.3 Discussion Among the obtined three clssifier in feture level fusion, the one which simultneously used both LiDAR nd Imge fetures presented better results. Furthermore, fusing the three clssifiction through clssifier fusion method proved n improvement in clssifiction results. Figure 10 shows resulted overll ccurcy nd kpp coefficient for ll the clssifictions. As cn be seen from the figure, the best performnce ws chieved for finl decision level fusion while obtined results from merely imge fetures ws dispiriting. The performnce of the clssifier which used merely imge fetures in detecting trees nd crs clsses ws very disppointing. Low vlues of user ccurcies for these clsses in Figure 6 proved this. The overll ccurcy nd kpp coefficient of the clssifier which used merely LiDAR fetures is pproximtely the sme s the one which used ll LiDAR nd Imge fetures. However, it is seen in Figure 7 tht lck of spectrl informtion in LiDAR dt cuses mixing grss, sphlt rods nd concrete rods clsses, especilly in south-estern prt of the test re. Moreover, s previously reported in reserches multiple dt resources obtined more promising results in comprison with ech dt resource individully(bigdeli et l., 2014). Figure 8. Obtined results from Byesin clssifiers bsed on LiDAR nd imge feture spce. The results of three clssifiction lgorithm were finlly integrted bsed on dt fusion lgorithm. Finl clssifiction results is shown in Figure 9. As cn be seen from the figure, the confusion mtrix obtined from fusion of previous clssifiction presents more promising results. Figure 10. Comprison between clssifiction results. 5. CONCLUSION In this pper, fusion of high resolution eril orthophoto nd LiDAR dt bsed on in lgorithm were discussed. Three different clssifiction were designed using trining dt set: (1) using merely LiDAR dt, (2) using merely imge dt, (3) using ll extrcted fetures from LiDAR nd imge. The results of these clssifiction were integrted using lgorithm. Among ll the clssifiction results, the results of finl decision fusion were the best. Although the fetures nd number of clsses hve importnt roles in clssifiction, it is theoreticlly expected tht the sme results would be chieved for different feture spces nd number of clsses. However, it is recommended to test the lgorithm for other cse studies. It is lso suggested to test other decision fusion lgorithm such s fuzzy inference system. Figure 9. Finl obtined clssifiction from decision level fusion. REFERENCES Bigdeli, B., Smdzdegn, F., Reinrtz, P., A decision fusion method bsed on multiple support vector mchine system doi: /isprsrchives-xl-1-w

6 for fusion of hyperspectrl nd LIDAR dt. Interntionl Journl of Imge nd Dt Fusion 5, Dlponte, M., Bruzzone, L., Ginelle, D., Fusion of hyperspectrl nd LIDAR remote sensing dt for clssifiction of complex forest res. Geoscience nd Remote Sensing, IEEE Trnsctions on 46, GRSS_DFC, IEEE GRSS Dt Fusion, 2015 IEEE GRSS Dt Fusion Contest., in: IEEE (Ed.), Online: Hong, J., Dt fusion of LiDAR nd imge dt for genertion of high-qulity urbn DSM, Proceedings of the oint urbn remote sensing event. IEEE., Shnghi, Chin. Khoshelhm, K., Nedkov, S., Nrdinocchi, C., A comprison of Byesin nd evidence-bsed fusion methods for utomted building detection in eril dt. Interntionl Archives of the Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences 37, Lee, D.H., Lee, K.M., Lee, S.U., Fusion of lidr nd imgery for relible building extrction. Photogrmmetric Engineering & Remote Sensing 74, Li, H., Gu, H., Hn, Y., Yng, J., Fusion of high-resolution eril imgery nd lidr dt for obect-oriented urbn lnd-cover clssifiction bsed on svm. Proceedings of the ISPRS Working Group IV/1:â œdynmic nd Multi-dimensionl GIS, Li, Y., Wu, H., An, R., Xu, H., He, Q., Xu, J., An improved building boundry extrction lgorithm bsed on fusion of opticl imgery nd LiDAR dt. Optik-Interntionl Journl for Light nd Electron Optics 124, Mlpic, J.A., Alonso, M.C., Ppí, F., Arozren, A., Mrtínez De Agirre, A., Chnge detection of buildings from stellite imgery nd lidr dt. Interntionl ournl of remote sensing 34, Pedergnn, M., Mrpu, P.R., Mur, M.D., Benediktsson, J.A., Bruzzone, L., Clssifiction of remote sensing opticl nd lidr dt using extended ttribute profiles. Selected Topics in Signl Processing, IEEE Journl of 6, Rottensteiner, F., Trinder, J., Clode, S., Kubik, K., Using the Dempster-Shfer method for the fusion of LIDAR dt nd multi-spectrl imges for building detection. Informtion fusion 6, Schenk, T., CsthA, B., Fusion of LIDAR dt nd eril imgery for more complete surfce description. Interntionl Archives of Photogrmmetry Remote Sensing nd Sptil Informtion Sciences 34, Sohn, G., Dowmn, I., Dt fusion of high-resolution stellite imgery nd LiDAR dt for utomtic building extrction. ISPRS Journl of Photogrmmetry nd Remote Sensing 62, Trinder, J., Slh, M., Aeril imges nd LiDAR dt fusion for disster chnge detection. ISPRS Annls of Photogrmmetry, Remote Sensing nd Sptil Informtion Sciences 1, Yousef, A., Iftekhruddin, K., Shoreline extrction from the fusion of LiDAR DEM dt nd eril imges using mutul informtion nd genetic lgrithms, Neurl Networks (IJCNN), 2014 Interntionl Joint Conference on. IEEE, pp Zbuwl, S., Nguyen, H., Wei, H., Ydegr, J., Fusion of LiDAR nd eril imgery for ccurte building footprint extrction, IS&T/SPIE Electronic Imging. Interntionl Society for Optics nd Photonics, pp Z-72510Z doi: /isprsrchives-xl-1-w