ALEXNET FEATURE EXTRACTION AND MULTI-KERNEL LEARNING FOR OBJECT- ORIENTED CLASSIFICATION

Transcription

1 ALEXNET FEATURE EXTRACTION AND MULTI-KERNEL LEARNING FOR OBJECT- ORIENTED CLASSIFICATION Lng Dng 1, Hongy L 2, *, Changmao Hu 2, We Zhang 2, Shumn Wang 1 1 Insttute of Earthquake Forecastng, Chna Earthquake Admnstraton, Bejng, Chna 2 Insttute of Remote Sensng and Dgtal Earth, Chnese Academy of Scences, Bejng, Chna - lhy_2003@163.com Commsson III, WG III/1 KEY WORDS: AlexNet, GLCM texture, Mult-kernel learnng, Object-orented classfcaton, feature extracton,svm ABSTRACT: In vew of the fact that the deep convolutonal neural network has stronger ablty of feature learnng and feature expresson, an exploratory research s done on feature extracton and classfcaton for hgh resoluton remote sensng mages. Takng the Google mage wth 0.3 meter spatal resoluton n Ludan area of Yunnan Provnce as an example, the mage segmentaton object was taken as the basc unt, and the pre-traned AlexNet deep convoluton neural network model was used for feature extracton. And the spectral features,alexnet features and GLCM texture features are combned wth mult-kernel learnng and SVM classfer, fnally the classfcaton results were compared and analyzed. The results show that the deep convoluton neural network can extract more accurate remote sensng mage features, and sgnfcantly mprove the overall accuracy of classfcaton, and provde a reference value for earthquake dsaster nvestgaton and remote sensng dsaster evaluaton. 1. INTRODUCTION Hgh resoluton satellte magery s an mage to obtan nformaton of object detaled on the earth s surface.hgh resoluton mage can be used to classfy land use from the process of vsual nterpretaton and classfcaton mage(chen etal.,2015).vsual nterpretaton and classfcaton mage of hgh resoluton mage can produce good classfcaton accuracy(gong etal.,2013;chen etal.,2007;chen etal.,2014). Vsual nterpretaton and classfcaton mage have the dsadvantage related to tme and energy effcency nvolved n satellte mage classfcaton so that classfcaton can be done automatcally to make the process of mage classfcaton faster.wth the ncrease of scales and objects of classfcaton, the computatonal complexty of object-orented mage processng method s ncreasng rapdly, and the classfcaton accuracy has decreased. Recently, remote sensng mage classfcaton s manly based on overlay spectral feature classfcaton, researchers have proposed addng texture feature classfcaton, although the classfcaton accuracy s mproved, but the texture nformaton s stll very lmted to mprove the classfcaton accuracy (Chen etal.,2007;chen etal.,2014), In recent years, Convolutonal Neural Networks (CNN) has made a seres of breakthroughs n mage classfcaton, target detecton, semantc segmentaton and face recognton. The convoluton neural network combnes feature extracton and classfcaton as a whole.the local connecton weghts, sharng and poolng operaton and other characterstcs can effectvely reduce the number of tranng parameters, reduce the complexty of the network and make the model nvarant to mage translaton, zoom, wth a certan degree of dstorton, and has strong robustness and fault tolerance(zhou etal.,2017),compared wth the machne learnng method, t has more powerful ablty of feature learnng and feature expresson (Lu etal.,2016).the convoluton neural network has acheved successful applcaton n hgh-resoluton remote sensng mage scene recognton(hu etal.,2015;zhong etal.,2016; Marmans etal.,2016).those experments have proved the powerful feature extracton capablty of convoluton neural networks.remote sensng mage recognton s very smlar to land use classfcaton. It s necessary to buld classfcaton and recognton related to scene semantcs, whch are very dfferent from land cover classfcaton.the classfcaton of landsat cover s manly based on the spectrum, texture, and so on.the man dffcultes are the dffculty of the characterstc expresson caused by the dversty of the nternal spectrum and the problem of the characterstc expresson of the mxed spectrum s caused by the mxed pxel.can the powerful feature extracton capablty of the convoluton neural network be used to mprove the accuracy of surface coverage classfcaton for medum/hgh resoluton remote sensng mages? The related research on the mprovement of the classfcaton accuracy for hgh resoluton remote sensng mages s very few at present.so, the convoluton neural network s used to study the feature extracton and classfcaton of hgh resoluton remote sensng mages. Ths paper studes the convolutonal neural network for feature extracton and classfcaton of hgh resoluton remote sensng mage, takng Ludan post-earthquake Google mage wth 0.3 meter spatal resoluton of mages as expermental data, the object of the segmentaton mage s the basc unt, AlexNet convoluton neural network(krzhevsky etal.,2012) s used for deep feature extracton, whch respectvely combnes wth spectral feature and GLCM texture. and then, the mult- kernel learnng s used for fuson feature,svm classfer s used for classfcaton, The expermental results show that the deep feature can extract more accurate object features and get hgher classfcaton accuracy.t also shows the shortage of AlexNet n * Correspondng author Authors CC BY 4.0 Lcense. 277

2 the classfcaton of hghlghtng and lowlghtng feature extracton. 2. ALEXNET FEATURE EXTRACTION AND MULTI-KERNEL LEARNING CLASSIFICATION The classfcaton process s shown Fgure 1 based on the convoluton neural network AlexNet and SVM, It manly conssts of three steps: Fgure 1. The flow chart of classfcaton wth deep features extracton by deep CNN 1) Image preprocessng: snce the pre-traned AlexNet network model on the ImageNet data s as a feature extractor, the objects of remote sensng mage segmentaton are as the nput of AlexNet model. 2) Deep feature extracton: the objects of mage segmentaton are as the feature extracton unt. Accordng to the AlexNet nput requrements, each object s normalzed and neghborhood nterpolaton samplng(krzhevsky etal.,2012). In ths paper. The feature extracton s completed on the MatConvNet(Vedald etal.,2015)platform. 3) Mult-kernel learnng SVM classfcaton: Dfferent mage features correspond to a kernel functon. These kernel functons form a unfed kernel functon through a combnaton of weghts and then, the kernel functon s used to classfy. In ths paper, the selected kernel functon s lnear kernel functon, and the optmal soluton s selected by usng the most commonly used grd selecton method. It s used to fuse spectral features, deep features, and GLCM texture features, and to extend to object-t orented classfcaton. the compound kernel method s used to fuse two features (spectral features and deep features), and ts weghted kernel method s very attractve, because the method balances the spatal nformaton and spectral characterstcs. The weghted core s n(1): K(x,x)= K(x,x)+(1- )K(x,x ) (1) s s t t j j t j where s a postve parameter that needs to be regulated n the tranng process. A pxel x s redefned by usng spectral s N features n the spectral feature doman s x R,and redefned t N n spatal feature doman usng spatal features, t x R, N and N relatvely represent the number of bands of the s t vectors of spectral and spatal characterstcs, respectvely. and K are the kernel matrces that descrbe ther spectral t nformaton and spatal nformaton. K s the overall kernel matrx. In the orgnal paper, t s used for pxel based classfcaton. In ths study, the complex kernel method s used to fuse spectral features and complex deep features, and s extended to the object-orented classfcaton method. 3. RESULTS AND ANALYSIS The expermental data are the mage of Google post-earthquake n Ludan, Yunnan. The date of acquston s August 20, 2017, the spatal resoluton s 0.3 meter. The mage sze s 2400*2400. After segmentaton, there are 1085 objects (see Fgure 2). The categores nclude woodlands, ntact buldngs, cultvated land, bare land and collapsed buldngs. The tranng sample and the test sample are randomly selected. K s (a) (b) (c) Fgure 2. Images of the expermental area: (a) orgnal mage;(b)segmented map;(c) sample dstrbuton Authors CC BY 4.0 Lcense. 278

3 The man experments of ths paper are as follows: 1) The effects of dfferent layers features on the classfcaton results; In order to analyze whch feature of the AlexNet has more expressve ablty, the last two fully connected layers Fc6, Fc7 and all convoluton layers are extracted.n Fgure 3, t can be seen that the classfcaton accuracy s on the rse wth the ncrease of the depth of the number of layers.the classfcaton accuracy of the fully connected layer s hgher than that of convoluton layers.ths s because the characterstcs of the deep layers are more abstractve and more expressve.but the classfcaton accuracy of Fc6 s hgher than Fc7, ths s because the pre-traned AlexNet s obtaned n ImageNet natural mage, whle the semantc features of Fc7 s stronger, but t s more n lne wth the classfcaton attrbute of the tranng set, so the classfcaton accuracy decreases.therefore, for remote sensng mages on the non-tranng set, the Fc6 s more expressve. Fgure 3. The classfcaton accuracy comparson chart wth deep feature layers 2) The nfluence of the sze of dfferent neghborhood wndows on the classfcaton results. the sze of the wndow s large, the nformaton contaned s too much, and there are many redundant nformaton that wll affect the classfcaton results, nstead, the classfcaton accuracy s reduced. 3) Comparson and analyss of expermental results The optmal classfcaton accuracy can be obtaned when the sze of the neghborhood wndow s 9 * 9, and the Fc6 features are extracted. In order to analyze the effectveness of the proposed method, the same tranng samples and testng samples were used to compare 3 methods of the experments. (1) Method 1:Mult-kernel learnng and SVM classfcaton s carred out usng spectral features and 8-dmensonal GLCM texture features. (2) Method 2:Mult-kernel learnng and SVM classfcaton s carred out usng deep features and spectral features. (3) Method 3: SVM classfcaton s carred out usng deep features. In the quanttatve evaluaton of classfcaton accuracy (see Fgure 5), t can be shown that the overall classfcaton accuracy fused the deep features s hgher than that of the other methods.in partcular, the deep feature added to the spectral nformaton makes the classfcaton precson hghest. In Fgure 4,there are a large number of msclassfcaton of woodlands nto cultvated land n Method 1.Compared to Method 1, the overall accuracy of the classfcaton s mproved obvously by Method 3,t uses the deep features so the small objects are relatvely few(see Fgure 6).Method 2 the classfcaton results are the best, and the msclassfcaton phenomenon s very few(see Fgure 6). It can keep the contnuty of the objects n the classfcaton result, and can save the post-processng operaton of the classfcaton. But careful observaton of the detals of the classfcaton results n Method 2 (see Fgure 6) can be found that the range of collapsed buldngs expanded than actual collapsed buldngs. Intact buldngs s less than actual buldngs. In summary, Method 2 makes the extracton area ncrease or shrnk by makng hghbrghtng and lowbrghtng objects (such as ntact buldngs and collapsed buldngs). n Table 1, The msclassfcaton and leakage of the bare land and collapsed buldngs are relatvely serous n Method 2.t may be due to the applcaton of ReLU nonlnear exctaton functon and maxmum poolng functon n the AlexNet model, makng the hghbrghtng and lowbrghtng objects (such as ntact buldngs and bare land) more erroneous. Fgure 4. Classfcaton accuracy chart wth dfferent wndow szes In order to analyze the effect of wndow sze on the classfcaton results,we select the optmal wndow sze to extract Fc6.We has selected 4 * 4, 6 * 6, 9 * 9, 11 * 11, 14 * 14, and 18 * 18 wndow szes to analyze the classfcaton results.fgure 4 s a comparson chart of the classfcaton results. As can be seen from Fgure 4, as the sze of the wndow ncreases, the accuracy ncreases, the maxmum value s reached at 9 * 9, and the precson wll decrease when the sze s ncreased.ths s because the sze of the wndow s small, the neghborhood nformaton s too lttle, t can not extract the features of the objects very well, so the precson s low.whle Fgure 5. The accuracy comparson chart by dfferent methods Authors CC BY 4.0 Lcense. 279

4 (a) (b) (c) Fgure 6. Classfton mage obtaned by dfferent methods (a) Spectral-texture features; (b)spectral-alexnet features; (c) AlexNet features collapsed buldngs collapsed buldngs collapsed buldngs collapsed buldngs collapsed buldngs collapsed buldngs ntact buldngs woodlands cultvated land bare land Table 4. Confuson matrces obtan by Spectral-AlexNet features by Method 2 4. CONCLUSION Ths paper makes use of AlexNet features for mult-kernel learnng and classfcaton. The features of fully connected layer of AlexNet are more expressve than the features of the convolutonal layer. Wth the ncrease of the wndow sze, the overall classfcaton accuracy s ncreased frst and then reduced, the approprate wndow sze s chosen for feature extracton; Compared to spectral features and texture features, deep convolutonal neural network s more expressve, and can obvously mprove the classfcaton accuracy. Because ReLU s used as a nonlnear exctaton functon n the AlexNet model, and the maxmum value s used to pool, the phenomenon of msclassfcaton and leakage wll occur for hghbrghtng and lowbrghtng objects. In the future work, AlexNet model wll be transformed to further mprove the classfcaton accuracy by selectng the approprate nonlnear exctaton functon, pool operaton and characterstc layer number ACKNOWLEDGEMENTS Ths work was supported n part by the Natonal Hgh Technology Research and Development Program of Chna under Grant 2013AA12A301, and n part by the Fundamental Research Funds of the Insttute of Earthquake Forecastng,Chna Earthquake Admnstraton under Grant 2015IES0203. REFERENCES Chen J, Chen J, Lao A P, et al.,2015.global land cover mappng at 30m resoluton: a pok-based operatonal approach. ISPRS Journal of Photogrammetry and Remote Sensng, 103,pp.7-27 Gong P, Wang J, Yu L, et al.,2013. Fner resoluton observaton and montorng of global land cover: frst mappng results wth Authors CC BY 4.0 Lcense. 280

5 Landsat TM and ETM+ data. Internatonal Journal of Remote Sensng, 34(7), pp Chen B, Zhang Y J, Chen L.,2007.RS mage classfcaton based on SVM method wth texture. Engneerng of Surveyng and Mappng, 16(5),pp Chen G F, Zeng G W, Chen H, et al.,2014. Study of RS mage classfcaton method based on texture features and neural network algorthm. Journal of Chnese Agrcultural Mechanzaton, 35(1),pp Zhou F Y, Jn L P, Dong J.,2017.Revew of convolutonal neural network. Chnese Journal of Computers, 40(7),pp Lu H T, Zhang Q C.,2016. Applcaton of deep convolutonal neural network n computer vson. Journal of Data Acquston and Processng, 31(1),pp Hu F, Xa G S, Hu J W, et al.,2015. Transferrng deep convolutonal neural networks for the scene classfcaton of hgh-resoluton remote sensng magery. Remote Sensng. 7(11),pp Zhong Y F, Fe F, Zhang L P.,2016. Large patch convolutonal neural networks for the scene classfcaton of hgh spatalresoluton magery. Journal of Appled Remote Sensng. 10(2),pp (1-20). Marmans D, Datcu M, Esch T, et al.,2016. Deep learnng earth observaton classfcaton usng ImageNet pretraned networks. IEEE Geoscence and Remote Sensng Letter, 13(1): pp Krzhevsky A, Sutskever I, Hnton G E.,2012. ImageNet classfcaton wth deep convolutonal neural networks[c]//proceedngs of Advances n Neural Informaton Processng Systems. Lake Tahoe, USA: NIPS, pp Vedald A, Lenc K.,2015. MatConvNet-Convolutonal Neural Networks for MATLAB[C]//Proceedng of the 23rd ACM Internatonal Conference on Multmeda. Brsbane, Australa:ACM,pp Authors CC BY 4.0 Lcense. 281