x x 2 x Iput layer = quatity of classificatio mode X T = traspositio matrix The core of such coditioal probability estimatig method is calculatig the

Transcription

1 COMPARATIVE RESEARCHES ON PROBABILISTIC NEURAL NETWORKS AND MULTI-LAYER PERCEPTRON NETWORKS FOR REMOTE SENSING IMAGE SEGMENTATION Liu Gag a, b, * a School of Electroic Iformatio, Wuha Uiversity, , Hubei provice, P. R. Chia, liugagwh@hotmail.com b State Key Laboratory of Iformatio Egieerig i Surveyig, Mappig ad Remote Sesig, Wuha Uiversity KEY WORDS: Remote sesig, Segmetatio, Probabilistic Neural Networks (PNNs), Multi-Layer Perceptio Networks (MLPNs) ABSTRACT: Image segmetatio is oe of the most importat methods for extractig iformatio of iterest from remote sesig image data, but it still remais some problems, leadig to low quality segmetatio. The research focuses o image segmetatio based o PNNs ad MLPNs. It presets to costruct a PNN model ad tues a satisfied PNN for hyper-spectral image segmetatio. Furthermore, the paper gives a comparative study o segmetatio methods based o PNNs ad MLPNs. It is cocluded that PNNs have quick speed of learig ad traiig. The mai advatage of a PNN is its ability to output probabilities i patter recogitio. Image segmetatio based o PNNs is a effective ad efficiet method i image aalysis, it obtais a bit higher segmetatio overall accuracy tha MLPNs.. INTRODUCTION The amout of remote sesig data is very large, ragig from several megabytes to thousads megabytes, it leads to difficult ad complex image processig. A eural etwork is a iformatio processig system that iteds to simulate the architectures of the huma beig s brais ad how they work. It is a kid of self-adapted ad o-liear system, which cosists of a large umber of coected euros. Eve though a sigle euro has simple structure ad fuctio, the systematic behaviour of a great quatity of combiatorial euros could be very sophisticated. Neural etwork has advatage of dealig with o-liear problems ad cosequetly is applied to more ad more research fields, ad its priciple is usually used for patter recogitio. Extractio of remote sesig image iformatio based o eural etworks developed rapidly recetly, ad it has gaied satisfied results i practical works. Image segmetatio is essetial for iformatio extractio from remote sesig image; it is oe of the most importat ad fudametal techologies for image processig; ad it is idispesable to all uderstadig system ad auto recogitio system. Both PNNs ad MLPNs are typical eural etworks. As a classifier, MLPN has bee successfully applied to classificatio of remote sesig image ad PNN is seldom applied to such work. I previous work, such low quality segmetatio problems as obect mergig, obect boudary localizatio, obect boudary ambiguity, obect fragmetatio are still existed i segmetatio based o eural etworks. This research focuses o remote sesig image segmetatio based o PNNs ad MLPNs; it presets to build a PNN model for segmetatio ad gives a comparative study o segmetatio based o PNNs ad MLPNs. images based o artificial eural etwork MLP (Ha, 2004). Tag, etc. studied o images classificatio based o spectral decompositio of graphs usig probabilistic eural etworks (Tag, etc. 2006). Several studies exist o Matlab realizatio of sesig image classificatio based o probabilistic eural etwork (Li, etc. 2008). Researchers have studied about probabilistic eural etworks for extractig remote sesig iformatio of rice platig Area (Yag, etc. 2007). Wu worked o classificatio for remote sesig fused image ad TM image usig MLP (Wu, etc. 200). 2. PNNS AND MLPNS 2. Structure of PNNs ad MLPNs The aim of eural etwork computatio is to perform easily ad simply o the etworks themselves. PNNs were proposed by Specht i 989, it is a type of Radial Basis Fuctio (RBF) etwork, which is suitable for patter classificatio. The PNN classifier is basically a classifier, of which the etwork formulatio is based o the probability desity estimatio of the iput sigals. Oe of its uses is to examie ukows ad to decide to which class they belog (Specht, 989). Specht adopted Bayes classificatio rule ad desity estimatio based o Gaussia fuctio, which was proposed by Parze. He costructed a type of four layer parallel processig etwork, icludig iput layer, patter layer, summatio layer ad output layer (See Figure below). I the past, some researchers were iterested i studyig o correlated works. Ha did works o classificatio model of RS * Correspodig author. liugagwh@hotmail.com 25

2 x x 2 x Iput layer = quatity of classificatio mode X T = traspositio matrix The core of such coditioal probability estimatig method is calculatig the sum of Gaussia ode fuctio. The cetre of odes is samplig poits of traiig patter. The smoothig factor is stadard deviatio σ of Gaussia fuctio. Figure. Typical structure of a PNN A typical structure of PNNs is show i above figure. The iput layer accepts iput vectors. The o-liear dot product processig of iput vectors ad weight vectors is implemeted i the patter layer. The patter layer is the core of a PNN. Durig traiig, the patter vectors i the traiig set are simply copied to the patter layer of the PNN. The classified samples probabilities are calculated i the summatio layer. The output layer is a threshold discrimiator that decides which of its iput from the summatio uits is the maximum, ad fially output classified results. Perceptio etwork was proposed by Roseblatt i 957. Rumelhart preseted a model of MLPNs, which is a etwork with supervised learig algorithm. A MLPN is a popular eural etwork i a lot of applicatios. It is a feed-forward, or o-recurret eural etwork. A MLPN usually has oe or several hidde layers betwee iput layer ad output layer. Each layer cotais a series of euros, ad euros i a layer do ot coect to each other. There is oly oe-way coectio with eighbour layers. Iformatio could oly trasmit alog the directio that is from iput layer to output layer. After weight summatio of the iputs ad implemetatio of o-liear excitatio fuctio, the outputs could be obtaied. 2.2 Algorithms of PNNs ad MLPNs I hierarchical model of a PNN, the summatio layer oly coects to patter layer of correspodig categories. A PNN does ot use a iterative traiig algorithm ad it usually uses expoetial fuctios istead of sigmoid fuctios. The summatio estimated probabilities are o the basis of Parze s method. The algorithm of coditioal probability is as follows (Equatio ). T X W () px ( / fk ) = exp m 2 m 2 (2 π) σ pi f σ k Where Patter layer X = sample vectors to be idetified W = traiig sample vectors m = vector dimesios σ = smoothig factor f k = classificatio mode Summatio layer Output layer I output layer, accordig to probabilities evaluatio of iput vectors, Bayes classificatio rule would assig the iput vectors to the class with maximum posterior probability. A biggest advatage of PNNs is the fact that the output is probabilistic, which makes iterpretatio of output easy. Back Propagatio (BP) learig algorithm is ofte widely used i MLPNs model, it turs the iput ad output problem of samples to o-liear optimizatio problems. It is traied by error back propagatio. BP algorithm is a sort of supervised learig algorithm, it has hidde odes. It miimises a cotiuous error fuctio or obective fuctio. BP is a gradiet descet method of traiig. If there is existed deviatio whe compared the output gaied by etwork forward reasoig to expected output sample, the weight coefficiet should be adusted. For forward propagatio, the BP algorithm is as follows (Equatio 2). The algorithm of learig process is show i Equatio 3. The algorithm for output odes is show i Equatio 4 ad that for o-output odes is show i Equatio 5. Y = f W X i W ( + ) = W ( ) + η δ x (3) δ = ( Ti Yi ) f ' W xi (4) δ ' = f ' W xi δ kw (5) i= Where i f ( x) =, actio fuctio of etworks exp( x) X = iput samples Y = output samples T = expected output samples η = learig rate δ = error sigal W = weight coefficiet f (x) = derivatio of f(x) At the begiig of etwork traiig, the miimum radom itercoected weight values ad threshold values should be selected. Uder the coditio of samples cyclic loadig ad weight values adustig, the cost fuctio could desced to accepted tolerace ad the actual output is equal to desired output i some error rages. The learig process of BP algorithm icludes forward propagatio ad back propagatio. I a word, the algorithm of BP cosists of three stages: feedforward of the iput traiig patter, calculatio ad back propagatio of the associated errors ad weight adustmet. (2) 26

3 3. Data source 3. METHODOLOGY Hyper-spectral remote sesig data are selected to test ad evaluate the processig ad obtaied results. Therefore, it could lead to a easy ad clear procedure to segmet differet lad covers by selected bads image. data should iclude distiguishable lad covers. This makes the study typical ad available. The origial images are show as Figure 2, Figure 3 ad Figure 4 above. Amog the 30 bads of the provided image data, oly the bad 2, bad 9 ad bad 2(respectively show i Figure 2, Figure 3, ad Figure4) are kept for processig through this study. The reaso is that differet spectral characteristics of differet bads lead to the represetative uses of them ad differet segmeted results. 3.2 Data processig Before segmetatio, image data pre-processig icludig oise elimiatio, geometric correctio, atmospheric correctio ad dimesioal reducig, should be doe. I image processig, the performace of a MLPN ad a PNN are affected by some factors, such as iput data types, iput data sequece, umber of odes i differet layers, ad differet traiig parameters icludig mometum, learig rates ad umber of epochs. Figure 2. Bad origial image O the basis of priciples described before, firstly the traiig area of differet lad covers is selected from the same area i origial image data. The pixels i a image are selected as a sample correspodig to each category. The differet bads of image are selected as iputs (Here bad, bad 9 ad bad 2 are used), ad the areas with correspodig lad covers are as outputs. The etwork traiig ad image segmetatio are implemeted with those learig samples. Because of its importace i testig, the spread rate of a PNN model should be adusted costatly, i order to get a ideal value. I the experimets, the better segmeted image is obtaied whe the spread rate is assiged to 0.0. I the mea time, a MLPN with a optimal BP algorithm is tested usig the same samples dataset. Figure 3. Bad 9 origial image As to the method of PNNs, sometimes the probabilities belog to differet categories might be equal because of ucertaity. For example, the differece of probabilities is little, ad the it is difficult to determie the results eve though we ca fid the absolutely biggest probability. To solve this problem, adustmet of traiig parameters ad traiig data could be performed. 3.3 Experimetal results Oly two selected images of the obtaied experimetal results are show i Figure 5 (Segmeted image by MLPN, system error = 0.07, traiig mometum = 0.5, coefficiet = 0.7, learig rate = 0.7) ad Figure 6 (Segmeted image by PNN, σ = 0., allowable error = 0, traiig RSM = 0.). Each image has bee segmeted ito four obvious categories, which separately represet differet areas of water, beach, grasslad ad soil. Figure 4. Bad 2 origial image The origial data are gaied i Poyag Lake, Jiagxi provice, P. R. Chia, which have 30 bads images ad the size of each image is 52 cols 52 rows. To achieve the study goal, the 27

4 Segmeted class Groud truth class Class Ⅰ Ⅱ Ⅲ Ⅳ Total Ⅰ Ⅱ 5 6 Ⅲ 3 5 Ⅳ Total Table 2. PNNs overall accuracy = 86.25% Figure 5. Segmeted image by MLPNs As show i above tables, segmetatio based o MLPNs ad PNNs separately obtai 83.75% ad 86.25% overall accuracy i the experimets. The evaluatio idicates PNNs gai a bit higher accuracy. I fact, umbers of samples strogly ifluece the accuracy statemets. If we could get more groud truth poits, or we segmet a image ito more categories, the evaluatig error would be decreased. 3.4 Discussios The experimets demostrate that applyig the methods of eural etworks to image segmetatio. MLPNs have steady workig state ad simple cofiguratio. But it still has some disadvatages, for example, it eeds eough samples for etwork traiig ad the etwork easily get ito the problem of local miimum. PNNs could avoid these disadvatages because it is based o the thoughts of probability statistics ad Bayes classificatio rule. Figure 6. Segmeted image by PNNs There are some differece betwee Figure 5 ad Figure 6. Firstly, the segmeted regios have differet rages. Secodly, the segmeted image by PNNs (Figure 6) looks smoother. Third, there is more usegmeted regio i Figure 5 (Segmeted images by MLPNs). The overall accuracy is calculated by the ratio of the sum of correctly segmeted pixels i all classes to the sum of the total umber of pixels tested. I this case, the groud truth poits are used as refereces. The overall accuracy by MLPNs ad PNNs are show i Table ad Table 2. Segmeted class Groud truth class Class Ⅰ Ⅱ Ⅲ Ⅳ Total Ⅰ Ⅱ Ⅲ 2 3 Ⅳ Total Table. MLPNs overall accuracy = 83.75% I experimets, if more classes would be obtaied, the structure of MLPNs should be more complicated. With the result that MLPNs would ot get the same high quality segmetatio as PNNs. It is ot easy to kow how strogly the differet parameters of a etwork ifluece the performace. The optimal learig parameters ad traiig parameters should be defied uder the coditios of various tests. I specific procedure of segmetatio, the problems are its uder-costraied ature ad the lack of defiitio of the correct segmetatio. As a result, it is difficult to kow the quality of segmeted image before the results come out. There exist may methods of performace improvemet, icludig optimisig smoothig factor σ, adustig some parameters of PNNs costructio ad employig separate sigma weights. 4. CONCLUSIONS The experimet obtais ideal effects i the study. The overall accuracy of segmetatio achieves 83.75% ad %. The time cost of segmetatio with MLPN ad PNN is also satisfied. I coclusios, MLPNs ad PNNs have better ability i patter recogitio. Oe mai advatage of usig PNNs is the ability to output a probability for each of its classificatio; the other is PNNs eed ot repetitio of etwork traiig. MLPNs eed adequate amout of samples i etwork traiig. With compared to MLPNs, the architecture ad etwork desigig of PNNs are simple. PNNs have quick covergece speed; they could ot easily fall ito local miimum problems. Cosequetly, PNNs have high processig efficiecy ad they are quite suitable for data real time processig. Because of stable euros of etwork layers, MLPNs ad PNNs could be 28

5 simply implemeted i computer hardware. PNNs have better characteristic of fault tolerace. It is discovered that the characteristic vectors dimesios strogly iflueced accuracy statemets. With the icreasig of characteristic vector dimesios, the segmetatio accuracy could be improved. REFERENCES Ha, L., The Classificatio Model of RS Images Based o Artificial Neural Network. Bulleti of Surveyig ad Mappig, 9, pp He, R. B., 200. MATLAB 6 Egieerig Calculatios ad Applicatio. Chogqig Uiversity Press, Chogqig, pp Li, H. Y., Fa, W. Y., MatlabRealizatio of Sesig Image Classificatio Based o Probabilistic Neural Network. Joural of Northeast Forestry Uiversity, 36(6), pp Specht, D. F., 990. Probabilistic Neural Networks. Neural Networks, 3(), pp Tag, J., Zhag, C. Y., Luo, B., Images Classificatio Based o Spectral Decompositio of Graphs Usig Probabilistic Neural Networks. Joural of Images ad Graphics, (5), pp Wu, L. X., Ya, T. L., Zhag, W., 200. Classificatio for RS Fused Image ad TM Image Usig Multi-Layer Perceptio Neural Network. Chiese Joural of Soil Sciece, 32(S0), pp Yag, X. H., Huag, J. F., Study o Probabilistic Neural Network for Extractig Remote Sesig Iformatio of Rice Platig area. Joural of Zheag Uiversity, 33(6), pp Zhag, Y. J., 200. Image Segmetatio. Sciece Press, Beig, pp ACKNOWLEDGEMENTS This study is fuded by Key Laboratory of Geo-iformatio of State Bureau of Surveyig ad Mappig (No ), ad Natioal Key Basic Research Program (Program 973) (No. 2006CB70303). Thaks a lot for their techical ad fiacial supports. 29