Comparison Studies on Classification for Remote Sensing Image Based on Data Mining Method

Transcription

1 Hag Xao ad Xub Zhag Comparso Studes o Classfcato for Remote Sesg Image Based o Data Mg Method Hag XIAO 1, Xub ZHANG 1 1: School of Electroc, Iformato ad Electrcal Egeerg Shagha Jaotog Uversty No. 1954, Huasha Road, Shagha P.R. Cha xaohag@stu.edu.c, Abstract: - Data mg methods have bee wdely appled o the area of remote sesg classfcato recet years. I these methods, eural etwork, rough sets ad support vector mache (SVM have receved more ad more attetos. Although all of them have great advatages o dealg wth mprecse ad complete data, there exsts essetal dfferece amog them. Utl ow, researches of these three methods have bee troduced lots of lteratures but how to combe these theores wth the applcato of remote sesg s a mportat tedecy the later research. However, all of them have ther ow advatage ad dsadvatage. To reveal ther dfferet characters o applcato of remote sesg classfcato, eural etwork, rough sets ad support vector mache are appled to the area of remote sesg mage classfcato respectvely. Comparso result amog these three methods wll be helpful for the studes o emote sesg mage classfcato. Ad also the paper provdes us a ew vewpot o remote sesg mage classfcato the future work. Key-Words: - remote sesg mage classfcato, data mg, eural etwork, varable precso rough sets model, support vector mache, comparso studes 1 Itroducto Studes of remote sesg mage classfcato have receved more ad more atteto sce remote sesg was proposed 1960 s as a sythetcal techque whch has bee used a wde rage of applcatos such as geoscece, agrculture, ad evromet ad so o. Wth cotual crease of mage resoluto, more useful data ad formato ca be obtaed from remote sesg mage. Remote sesg mage classfcato s oe of the most mportat techologes the feld of remote sesg mage processg. How to crease precso of classfcato s a key ssue remote sesg mage processg. To extract useful formato from remote sesg mage, dfferet data mg methods are adopted o ths research area to mprove the precso.curretly, the maor methods remote sesg mage classfcato are statstc classfcato method, structural classfcato method ad fuzzy classfcato method. I these methods, statstc such as mea, varace, stadard devato ad dscrete degree are take as crtera to dstgush dfferet categores. All of them eed umerous statstc calculato but a low classfcato precso. Neural etwork has bee used remote sesg mage classfcato these years ad gotte a satsfyg result. There have plety of lteratures the feld [1][][3]. However, there also have several weakess eural etwork such as slow learg rate, dffcult covergece, complex etwork structure ad umbguous meag of etwork. Wth the developmet of research, there appear may ew theores formato processg ad data mg sce 1980 s. I these data mg theores, rough sets theory receves a most attracto. However, remote sesg mage classfcato, few lteratures o rough sets the feld are foud the research [4]. Large advatages have bee show both rough sets theory ad eural etwork o dealg wth varous mprecse, complete formato. However, there exsts essetal dfferece betwee them. Rough sets smulate abstract logc md of our huma beg whle eural etworks smulate tuto md. Rough sets theory express logc rules based o dscerblty relato ad kowledge reducto whle eural etworks state relato betwee put ad output by usg olear mappg. I geeral, eural etworks ca ot reduce dmesos of puts. More complex structures ad trag cost requred eural etworks of a hgher put dmesos. Rough sets theory ca be used to decrease redudace amog put formato through fdg ther relatos, but rough sets theory s very sestve to oses. Therefore, the good results derved from sample data may ot appear good whe they are appled the set of test data. That s to say, rough sets have a weak error tolerace ad geeralzato performace. Whereas, eural etworks have a better capablty of at-ose, ISSN: Issue 5, Volume 7, May 008

2 Hag Xao ad Xub Zhag self-orgazato, ad geeralzato [5]. Fortuately, varable precso rough sets (VPRS model proposed by Zarko provde us a very useful tool to solve the problem [6]. Varable precso rough sets model allow for some degree of msclassfcato, whch ca avod the hgh sestvty of computatoal results whch s ecessary to crease the system redudace. Ths paper maly troduces applcato of the varable precso rough sets as a example o remote sesg mage classfcato. I the result, performace of system ca be foud to acheve much mprovemet after usg the varable precso rough sets. Besdes rough sets theory, support vector mache (SVM s also a very attractve method data mg area. However, remote sesg mage classfcato, SVM s stll ts begg stage the same as rough set. There s eve fewer researches o SVM feld of remote sesg mage classfcato [7][8]. SVM s a ew data mg ad mache learg theory proposed by Vapk et al. md 1990s. It s a uversal method to solve multdmesoal fucto. It has bee appled some areas such as fucto smulato, patter recogto ad data classfcato ad obtaed a perfect result. There exst some defects eural etwork such as determato of etwork structure, local mma problems, uder learg ad over learg. All of them restrct the applcato of eural etwork. SVM has advatages solvg the problems of olear, patter selected, hgh dmeso, small specme, whch s good complemetary wth eural etwork. Ths paper s orgazed as follows. I secto II, basc theores of three methods are brefly revewed. I secto III, we apply three methods to the classfcato of remote sesg mage respectvely ad gve the comparatve results. Secto IV s the aalyss of the three methods whch advatage ad dsadvatage of three methods are aalyzed ad some suggestos for future research are also preseted. The last secto s cocluso. Bref Revew of Neural etwork, Rough Sets ad Support Vector Mache.1 Neural etwork Sce eural etwork researches revved 1980s, substatal progress has bee acheved applcato as well as theory. Neural etworks have bee wdely appled patter recogto, cotrol optmzato, predctg maagemet ad so o. I the feld of data mg, eural etworks have bee combed wth geetc algorthm, fuzzy sets [9]. Classfcato s a very mportat task area of formato processg ad kowledge dscovery. Classfcato of eural etwork s a supervsed trag algorthm. It has a hgh tolerace capablty ad self-orgazato performace. Lots of works have bee doe ad large umbers of lteratures have bee troduced the feld of eural etworks. Presetly, most methods of eural etwork remote sesg mage classfcato use BP learg algorthm for supervsed learg classfcato. BP etwork s a feedforward etwork whch s fact a olear crtero fucto[10].. Varable Precso Rough Sets Rough sets theory s a ew mathematcal tool data mg area to deal wth vagueess ad ucertaty data, whch ca aalyze ad deal wth varous mprecse ad complete formato. However, tradtoal rough sets are very sestve to eve small msclassfcato errors whch restrct ts applcato greatly. Hece, t s ecessary to crease the system redudace. Here, we maly troduce the VPRS model. Ad VPRS s also take as a example the followg applcato. I covetoal rough sets, uverse U s kow ad cocluso s oly sutable for obects belogg tou. It s very dffcult to satsfy the costras practce. To solve the problem, a method must be foud to geeralze coclusos obtaed from sample data to a more wde area. VPRS s proposed by W.Zarko to solve the problem. Let X, Y be o-empty sets fte feld. If there exst x Y for all x X, we call that X Y. It s obvously that o msclassfcato errors are allowed for the codto. A ew dea s preseted VPRS whch gve a ew measuremet method o cluso relato as follows. 1 card ( X I Y / card ( X f card ( X > 0 (1 c( X, Y = 0 f card ( X = 0 where card ( deote cardal umber of sets. c( X, Y deotes degree of msclassfcato set X to Y. That s to say, there are c( X, Y *100% elemets msclassfed. Obvously, X Y whe c ( X, Y = 0. Therefore, we ca gve a admssble msclassfcato error ( Accordg to the defto, there s: Y X f ad oly f c( X, Y ( ISSN: Issue 5, Volume 7, May 008

3 Hag Xao ad Xub Zhag the Zarko proposed deftos as follows, Suppose that U s uverse, R s dscerblty relato ou. R = {E 1, E, L, E } are parttos of equvalet classes o U. -lower approxmato ( -postve rego of set X, { E R : c( X } R X = U (3 -upper approxmato( -egatve rego of set X, { X = E R : c( X < 1 } R U (4 -boudary rego, { E R : < c( X < 1 } BNR X = U (5 -egatve rego, { E R : c( X 1 } NEGR X = U (6 Zarko gves a very mportat defto VPRS, amely qualty of classfcato. γ ( P, = card( POS( P, / card( U (7 whch POS( P, s a -postve rego o partto Q. Attrbute reducto ad optmal set of attrbute are the most mportat cocepto rough sets model. VPRS provde us two mportat crtera [6], 1. γ ( P, = γ ( RED( P,,. No attrbute ca be elmated from RED( P, wthout affectg the requremet1. There have bee may algorthms for attrbute reducto. Optmal reducto ca be derved from combed mmum cost crtero aturally f t s possble to assg a cost fucto to attrbutes. I the absece of attrbute cost fucto, two basc approaches were preseted by Zarko whch optmal reducto ca be determed accordg to the umber of attrbutes ad rules [6]..3 Support Vector Mache Support vector mache s a ew mache learg theory proposed md 1990s whch s arsed from the statstc learg theory fouded by the Vapk research groups 1960s. It has bee successfully appled to fucto smulato, patter recogto ad data classfcato[11]..3.1 Learly Separable SVM Suppose that there are samples vectors ( x 1, y 1, ( x, y, L, ( x, y belogg to two separate classes where y {+1, 1}. Our target s to buld a crtero fucto to separate the two classes. If there exsts a hyperplae w x + b = 0 whch makes, ( w x + b 1, y = 1 equvalet to ( w x + b 1, y = 1 [ ( w x + b] 1 0 (8 y We ca obta the optmzg problem as follows, 1 m w (9 The Lagrage fucto ca be defed as below, 1 L( w, b, α = ( w w (10 α { y[( w x + b] 1} = 1 where are the Lagrage multplers. (10 ca be α trasformed to ts dual problem order to mmze the equato. Accordg to Küh-Tucker codto, we ca obta the optmal classfcato fucto, f ( x = sg{( w x (11 = sg{ α y ( x x = 1 sg s the symbolc fucto..3. Learly No-separable SVM Separable crtero fucto s bult o the Eucldea dstace, that s K ( x, x = x x. For o-lear problem, sample x ca be mapped oto a hgher dmesoal space ad use the lear classfer o t. That s, we make the trasformato o x, Φ : R H. x Φ(x = ( φ1( x, φ ( x, L (1 the the crtero fucto s chaged to: f ( x = sg{ w Φ( x (13 = sg{ α y Φ( x Φ( x = 1 Therefore, (11 s oly related to er product of trag samples: ( x x. Thus, oly the er product calculatos are eeded the hgher dmesoal space whch ca be realzed usg the fucto tal space. Itroducto of kerel fucto make us solve the problem o the put space wthout calculatg the o-lear mapped patters Φ explctly. Therefore, whe we costruct the classfcato fucto, we make the comparso o the put space ad make the o-lear trasformato to results. Thus eormous work wll be fshed o put space stead of o hgher dmesoal space. Thus ts classfcato fucto s obtaed, ISSN: Issue 5, Volume 7, May 008

4 Hag Xao ad Xub Zhag f ( x = sg{ α y K ( x x (14 = 1 Ths s called support vector mache. Complexty of costructg SVM reles o the umber of support vector stead of dmeso of egespace. Dfferet algorthms are formed usg dfferet kerel fuctos. The kerel fuctos commo use are Polyomal fucto, Radal Bass Fucto (RBF, Mult-layer Perceptro ad so o that are show as follows, (1 Polymoal kerel: d x = [( x, x + 1] ( RBF kerel: x { } = exp γ x - x (3 Sgmod kerel: T x = tah( γ x x Θ (4 Sple kerel : 1 x = 1 + ( x x + ( x x ( x x 1 3 ( x x 6 where ( x x = m( x, x (5 Fourer kerel: N + 1 s ( x x x = x x s 3 Emprcal Comparso Studes of Neural Network, VPRS ad SVM o Remote Sesg Image Classfcato Neural etwork, rough set ad support vector mache are brefly troduced above sectos. From the troducto, we ca fd all of them have ther ow features o data classfcatos. To reveal ther dfferet performace, here, these methods are appled o classfcato for the remote sesg mage respectvely. Also, emprcal comparsos are extracted from the expermet results to descrbe the dfferece amog dfferet eural etwork, ad VPRS wth dfferet parameters ad SVM wth dfferet kerel fuctos. Expermetal data of remote sesg mage are obtaed from the Ladsat mage of someplace Cha. The data clude 1,, 3,4,5,7 sx bads whch correspod to blue, gree, red, ear frared, frared ad far frared bad. Ther relatve mages ca be foud from Fg.1 (a to Fg.1 (f. Illustrato of bads s show table 1. The mage s Obects the mage are classfed by Fg.1(a Fg.1(b Fg.1(c Fg.1(d Fg.1(e Fg.1(f Fg.1(a~(f are obtaed from bad 1,,3,4,5 ad 7 respectvely ISSN: Issue 5, Volume 7, May 008

5 Hag Xao ad Xub Zhag Table 1. Bads of remote sesg mage Bads Blue Gree Red Near Ifrared Ifrared Far Ifrared wavelegth(μm artfcal method to fve categores: rver, house, forest, grass ad road. Gray scale s used as features for classfcato [1]. To avod ose dsturbace, mage blocks are extracted from dfferet bad mages the same place. Calculate the mea gray value μ of the block of each bad respectvely. Therefore, for each samples, a mea vector gray ca be obtaed whch deoted as ( μ 1, μ, μ 3, μ4, μ5, μ7. 10 samples are selected from each class whch 80 s for trag ad the other 40 for testg. There are total 600 data selected for the expermet. Samples are traed ad tested by usg eural etwork, rough sets ad SVM respectvely. For eural etworks, we use the ordary BP etwork (BPNN ad mprovemet BP etwork traed by msclassfcato error. For support vector mache, RBF, sple fucto, bsple, aovasple ad sgmod are choosed as the kerel fucto of support vector mache. Because classfcato of remote sesg mage the paper s a mult-classes problem whle SVM maly solve two-class problem, the multlevel classfcato strategy the paper [7] s adopted to solve ths problem. I paper [7], mult-classes problem ca be trasferred to may sub two-class problems by usg multlevel method. The expermet s worked o the Petum IV 1.5G CPU, 51M memory PC. The results are show table ad table 3. From talbe ad table3, tme cosumed s the logest BPLM, although precso s the hghest t. BPNN s ot coverget the expermet. The results show defects of eural etwork remote Table. Result of classfcato by dfferet methods Method Rver House Forest Grass Road Neural BPNN ot coverget Network BPLM 79.1% 88.3% 88.4% 64.3% 88.% =0 55.1% 50.0% 70.7% 1.% 60.0% Rough Logc = % 61.3% 71.3% 17.1% 69.% = % 66.1% 75.3% 6.1% 77.7% RBF 95.1% 60.3% 65.5% 95.0% 51.7% sple 95.7% 80.7% 70.0% 95.3% 71.8% SVM bsple 94.7% 65.5% 65.8% 68.8% 75.1% aovasple 96.1% 80.3% 55.9% 95.1% 55.4% sgmod 60.1% 0.% 35.4% 65.3% 39.3% Table 3. Precso of classfcato Method Trag Precso Tme(s Trag Sample Test Sample Neural BPNN ot covergece Network BPLM % 8.1% = % 47.4% Rough =0.3 Logc % 54.8% = % 61.0% RBF % 73.6% sple % 8.7% SVM bsple % 74.0% aovasple % 76.6% sgmod % 44.% Note: for the rough logc, trag tme s tme of costructg the rough set rules Leveberg-Marquart algorthm (BPLM. For VPRS, we select the 0, 0.3 ad 0.45 respectvely as sesg mage classfcato. Hgher dmesos of put formato wll lead to complex etwork ISSN: Issue 5, Volume 7, May 008

6 Hag Xao ad Xub Zhag structure ad log tme trag. I VPRS, short tme are cosumed o costructg the rough set rules. Good results are obtaed the trag samples. However, the precso of classfcato to test data are ot deal whch also accout for the lmtato of rough sets. We fd that although performace of system s mproved to some extet through chagg from 0 to 0.45, the capacty of recogto ca ot be mproved essetally because of the defect rough sets. From the sythetc performace of the three methods, we fd the SVM get the best results. I SVM, except for the sgmod kerel, all the kerel fuctos obta a better result. It spet shorter tme SVM tha rough sets. The precso s hgh ad t s eve hgher tha eural etworks whe usg sple kerel. All of them show a brght future of SVM o remote sesg mage classfcato. 4 Aalyss I the paper, we apply eural etwork, varable precso rough sets ad support vector mache to remote sesg mage classfcato respectvely. Advatages ad dsadvatages of the three methods are aalyzed ad summarzed detaled as belows. Precso of classfcato s hgh usg eural etwork. However, the trag tme s log ad t probably exsts local mma. For mult-put system lke remote sesg mage, put vectors ca be lesse ad etwork structures be smplfed by usg rough sets. Whereas, eural etworks structures wll become very complex whe creasg the put dmesos. The meag of rough sets etwork s very clear oce the rough logcs are determed, whle the umber of hdde layer ad euros eural etworks are determed by experece. I VPRS, some ecessary redudace are remaed the process of formato reducto whch crease the at-ose performace of system ad decrease the loss of the useful formato. Furthermore, t provdes us a varable parameter for us accordg to our requremet. Although VPRS obta more mprovemet tha tradtoal rough sets remote sesg mage classfcato, t has also some dsadvatages. Frst, the msclassfcato error s ot cotuous, that s to say, the precso of VPRS s ot cotuous. Secod, selecto of optmal attrbutes stll reles o the specfc costrats. I the paper, we oly smply select them accordg to [6]. I the specfc codto, t does ot mea that s larger ad structure of etwork s more complex. Ths s related to the umber of attrbute ad logc selected. We ca ot try to mprove the redudace through smply creasg. Furthermore, besdes remote sesg mage classfcato, the umbers of equvalet classes also restrct the system performace. The umber s more, the precso s hgher, but the complexty also become hgher ad trag tme become loger. The problem also eeds to be dscussed the future research. I the expermet, we fd that hgh precso ad short trag tme ca be obtaed usg SVM. Comparg wth eural etwork, SVM s more sutable for processg the complex ad hgh dmesoal data. However, there are stll may problems to be solved SVM. The performace of SVM largely depeds o the kerel. Selecto of kerel fucto lmts the applcato of SVM greatly. From the expermet secto III, the result s ot deal whe Sgmod kerel are used ad trag tme s loger whe use RBF kerel although other results are very good. Now, the research of kerel s stll at ts begg stage [13]. The classfcato usg SVM stll focus o the two classes. How to process the problem of multclasses classfcato eeds to be studed the future. 5 Cocluso Neural etwork, rough sets ad support vector mache are three attractve ad effectve data mg methods recet years o dealg wth varous mprecse ad comeplete data. However, there exsts essetal dfferece amog them. Except for eural etwork, utl ow rough sets ad support vector mache are seldom used the research of remote sesg mage classfcato. I the paper, detal aalyss are descrbed whch provdes us a ew vewpot o remote sesg mage classfcato. From the expermet, we fd etwork, rough sets ad SVM all have ther advatages ad dsadvatages remote sesg mage classfcato. Therefore, how to combe the three theores ad apply them to remote sesg mage classfcato better s a mportat tedecy the later research. Ths prmary study ca be foud the lterature [14]. Refereces: [1] Heerma P D., Khazee N, Classfcato of multspectral remote sesg data usg a back propagato eural etwork, IEEE Tras o Geoscece ad Remote Sesg, Vol.30, No.1, 199, pp ISSN: Issue 5, Volume 7, May 008

7 Hag Xao ad Xub Zhag [] Atkso P M, Tatall A R L, Neural etworks remote sesg, It J Remote Sesg,Vol.8, No.4, 1997, pp [3] Solares. Crsta, Saz. Aa Mara, Bayesa etwork classfers. A applcato to remote sesg mage classfcato, WSEAS Trasactos o Systems, Vol. 4, No. 4, Aprl, 005, pp [4] Pal S K, Mtra P, Multspectral mage segmetato usg the rough set talzed EM algorthm, IEEE Trasactos o Geoscece ad Remote Sesg, Vol.40, No.11, 00, pp [5] Mak B, Muakata T, Rule extracto from expert heurstcs: A comparatve study of rough sets wth eural etworks ad ID3, Europea Joural of Operatoal Research, Vol.136, No.1, 00, pp [6] Zarko W, Varable Precso Rough Set Model, Joural of Computer ad System Sceces, 1993(46, pp [7] Azm-Sadad M R., Zekavat S A, Cloud classfcato usg support vector maches, IEEE 000 Iteratoal Proceedgs o Geoscece ad Remote Sesg Symposum, 000(, pp [8] Melga F, Bruzzoe L, Support vector maches for classfcato of hyperspectral remote-sesg mages, IEEE Iteratoal o Geoscece ad Remote Sesg Symposum, 00(1, pp [9] Wag W Y, L Y H, Evolutoary Learg of BMF Fuzzy-Neural Networks Usg a Reduced-Form Geetc Algorthm, IEEE Trasactos o Systems, Ma ad Cyberetcs - Part B: Cyberetcs, 003, pp [10] Stasty J, Skorpl V, Neural etworks learg methods comparso, WSEAS Trasactos o Crcuts ad Systems, Vol. 4, No. 4, Aprl, 005, pp [11] Paares G, Support vector maches for shade detfcato urba areas, WSEAS Trasactos o Iformato Scece ad Applcatos, Vol., No. 1, Jauary, 005, pp [1] Fsher P F, Vsualzato of the Re Lablty classfed Remotely Sesed Images, Photogrammetrc Egeerg ad Remote Sesg, 1994(7, pp [13] Burges C J C, Buldg Locally Ivarat Kerels, I: Proceedgs of the 1997 NIPS Workshop o Support Vector Maches, [14] Wu Zhaocog, Research o remote sesg mage classfcato usg eural etwork based o rough sets, 001 Iteratoal Coferece o Ifo-Tech ad Ifo-Net - ICII 001, 001(1, pp ISSN: Issue 5, Volume 7, May 008