Constructive Semi-Supervised Classification Algorithm and Its Implement in Data Mining

Transcription

1 Costructve Sem-Supervsed Classfcato Algorthm ad Its Implemet Data Mg Arvd Sgh Chadel, Arua Twar, ad Naredra S. Chaudhar Departmet of Computer Egg. Shr GS Ist of Tech.& Sc. SGSITS, 3, Par Road, Idore (M.P.) Ida Departmet of Computer Scece ad Egeerg (CSE) IIT, Idore M-Bloc, IET-DAVV Campus, Khadwa Road, Idore-4507(M.P.) Abstract. I ths paper, we propose a ovel fast trag algorthm called Costructve Sem-Supervsed Classfcato Algorthm (CS-SCA) for eural etwor costructo based o the cocept of geometrcal expaso. Parameters are updated accordg to the geometrcal locato of the trag samples the put space, ad each sample the trag set s leared oly oce. It s a semsupervsed based approach, the trag samples are sem-labeled.e. for some samples, labels are ow ad for some samples, data labels are ot ow. The method starts wth clusterg, whch s doe by usg the cocept of geometrcal expaso. I clusterg process varous clusters are formed. The clusters are vsualzes terms of hyperspheres. Oce clusterg process over labelg of hyperspheres s doe, whch class s assged to each hypersphere for classfyg the mult-dmesoal data. Ths costructve learg avods bld selecto of eural etwor structure. The method proposes here s exhaustvely tested wth dfferet bechmar datasets ad t s foud that, o creasg value of trag parameters umber of hdde euros ad trag tme both are gettg decrease. Through our expermetal wor we coclude that CS-SCA result smple eural etwor structure by less trag tme. Keywords: Semsupervsed classfcato, Geometrcal Expaso, Bary Neural Networ, Hyperspheres. Itroducto Costructve learg begs wth a mmal or empty structure, ad dramatcally creases the etwor by addg hdde euros utl a satsfactory soluto s foud. Numbers of Costructve learg algorthms are avalable to overcome the problem of tradtoal algorthms for classfcato. It cludes Fast Coverg Learg Algorthm (FCLA) for Supervsed Learg [], Costructve Usupervsed Learg S. Chaudhury et al. (Eds.): PReMI 009, LNCS 5909, pp. 6 67, 009. Sprger-Verlag Berl Hedelberg 009

2 CS-SCA ad Its Implemet Data Mg 63 Algorthm (CULA) [], Costructve set Coverg learg algorthm (CSCLA) by Ma ad Yag [4], Boolea-le trag algorthm (BLTA) by Gray ad Mchel [5], expad ad trucate learg algorthm (ETL) by Km ad Par [3]. BLTA s a dyamc techque ad derves ts orgal prcple from Boolea algebra wth exteso. ETL fds a set of requred separatg hyperplaes ad automatcally determes a requred umber of euros the hdde layer based o geometrcal aalyss of the trag set. CSCLA was proposed based o the dea of weghted Hammg dstace hypersphere. I geeral, ETL, IETL, CSCLA, have o geeralzato capablty. BLTA has geeralzato capablty, but eeds more hdde euros. Moreover to t, FCLA s supervsed learg algorthm, thus labeled samples are used for learg but labeled data sample are expesve to obta as they requre the effort of expereced huma aotators. O the other had ulabeled data samples are easy to obta but there are very few way to process them. Thus approach of semsupervsed was used that uses large amout of ulabeled data sample wth small amout of labeled data sample to buld the classfer. I ths paper, we propose a ovel fast trag algorthm called Costructve semsupervsed classfcato Approach (CS-SCA) for eural etwor costructo. The proposed method s mplemeted usg two processes, frst s clusterg ad secod s labelg. We llustrate the advatages of CS-SCA by usg t classfcato problems. There are varous features of CS-SCA le t s a sem-supervsed costructve approach. Sample reorderg s allowed proposed classfer ad because of reorderg, learg s fast ths approach. As we ow that CS-SCA s a sem-supervsed approach that s why t requres less huma effort. Ths CS-SCA approach s tested wth umber of bechmar datasets ad compared wth SVM [6] based classfer. The paper s orgazed as follows. Secto gves a overvew of CS-SCA. Secto 3 explas the method for CS-SCA detal gve algorthmc formulato of our methodology. I secto 4, we gve expermetal results to demostrate the usefuless of our approach; t also cotas detal of data preparato. These expermetal results clude two well-ow datasets [7], amely, Rply dataset ad Wscos Breast cacer dataset. Fally, secto 5, we gve cocludg remars. Overvew of CS-SCA. Basc Cocept Boolea fuctos have the geometrcal property whch maes t possble to trasform o-lear represetato to lear represetato for each hdde euro. We cosder a Boolea fucto wth put ad oe output, y = f ( x, x,..., x ), y ad x ( 0,), = (... ). Where (0,) ( ca be cosdered as a These bary patters 0,) dmesoal ut hypercube.ths ex-hypersphere s defed as the referece hypersphere (RHS) [5] as follows: ( x / ) + ( x / ) ( x / ) = / 4. ()

3 64 A.S. Chadel, A. Twar, ad N.S. Chaudhar. Networ Costructo CS-SCA costructs a three-layered feed forward eural etwor wth a put layer, a hdde layer ad a output layer, as show Fg-. We llustrate the advatages of CS-SCA by ts mplemet classfcato problems. Fg.. Neural Networ Structure by CS-SCA 3 Proposed Method: CS-SCA CS-SCA begs wth a empty hdde layer. To lear a sample, CS-SCA ether adds a ew hdde euro to represet t or updates the parameters of some hdde euro by expadg ts correspodg hypersphere. Ths s doe by clusterg process ad oce clusterg gets over by usg the cocept of majorty votg labelg of hypersphere s doe. 3. Clusterg Process CS-SCA costructs a three-layered feed forward eural etwor, of whch frst layer represet to the put data sample that wll be the bary coded format. The put data samples wll be grouped to varous clusters. The mddle layer of etwor archtecture represets the hyperspheres(hdde euro).a hdde euro Fg. represets a correspodg hyper sphere wth ceter c ad radus r. Whle costructg a v hdde euro, suppose that { x, x,..., x } are v (true) sample cluded oe hyper sphere (hdde euro). I terms of these samples, the ceter s defed as the gravty ceter c = c, c,..., c ); ( c = v = The radus r s defed as the mmal Eucldea dstace such that all the v vertces are exactly or o the surface of the correspodg hyper sphere. v r m j = x v x j v = c = m j ( ( x / = c ) ) = Where s the dmeso of the put ad * s the eucldea dstace. Gve c ad r we ca separate these v true sample from the remag samples. I aother words, ths correspodg hypersphere represets these v true samples. ()

4 CS-SCA ad Its Implemet Data Mg 65 Two secodary cetral rad r ad r3 are troduced to fd compact cluster. Samples r < r <. should be a compact cluster where 3 CS-SCA begs wth a empty hdde layer. To costruct the eural etwor, we exame whether a comg "true" sample ca be covered by oe of the exstg hdde euros. Whe the frst sample x comes, the hdde layer s empty ad o hdde euro covers ths sample. A ew hdde euro, the frst hdde euro, s created to represet t. Ths ew created hdde euro represets a hyper sphere cetered at x. Samples, whch have bee represeted, are removed after parameter updatg. The trag process goes o. A comg sample x causes oe of the followg actos.. Update the parameters of some hdde euro, ad remove c;. Create a ew hdde euro to represet t, ad remove x ; 3. Bac up x to be leared the ext trag crcle. j j j j Gve a hdde euro j wth the ceter c ad three rad r, r ad r 3, we frstly compute the fucto for the hdde euro j defed as: r j j f ( w, x ) = w x (3) j j j j th Where w s ( w, w,..., w ), the weght vector ad x s the vertex. The trag process s cotued as follows: j I. If f ( w, x ) t j, already covered, so othg eeds to be doe. j j j II. If t > f ( w, x ) t the sample x s wth the "clam rego"; so to clude t a mmedate expaso s eeded. j III. If f ( w, x ) t j 3 the sample s cofusg sample so bac up = x to be dealt wth the ext trag crcle. j IV. If for all j s, f ( w, x ) < j t 3 the create a ew hdde euro ad remove the sample x from the trag set. Thus the umber of euros geerated s equal to the umber of clusters. After ths, the labeled samples are useful for labelg the clusters. The detals are gve ext. 3. Labelg Process I labelg process labels are assged to hyperspheres formed after the clusterg process by usg the mechasm of Majorty votg cocept. Thus these labeled hypersphere ca be represeted as output euro the output layer of etwor archtecture. After clusterg whe hyperspheres are detfed, we assg labels to hyperspheres.. Repeat the step ad step 3 for each of the hyper sphere.. Perform majorty votg by cout umber of samples belogs to oe partcular class.

5 66 A.S. Chadel, A. Twar, ad N.S. Chaudhar 3. Majorty of samples of partcular class a dvdual hypersphere would decde the class of that hypersphere. 4. If a partcular hypersphere s ot coverg ay labeled data that case merge ths hypersphere wth other whch s closure to t. 4 Expermetal Wor We used a Persoal Computer (PC) wth Petum processor wth.99 GHz speed ad GB of RAM havg wdows XP operatg system for testg. We used Matlab 7.0. for mplemetato. Table. Dataset Dmesos, Number of classes Trag Testg Fsher s Irs 4, Breast Cacer 9, Balace Scale 4, Rply, Each trag samples x = ( x, x,..., x ) are ormalzed as follows: x = x m( x ) / max( x ) m( x ) (4) After ths trasformato, each data sample s trasformed the rage 0 ad. CS- SCA requres bary form of put data therefore after ormalzato re-quatzes the data to eght levels as follows.. Apply each sample as a put quatzed fucto gve step 3.. Quatzed value ca be obtaed by: y = uecode ( u,, v) 3. Repeat step 3, tll the whole sample bary coded After data preparato, for expermetato 80% of the orgal data tae as trag data ad rest 0% cosdered as testg samples. The datasets used for expermetato are gve table. Results are evaluated terms of classfcato accuracy, trag tme, cofusg samples ad umber of hyperspheres requred. For dfferet value of trag parameter results for each dataset are gettg chage. After calculatg the performace of CS-SCA, same datasets are appled SVM based classfer [6], to compare the performace of both the classfers, terms of Classfcato accuracy, Trag tme. I SVM based classfer, trag parameterα used clusterg process. Number of support vector SVM based classfer depeds o the value of trag parameterα. Comparso results of both the classfer are dsplayed Table3.

6 CS-SCA ad Its Implemet Data Mg 67 Table. For 0-fold cross valdato results Dataset Average Accuracy Wscos 85. % Beast Cacer Rply 80. % Dataset Table 3. Comparso wth SVM Accuracy by CS- SCA Accuracy SVM Trag Tme CS-SCA Trag Tme SVM Fsher s Irs 9.59 % 77 % 0.96 sec sec. Balace Scale Wscos breast cacer Rply 80.6 % 77 %.8 sec sec. 85 % 80. % 70 % 75 % 4.9 sec sec. 086 sec. 36. sec. We gve results for 0-fold cross valdato o Wscos Breast Cacer ad Rply dataset table show above. For 0-fold cross valdato 90% of the data tae as trag ad rest 0% tae as testg data. From the results show above table3, t s clear that for each dataset CS-SCA s gvg better accuracy ad requres less trag tme compare to SVM based classfer. 5 Cocludg Remars A bary eural etwor based Sem-supervsed classfer s costructed usg the cocept of geometrcal expaso, whch classfy sem-labeled data. The classfcato s performed usg two processes, frst s clusterg ad secod s labelg. Varous bechmar datasets used to demostrate the performace of CS-SCA terms of accuracy ad umber of hypersphere etc. After that same datasets s appled SVM based classfer to compare ts performace wth developed classfer. It s foud that CS-SCA gves better performace terms of accuracy, trag tme etc. Refereces. Wag, D., Chaudhar, N.S.: A Costructve Usupervsed Learg Algorthm for Clusterg Bary Patters. I: Proceedgs of Iteratoal Jot Coferece o Neural Networs (IJCNN 004), Budapest, July 004, vol., pp (004). Wag, D., Chaudhar, N.S.: A Novel Trag Algorthm for Boolea Neural Networs Based o Mult-Level Geometrcal Expaso. Neurocomputg 57C, (004) 3. Km, J.H., Par, S.K.: The geometrcal learg of bary eural ewors. IEEE Trasacto. Neural Networs 6, (995) 4. Joo Er, M., Wu, S., Yag, G.: Dyamc Fuzzy Neural Networs. McGraw-Hll, New Yor (003) 5. Kwo, T.Y., Yeug, D.Y.: Costructve algorthms for structure learg feedforward eural etwors for regresso problems. IEEE Tras. Neural Networs 8, (997) 6. Chaudhar, N.S., Twar, A., Thomus, J.: Performace Evaluato of SVM Based Semsupervsed Classfcato Algorthm. I: Iteratoal Coferece o Cotrol, Automato, Robotcs ad Vso, Hao, Vetma, December 7-0 (008) 7.