FUZZY ARTMAP AND NEURAL NETWORK APPROACH TO ONLINE PROCESSING OF INPUTS WITH MISSING VALUES

Transcription

1 FUZZY ARTMAP AD EURAL ETWORK APPROACH TO OLIE PROCESSIG OF IPUTS WITH MISSIG VALUES F.V. elwamondo* and T. Mawala* * School of Electcal and Infomaton Engneeng, Unvesty of the Wtwatesand, Johannesbug, Pvate Bag 3, Wts, 2050, South Afca. Abstact: An ensemble based appoach fo dealng wth mssng data, wthout pedctng o mputng the mssng values s poposed. Ths technque s sutable fo onlne opeatons of neual netwoks and as a esult, s used fo onlne condton montong. The poposed technque s tested n both classfcaton and egesson poblems. An ensemble of Fuzzy-ARTMAPs s used fo classfcaton wheeas an ensemble of mult-laye peceptons s used fo the egesson poblem. Results obtaned usng ths ensemble-based technque ae compaed to those obtaned usng a combnaton of autoassocatve neual netwoks and genetc algothms and fndngs show that ths method can pefom up to 9% bette n egesson poblems. Anothe advantage of the poposed technque s that t elmnates the need fo fndng the best estmate of the data, and hence, saves tme. Key wods: Autoencode neual netwoks, Fuzzy-ARTMAP, Genetc algothms, Mssng data, MLP. ITRODUCTIO Real tme pocessng applcatons that ae hghly dependent on the newly avng data often suffe fom the poblem of mssng data. In cases whee decsons have to be made usng computatonal ntellgence technques, mssng data become a hndeng facto. The bggest challenge on one hand s that most computatonal ntellgence technques such as neual netwoks ae not able to pocess nput data wth mssng values and hence, cannot pefom classfcaton o egesson when some nput data ae mssng. Vaous heustcs fo mssng data have howeve been poposed n the lteatue []. The smplest method s known as lstwse deleton and ths method smply deletes nstances wth mssng values []. The majo dsadvantage of ths method s the damatc loss of nfomaton n data sets. Thee s also a well documented evdence showng that gnoance and deleton of cases wth mssng entes s not an effectve stategy [-2]. Othe common technques ae mputaton methods based on statstcal pocedues such as mean computaton, mputng the most domnant vaable n the database, hot deck mputaton and many moe. Some of the best mputaton technques nclude the Expectaton Maxmzaton (EM) algothm [3] as well as neual netwoks coupled wth optmsaton algothms such as genetc algothms as used n [4] and [5]. Imputaton technques whee mssng data ae eplaced by estmates ae nceasngly becomng popula. A geat deal of eseach has been done to fnd moe accuate ways of appoxmatng these estmates. Among othes, Abdella and Mawala [4] used neual netwoks togethe wth Genetc Algothms (GA) to appoxmate mssng data. Gabys [6] has also used euo-fuzzy technques n the pesence of mssng data fo patten ecognton poblems. The othe challenge n ths wok s that, onlne condton montong uses tme sees data and thee s often a lmted tme between the eadngs dependng on how fequently the senso s sampled. In classfcaton and egesson tasks, all decsons concenng how to poceed must be taken dung ths fnte tme peod. Methods usng optmsaton technques may take longe peods to convege to a elable estmate and ths depends entely on the complexty of the objectve functon beng optmsed. Ths calls fo bette technques to deal wth ths mssng data poblem. We ague n ths pape that t s not always necessay to have the actual mssng data pedcted. Dffeently sad, t s not n all cases that the decson s dependent on all actual values. Theefoe, a vast amount of computatonal esouces s wasted n attempts to pedct the mssng values, wheeas the ultmate esult could have been acheved wthout such values. In lght of ths challenge, ths pape nvestgates a poblem of condton montong whee computatonal ntellgence technques ae used to classfy and egess n the pesence of mssng data wthout the actual pedcton of mssng values. A novel appoach whee no attempt s made to ecove the mssng values, fo both egesson and classfcaton poblems, s pesented. An ensemble of fuzzy-artmap classfes to classfy n the pesence of mssng data s poposed. The algothm s futhe extended to a egesson applcaton whee Mult-laye Pecepton (MLP) s used n an attempt to get the coect output wth lmted nput vaables. The poposed method s

2 compaed to a technque that combnes neual netwoks wth Genetc Algothm (GA) to appoxmate the mssng data. 2. MISSIG DATA THEORY Accodng to Lttle and Rubn [], mssng data ae categozed nto thee basc types namely: Mssng at Random, (MAR), Mssng Completely at Random, (MCAR) and Mssng ot at Random, (MAR). MAR s also known as the gnoable case [3]. The pobablty of datum d fom a senso S to be mssng at andom s dependent on othe measued vaables fom othe sensos. A smple example of MAR s when senso T s only ead f senso S eadng s above a cetan theshold. In ths case, f the value ead fom senso S s below the theshold, thee wll be no need to ead senso T and hence, eadngs fom T wll be declaed mssng at andom. MCAR on the othe hand efes to a condton whee the pobablty of S values mssng s ndependent of any obseved data. In ths egad, the mssng value s nethe dependent on the pevous state of the senso no any eadng fom any othe senso. Lastly, MAR occus when data s nethe MAR no MCAR and s also efeed to as the non-gnoable case [, 3] as the mssng obsevaton s dependent on the outcome of nteest. A detaled descpton of mssng data theoy can be found n [3]. In ths pape, we shall assume that data s MAR. dstngush two pmay featues of an autoencode netwok, namely the auto-assocatve natue of the netwok and the pesence of a bottleneck that occus n the hdden layes of the netwok, esultng nto a buttefly-lke stuctue. In cases whee t s necessay to ecall the nput, autoencodes ae pefeed due to the emakable ablty to lean cetan lnea and non-lnea nteelatonshps such as coelaton and covaance nheent n the nput space. Autoencodes poject the nput onto some smalle set by ntensvely squashng t nto smalle detals. The optmal numbe of the hdden nodes of the autoencode, though dependent on the type of applcaton, must be smalle than that of the nput laye [8]. Autoencodes have been used n vaous applcatons ncludng the teatment of mssng data poblem by a numbe of eseaches ncludng [4] and [9]. In ths pape, auto-encodes ae constucted usng the MLP netwoks and taned usng back-popagaton. The stuctue of an autoencode constucted usng an MLP netwok s shown n Fgue. The fst step n appoxmatng the weght paametes of the model s fndng the appoxmate achtectue of the MLP, whee the achtectue s chaactezed by the numbe of hdden unts, the type of actvaton functon, as well as the numbe of nput and output vaables. The second step estmates the weght paametes usng the tanng set [7]. 3. BACKGROUD 3. eual netwok: mult-laye peceptons eual netwoks may be vewed as systems that lean the complex nput-output elatonshp fom any gven data. The tanng pocess of neual netwoks nvolves pesentng the netwok wth nputs and coespondng outputs and ths pocess s temed supevsed leanng. Thee ae vaous types of neual netwoks but we shall only dscuss the MLP snce they ae used n ths study. MLPs ae feed-fowad neual netwoks wth an achtectue compsng of the nput laye, hdden laye and the output laye. Each laye s fomed fom smalle unts known as neuons. euons eceve the nput sgnals x and popagate them fowad to the netwok and maps the complex elatonshp between nputs and the output. The fst step n appoxmatng the weght paametes of the model s fndng the appoxmate achtectue of the MLP, whee the achtectue s chaactezed by the numbe of hdden unts, the type of actvaton functon, as well as the numbe of nput and output vaables. The second step estmates the weght paametes usng the tanng set [7]. Tanng estmates the weght vecto W that ensues that the output s as close to the taget vecto as possble. Ths pape mplements the autoencode neual netwok as dscussed below. Autoencode neual netwoks: Autoencodes, also known as auto-assocatve neual netwoks, ae neual netwoks taned to ecall the nput space. Thompson et al [8] Fgue : The stuctue of a fou-nput fou-output autoencode Tanng estmates the weght vecto W to ensue that the output s as close to the taget vecto as possble. The poblem of dentfyng the weghts n the hdden layes s solved by maxmzng the pobablty of the weght paamete usng Bayes ule [8] as follows: Whee: p( W D) P( D W ) P( W ) P( D) = () D s the tanng data, P(W D) s the posteo pobablty, P(D W ) s called the lkelhood tem that balances between fttng the data well and helps n

3 avodng ovely complex models wheeas P(W ) s the po pobablty of W and P(D) s the evdence tem that nomalzes the posteo pobablty. The nput s tansfomed fom x to the mddle laye, a, usng weghts w j and bases b as follows [8]: a j = d = W j x + b j whee j = and j = 2 epesent the fst and second laye espectvely. The nput s futhe tansfomed usng the actvaton functon such as the hypebolc tangent (tanh) o the sgmod n the hdden laye. Moe nfomaton on neual netwoks can be found n [0]. 3.2 Genetc Algothms Genetc algothms use the concept of suvval of the fttest ove consecutve geneatons to solve optmsaton poblems []. As n bologcal evoluton, the ftness of each populaton membe n a geneaton s evaluated to detemne whethe t wll be used n the beedng of the next geneaton. In ceatng the next geneaton, the use of technques (such as nhetance, mutaton, natual selecton, and ecombnaton) common n the feld of evolutonay bology ae employed. The GA algothm mplemented n ths pape uses a populaton of stng chomosomes, whch epesent a pont n the seach space []. In ths pape, all GA paametes wee empcally detemned. GA s mplemented by followng thee man pocedues whch ae selecton, cossove and mutaton. The algothm lstng n Fgue 2 llustates how GA opeates. GA Algothm ). Ceate an ntal populaton P, begnnng at an ntal geneaton g = 0. 2). fo each populaton P, evaluate each populaton membe (chomosome) usng the defned ftness evaluaton functon possessng the knowledge of the competton envonment. 3). usng genetc opeatos such as nhetance, mutaton and cossove, alte P (g) to poduce P ( g +) fom the ft chomosomes n P (g). 4). epeat steps (2) and (3) fo the numbe of geneatons G equed. (2) 3.3 Fuzzy ARTMAP Fuzzy ARTMAP s a neual netwok achtectue developed by Capente et al [2] and s based on Adaptve Resonance Theoy (ART). The Fuzzy ARTMAP has been used n condton montong by Javadpou and Knapp [3], but the applcaton was not onlne. The Fuzzy ARTMAP achtectue s capable of fast, onlne, supevsed ncemental leanng, classfcaton and pedcton [2]. The fuzzy ARTMAP opeates by dvdng the nput space nto a numbe of hypeboxes, whch ae mapped to an output space. Instance based leanng s used, whee each ndvdual nput s mapped to a class label. Thee paametes namely the vglance ρ [0, ], the leanng ate β [0, ] and the choce paamete α, ae used to contol the leanng pocess. The choce paamete s geneally made small and a value of 0.0 was used n ths applcaton. The paamete β contols the adaptaton speed, whee 0 mples a slow speed and, the fastest. If β =, the hypeboxes get enlaged to nclude the pont epesented by the nput vecto. The vglance epesents the degee of belongng and t contols how lage any hypebox can become, esultng n new hypeboxes beng fomed. Lage values of ρ lead to a case whee smalle hypeboxes ae fomed and ths eventually lead to categoy polfeaton, whch can be vewed as ovetanng. A complete descpton of the Fuzzy ARTMAP s povded n [2]. In ths wok, Fuzzy ARTMAP s pefeed due to ts ncemental leanng ablty. As new data s sampled, thee wll be no need to etan the netwok as would be the case wth the MLP. 4. EURAL ETWORKS AD GEETIC ALGORITHM FOR MISSIG DATA The method used hee combnes the use of autoassocatve neual netwoks wth genetc algothms to appoxmate mssng data. Ths method has been used by Abdella and Mawala [4] to appoxmate mssng data n a database. A genetc algothm s used n ths wok to estmate the mssng values by optmsng an objectve functon as pesented shotly n ths secton. The complete vecto combnng the estmated and the obseved values s nput nto the autoencode as shown n Fgue 3. Symbols X k and X u epesent the known vaables and the unknown (o mssng) vaables espectvely. The combnaton of X k and X u epesent the full nput space. Fgue 2: Schematc epesentaton of the Genetc algothm opeaton

4 Fgue 3: Autoencode and GA Based mssng data estmato stuctue Consdeng that the method poposed hee uses an autoencode, one wll expect the nput to be vey smla to the output fo a well chosen achtectue of the autoencode. Ths s, howeve, only expected on a data set smla to the poblem space fom whch the ntecoelatons have been captued. The dffeence between the taget and the actual output s used as the eo and ths eo s defned as follows: = x f ( W, x) ε (3) whee x and W ae nput and weght vectos espectvely. To make sue the eo functon s always postve, the squae of the equaton s used. Ths leads to the followng equaton: ε = 2 ( x f ( W, x)) (4) Snce the nput vecto consst of both the known, X k and unknown, X u entes, the eo functon can be wtten as follows: X k X k ε =, f w (5) X u X u and ths equaton s used as the objectve functon that s mnmzed usng GA. 5. PROPOSED METHOD: ESEMBLE BASED TECHIQUE FOR MISSIG DATA The algothm poposed hee uses an ensemble of neual netwoks to pefom both classfcaton and egesson n the pesence of mssng data. Ensemble based appoaches have well been eseached and have been found to mpove classfcaton pefomances n vaous applcatons [4-5]. The potental of usng ensemble based appoach fo solvng the mssng data poblem emans unexploed n both classfcaton and egesson poblems. In the poposed method, batch tanng s pefomed wheeas testng s done onlne. Tanng s acheved usng a numbe of neual netwoks, each taned wth a dffeent combnaton of featues. Fo a condton 2 montong system that contans n sensos, the use has to state the value of n aval, whch s the numbe of featues most lkely to be avalable at any gven tme. Such nfomaton can be deduced fom the elablty of the sensos as specfed by manufactues. Senso manufactues often state specfcatons such as Meantme-between falues (MTBF) and Mean-tme-to-falue (MTTF) whch can help n detectng whch sensos ae most lkely to fal than othes. MTTF s used n cases whee a senso s eplaced afte a falue, wheeas MTBF denotes tme between falues whee the senso s epaed. Thee s nevetheless, no guaantee that falues wll follow manufactues specfcatons. When the numbe of sensos most lkely to be avalable has been detemned, the numbe of all possble netwoks can be calculated usng: n n n! = n( n = (6) aval n aval whee s the total numbe of all possble netwoks, n s the total numbe of featues and n aval s the numbe of featues most lkely to be avalable at any tme. Although the numbe n aval can be statstcally calculated, t has an effect on the numbe of netwoks that can be avalable. Let us consde a smple example whee the nput space has 5 featue, labelled : a, b, c, d and e and thee ae 3 featues that ae most lkely to be avalable at any tme. Usng equaton (6), vaable s found to be 0. These classfes wll be taned wth featues [abc, abd, abe, acd, ace, ade, bcd, bce, bde, cde]. In a case whee one vaable s mssng, say, a, only fou netwoks can be used fo testng, and these ae the classfes that do not use a n the tanng nput sequence. If we get a stuaton whee two vaables ae mssng, say a and b, we eman wth one classfe. As a esult, the numbe of classfes educes wth an ncease n a numbe of mssng nputs pe nstance. Each neual netwok s taned wth n aval featues. The valdaton pocess s then conducted and the outcome s used to decde on the combnaton scheme. The tanng pocess eques complete data to be avalable as tanng s done off-lne. The avalable data set s dvded nto the tanng set and the valdaton set. Each netwok ceated s tested on the valdaton set and s assgned a weght accodng to ts pefomance on the valdaton set. A dagammatc llustaton of the poposed ensemble appoach s pesented n Fgue 4. )!

5 usable egesson value of an nstance j. As a esult α. = We ty to solve ths by ecalculatng the weghts such that the sum of all weghts coespondng to usable s. Fgue 4: Dagammatc llustaton of the poposed ensemble based appoach fo mssng data Fo a classfcaton task, the weght s assgned usng the weghted majoty scheme gven by [6] as: E α = (7) = ( E j ) whee E s the estmate of model s eo on the valdaton set. Ths knd of weght assgnment has ts oots n what s called boostng and s based on the fact that a set of netwoks that poduces vayng esults can be combned to poduce bette esults than each ndvdual netwok n the ensemble [6]. The tanng algothm s pesented n Algothm and the paamete ntwk epesents the th neual netwok n the ensemble. The testng pocedue s dffeent fo classfcaton and egesson. In classfcaton, testng begns by selectng an elte classfe. Ths s chosen to be the classfe wth the best classfcaton ate on the valdaton set. To ths elte classfe, two moe classfes ae gadually added, ensung that an odd numbe s mantaned. Weghted majoty votng s used at each nstance untl the pefomance does not mpove o untl all classfes ae utlsed. In a case of egesson, all netwoks ae used all at once and the pedctons, togethe wth the weghts ae used to compute the fnal value. The fnal pedcted value s computed as follows: f ( x) = y α f ( x) (8) = whee α s the weght assgned dung the valdaton stage when no data wee mssng and s the total numbe of egessos. The paamete α s assgned such that α =. Consdeng that not all netwoks shall = be avalable dung testng, we defne usable as the numbe of egessos that ae usable n obtanng the 6. EXPERIMETAL RESULTS AD DISCUSSIO Ths secton pesents the esults obtaned n the expements conducted usng the two technques pesented above. Fstly, the esults of the poposed technque n a classfcaton poblem wll be pesented and late the method wll be tested n a egesson poblem. In both cases, the esults ae compaed to those obtaned afte mputng the mssng values usng the neual netwok-genetc algothm combnaton as dscussed above.

6 6. Applcaton to classfcaton Data set: The expement was pefomed usng the Dssolved Gas Analyss (DGA) data obtaned fom a tansfome bushng opeatng on-ste. The data consst of 0 featues, whch ae the gases that dssolved n the ol. The hypothess n ths expement s to detemne f the bushng condton (faulty o healthy) can be detemned whle some of the data ae mssng. The data was dvded nto the tanng set and the valdaton, each contanng 2000 nstances. Expemental setup: The classfcaton test was mplemented usng an ensemble of Fuzzy-ARTMAP netwoks. Two nputs wee consdeed moe lkely to be mssng and as a esult, 8 wee consdeed most lkely to be avalable. The onlne pocess was smulated whee data s sampled one nstance at a tme fo testng. The netwok paametes wee empcal detemned and the vglance paamete of 0.75 was used fo the Fuzzy- ARTMAP. The esults obtaned wee compaed to those obtaned usng the the -GA appoach, whee fo the GA, the cossove ate of 0. was used ove 25 geneatons, each wth a populaton sze of 20. All these paametes wee empcally detemned. Results: Usng equaton (6), a total of 45 netwoks was found to be the maxmum possble. The pefomance was calculated only afte 4000 cases have been evaluated and s shown n Fgue 5. The classfcaton nceases wth an ncease n the numbe of classfes used. Although all these classfes wee not taned wth all the nputs, the combnaton seems to wok bette than one netwok. The classfcaton accuacy obtaned unde mssng data goes as hgh as 98.2% whch compaes vey closely to a 00 % whch s obtanable when no data s mssng. Table : Compason between the poposed method and the -GA appoach Poposed -GA Algothm umbe of mssng 2 2 Accuacy (%) Run tme (s) The esults pesented n Table clealy show that the poposed algothms can be used as a means of solvng the mssng data poblem. The poposed algothm compaes vey well to the well know -GA appoach. The un tme fo testng the pefomance of the method vaes consdeably. It can be noted fom the table that fo the -GA method, un tme ncease wth nceasng numbe of mssng vaables pe nstance. Contay to the -GA, ou poposed method offes un tmes that decease wth nceasng numbe of nputs. The eason fo ths s that the numbe of Fuzzy-ARTMAP netwoks avalable educes wth an nceasng numbe of nputs as mentoned eale. Howeve, ths mpovement n speed comes at a cost of the dvesty. We tend to have less dvesty as the numbe of tanng nputs ncease. Futhemoe, ths method wll completely come to a falue n a case whee moe than n avl nputs wll be mssng at the same tme. 6. Applcaton to egesson In ths secton, we extend the algothm mplemented n the above secton to a egesson poblem. Instead of usng an ensemble of Fuzzy ARTMAP netwoks as n classfcaton, MLP netwoks ae used. The easons fo ths pactce ae two fold; fstly because MPL s ae excellent egessos and secondly, to show that the poposed algothm can be used wth any achtectue of neual netwoks. Database: The data fom a model of a Steam Geneato at Abbott Powe Plant [7] was used fo ths task. Ths data has fou nputs, whch ae the fuel, a, efeence level and the dstubance whch s defned by the load level. Thee ae two outputs whch we shall ty to pedct usng the poposed appoach n the pesence of mssng data. These outputs ae dum pessue and the steam flow. Fgue 5: Dagammatc llustaton of the poposed ensemble based appoach fo mssng data Usng the -GA appoach, a classfcaton of 96% was obtaned. Results ae tabulated n Table below. Expemental setup: Although Fuzzy-ARTMAP could not be used fo egesson, we extended the same appoach poposed above usng MLP neual netwoks fo egesson poblem. As befoe, ths wok egesses n ode to obtan two outputs whch ae the dum pessue and the steam flow. We assume n avl = 2 s the case and as a esult, only two nputs can be used. We ceate an ensemble of MLP netwoks, each wth fve hdden nodes and taned only usng two of the nputs to obtan the output. Due to lmted featues n the data set, ths wok

7 shall only consde a maxmum of one senso falue pe nstance. Each netwok was taned wth 200 tanng cycles usng the scaled conjugate gadent algothm and a hypebolc tangent actvaton functon. All these tanng paametes wee agan empcally detemned. Results: Snce testng s done onlne whee one nput aves at a tme, evaluaton of pefomance at each nstance would not gve a geneal vew of how the algothm woks. The wok theefoe evaluates the geneal pefomance usng the followng fomula only afte nstances have been pedcted. n Eo = τ 00% whee n τ s the numbe of pedctons wthn a cetan toleance. In ths pape, a toleance of 20% s used and was abtaly chosen. Results ae summazed n Table 2 Table 2: Regesson accuacy obtaned wthout estmatng the mssng values. Poposed -GA Algothm umbe of mssng 2 2 Pef (%) Tme Pef (%) Tme Dum Pessue Steam Flow Pef ndcates the accuacy n pecentage wheeas tme ndcates the unnng tme n seconds. Results show that the poposed method s well suted fo the poblem unde nvestgaton. The poposed method pefoms bette than the combnaton of GA and autoencode neual netwoks n the egesson poblem unde nvestgaton. The eason s that the eos that ae made when nputtng the mssng data n the -GA appoach ae futhe popagated to the output-pedcton stage. The ensemble based appoach poposed hee does not suffe fom ths poblem as thee s no attempt to appoxmate the mssng vaables. It can also be obseved that the ensemble based appoach takes less tme that the -GA method. The eason fo ths s that GA may take longe tmes to convege to elable estmates of the mssng values dependng on the objectve functon to be optmsed. Although, the pedcton tmes ae neglgbly small, an ensemble based technque takes moe tme to tan snce tanng nvolves a lot of netwoks. 7. COCLUSIO In ths pape a new technques fo dealng wth mssng data fo onlne condton montong poblem was pesented and studed. Fstly the poblem of classfyng n the pesence of mssng data was addessed, whee no attempts ae made to ecove the mssng values. The poblem doman was then extended to egesson. The (9) poposed technque pefoms bette than the -GA appoach, both n accuacy and tme effcency dung testng. The advantage of the poposed technque s that t elmnates the need fo fndng the best estmate of the data, and hence, saves tme. Futue wok wll exploe the ncemental leanng ablty of the Fuzzy ARTMAP n the poposed algothm. Acknowledgements The fnancal assstance of the atonal Reseach Foundaton (RF) of South Afca and the Cal and Emly Fuchs Foundaton s heeby acknowledged. 8. REFERECES [] R. J. A. Lttle and D. B. Rubn, Statstcal analyss wth mssng data. ew Yok: John Wley, 987. [2] J. Km and J. Cuy, The teatment of mssng data n multvaate analyss, Socologcal Methods and Reseach, vol. 6, pp , 977. [3] J. Schafe and J. Gaham, Mssng data: Ou vew of the state of the at, Psychologcal Methods, vol. 7, pp , [4] M. Abdella and T. Mawala, The use of genetc algothms and neual netwoks to appoxmate mssng data n database, Computng and Infomatcs, pp , [5] S. M. Dhlamn, F. V. elwamondo, and T. Mawala, Condton montong of hv bushngs n the pesence of mssng data usng evolutonay computng, WSEAS Tansactons on Powe Systems, vol., pp , [6] B. Gabys, euo-fuzzy appoach to pocessng nputs wth mssng values n patten ecognton poblems, Intenatonal Jounal of Appoxmate Reasonng, vol. 30, pp , [7]. Japkowcz, Supevsed leanng wth unsupevsed output sepaaton, In Intenatonal Confeence on Atfcal Intellgence and Soft Computng, pp , [8] B. B. Thompson, R. Maks, and M. A. El-Shakaw, On the contactve natue of autoencodes: Applcaton to senso estoaton, Poceedngs of the IEEE Intenatonal Jont Confeence on eual etwoks, pp , [9] A. Folov, A. Katashov, A. Goltsev, and R. Folk, Qualty and effcency of eteval fo wllshaw-lke autoassocatve netwoks, Computaton n eual Systems, vol. 6, 995. [0] C. M. Bshop, eual etwoks fo Patten Recognton. ew Yok: Oxfod Unvesty Pess, [] D. Goldbeg, Genetc Algothms n Seach, Optmzaton and Machne Leanng. Readng, MA: Addson-Wesley, 989. [2] G. Capente, S. Gossbeg,. Makuzon, J. Reynolds, and D. Rosen, Fuzzy ARTMAP: A neual netwok achtectue fo ncemental supevsed leanng of multdmensonal maps, IEEE Tansactons on eual etwoks, vol. 3, pp , 992. [3] R. Javadpou and G. Knapp, A fuzzy neual netwok appoach to condton montong, Socologcal Methods and Reseach, vol. 45, pp , [4] Y. Feund and R. Schape, A decson theoetc genealzaton of onlne leanng and an applcaton to boostng, n Poceedngs of the Second Euopean Confeence on Computatonal Leanng Theoy, pp , 995. [5] L. I. Kuncheva, Combnng Patten Classfes, Methods and Algothms. ew Yok: Wlley Intescence, [6] E. McGookn and D. Muay-Smth, Usng coespondence analyss to combne classfes, Machne Leanng, vol. 4, pp. 26, 997. [7] B. De Moo, Database fo the dentfcaton of systems, depatment of electcal engneeng, esat/ssta. Intenet Lstng, Last Acessed: 2 Apl URL: