Robust Proper Clustering Structure Fuzzy Modeling for Function Approximation

Size: px

Start display at page:

Download "Robust Proper Clustering Structure Fuzzy Modeling for Function Approximation"

Gerald Edwards
5 years ago
Views:

1 Robust Pope lusteng Stuctue Fuzzy Modelng fo Functon Appoxmaton hh-hng Hsao Depatment of Electcal Engneeng Kao Yuan Unvesty Kaohsung ounty,tawan RO Shun-Feng Su Depatment of Electcal Engneeng atonal Tawan Unvesty of Scence and Technology Tape, Tawan RO hen-ha huang Depatment of Electcal Engneeng atonal Ilan Unvesty I-lan, Tawan RO Abstact Tadtonal appoaches fo modelng TSK fuzzy ules ae tyng to adjust the paametes n models, and not consdeng the tanng data dstbuton Hence t wll esult n an mpope clusteng stuctue, especally, when outles exst In ths pape, a clusteng algothm temed as Robust Pope Stuctue Fuzzy Regesson Algothm (RPSFR) s poposed to defne fuzzy subspaces n a fuzzy egesson manne and also data clusteng wth obust capablty aganst outles I ITRODUTIO The Takag Sugeno Kang (TSK) type of fuzzy models has attacted a geat attenton of the fuzzy modelng communty due to ts good pefomance n vaous applcatons Fuzzy models can be constucted fom gven nonlnea system equatons by the dea of local appoxmaton [,] Such a way s smla to local lneazaton technque, so each fuzzy subspace pesent the ndvdual dstbuton state, that s such knd behavo vey popely descbe the chaactestc of data clusteng Ths stuctue of fuzzy model s called the pope clusteng stuctue Anothe way of constuctng fuzzy models s to dectly use nput-output tanng data Tadtonal leanng algothms [3-] fo ths knd of appoaches ae to tune system paametes accodng to modelng eo, and wthout consdeng the tanng data dstbuton Thus, the stuctue of the fuzzy model, afte leanng, especally when the tanng data wth outles, wll make ts specfc fuzzy subspaces fom the dsodely dstbuton state Such knd stuctue of fuzzy model usually wll be mpope clusteng stuctue As tanng data nsuffcent o not dstbute evenly, the fuzzy model of ths knd of tanng esult s unable to descbe the pototype system pecsely Thus, n ths pape, we popose a novel algothm fo obust fuzzy modelng, called Robust Pope Stuctue Fuzzy Regesson Algothm (RPSFR), that defne fuzzy subspaces n a fuzzy egesson manne and also data clusteng wth obust capablty aganst outles Ths pape ntegates the modelng eo and data cluste n a novel fuzzy modelng appoach The ognal RA clusteng algothm [3,4] adopts the dea of obust statstcs to educe the effects of outles and the concept of compettve agglomeaton to detemne the pope numbe of clustes The RFRA clusteng algothm [5] s modfed fom RA, Instead of usng clusteng concept n pattonng the fuzzy subspaces, t attempted to fnd fuzzy egessons fo fuzzy ules It s not consdeed the data dstbuton and usually, the esultant fuzzy patton may be mpope clusteng stuctue The poposed appoach s ntegated these two compettve agglomeaton pocesses Fstly, t called data clusteng pocess, that s, the pototype paametes ae the centes and vaances of clustes ext, called egesson clusteng pocess, the pototype paametes ae the paamete vecto n lnea egesson Thus, the Robust Pope Stuctue Fuzzy Regesson Algothm (RPSFR) can not only obtanng the TSK fuzzy model that aganst the outle, and also wth pope clusteng stuctue Moeove, the accuacy of the TSK modelng s also to mpove If moe pecson s equed, t also adopts a so-called Annealng Robust Backpopagaton (ARBP) [6] leanng algothm to fnetunng The emanng pat of the pape s outlned as follows Secton descbes the vaous clusteng algothms fo TSK fuzzy modelng In Secton 3, the RPSFR algothm s poposed to meanngfully defne a pope stuctue TSK fuzzy model Smulaton esults ae pesented n Secton 4 oncludng emaks ae pesented n secton 5 II TRADITIOAL LUSTERIG ALGORITHMS The consdeed poblem s to obtan a model fˆ fom a set of obsevatons, { x(), y ),( x(), y ),, ( x( ), y )} wth ( Ths wok was suppoted by atonal Scence ouncl of Tawan unde Gant S 94-3-E

2 x ( ) R n and y R, whee s the numbe of defned by clusteng the nput poton of the tanng data The ntal consequent s a sngleton and added the nput tanng data, x( ) = [ x( ), x ( ), L, ( )] s the -th nput vecto, and y s the desed output fo the nput x vaables fomed as lnea egesson by degee measue () Ths stuctue s dynamcal gowng netwok Moeove, the model s adjusted by supevsed leanng algothms to Suppose that those obsevatons ae obtaned fom an mpove the modelng accuacy Outles also easly affect t unknown functon y = f ( x, x, L, ) Ideally, we (3) ompettve Agglomeaton Type: Ths knd algothm stats by pattonng the data set nto a lage want to constuct an fˆ that can accuately epesent f n numbe of small clustes whch educes ts senstvty to tem of nput-output elatonshps A TSK fuzzy model ntalzaton As the algothm pogesses, adjacent clustes conssts of IF-THE ules that have the fom compete fo ponts, and clustes that lose the competton R : If x s A ( θ ) and x s A ( θ ),, s An ( θ n ) gadually vansh Thus, t can detemne the numbe of, () then h = f ( x, x,, ; a ) = a0 + a x + + an fo =,,,, whee s the numbe of ules, Al( θl ) s the fuzzy set of the -th ule fo x l wth the adjustable paamete set θ l, and a = ( a 0,, an ) s the paamete set n the consequent pat The pedcted output of the fuzzy model s nfeed as of the -th ule, h w = yˆ =, whee h s the output w = w = mn A θ ; x s the -th ule s ( ) l l l l=,, n fng stength, whch s obtaned as the mnmum of the fuzzy membeshp degees of all fuzzy vaables In Eq (), both the paametes of the pemse pats (e θ ) and of a ) of a TSK fuzzy model ae consequent pats (e equed to be dentfed Moeove, the numbe of ules must be specfed Thee ae thee knds of TSK fuzzy modelng fom tanng data fequently used n the lteatue ()Fxed clustes type: In ths knd appoach, the cluste numbe must be defned n advance Fuzzy - Regesson Model (FRM) clusteng algothm [5] s to fnd a set of tanng data whose nput-output elatonshp s somehow lnea, and then, those tanng data can be clusteed nto one fuzzy subspace Howeve, the obtaned fuzzy ules can be consdeed as a ough appoxmaton to the desed TSK fuzzy model The model s futhe adjusted by supevsed leanng algothms to mpove the modelng accuacy Ths stage s efeed to as the fne-tunng pocess Besdes, the FRM clusteng algothms s based on the pncple of least squae eo mnmzaton and s easly affected by outles, whch should be degaded n the clusteng pocess () Gowng lustes type: Ths knd clusteng s geneated a new ule o meged the ules by some degee measue fo each ncomng patten SOFI [6] s bascally a fuzzy modelng appoach beng equpped wth stuctue leanng capablty In the appoach, fuzzy subspaces ae l clustes va a pocess of compettve agglomeaton by tself [4] The RFRA clusteng algothm [6] s modfed fom RA, Instead of usng clusteng concept n pattonng the fuzzy subspaces, t attempted to fnd fuzzy egessons fo fuzzy ules Moeove, a obust leanng algothm s employed to efne the fuzzy model n the fne-tunng pocess Fo any eal-wold applcatons, the obtaned tanng data ae always subject to nose o maybe outles When nose becomes lage o outles exst, the fstly two knds modelng appoaches may ty to ft those mpope data n the tanng pocess and thus, the leaned systems ae coupted In othe wods, f thee exst outles and the tanng pocess lasts long enough, the obtaned systems may have ovefttng phenomena The compettve agglomeaton algothm ncludes the obust leanng algothms to ovecome ths poblem Howeve, all of those ae only consdeng the modelng eo, and then the esultng TSK fuzzy model n an mpope stuctue III RPSFR ALGORITHM FOR TSK MODELIG A novel appoach, temed as Robust Pope Stuctue Fuzzy Regesson (RPSFR) Algothm, s poposed In the RPSFR algothm, t s ntegated two compettve agglomeaton pocesses One s the data-clusteng pocess and the othe s egesson-clusteng pocess Fo the dataclusteng pocess, the RA algothm s appled and ts cost functon s defned as Jd = ujρ ( dj ) α wjuj, () = j= = j= = subject to, fo, whee and ae = u j j the numbes of fuzzy ules and of the tanng data, espectvely u j s the fng stength of the -th ule fo the j-th tanng patten, ρ ( ) s a obust loss functon assocated wth cluste, wj s the weght functon and

3 obtaned as w ρ ( d ) = d, and α s the paamete j j j usually called the agglomeaton paamete The Eucldean dstance measue s used, d j be the dstance between the j- th nput data and the cente of the -th cluste (ule) / n T dj =Σ ( x( j) θc )( Σ) ( x( j) θc ) () =,,, and j=,,, θ [ ] c = θc, L, θcn s the cente vecto of cluste Θ and Σ s ts dagonal covaance matx wth dagonal elements [ θv, L, θ ] v n To mnmze J d n (), the Lagange multple method s appled The Lagange functon s defned as L = u ( ) (3) jρ dj α wjuj λj uj = j= = j= j= = ext, the RFRA algothm s adjacent to do the egessonclusteng pocess The cost functon n () s eplaced j wth, and ewtten as J = uj ρ ( j ) α wjuj subject to = j= = j= uj =, fo j, all symbols ae smla to = pevously defnton j be the esdual between the j-th desed output of the modeled system and the output of the -th ule wth the j-th nput data; e, = y f x( j); a, (4) j j ( ) and the paamete vecto a fo the consequent pat of the - th ule s obtaned as T T a = X D X X D, =,,,, (5) [ ] Y ( n+) whee X R s matx wth as ts (k+)-th ow (entes n the fst ow of X ae all ), Y R s a vecto wth y as ts k-th element and D R s a dagonal k matx wth u k w k as ts k-th dagonal element wu k k k= = s called the obust cadnalty of cluste The obust cadnalty s a measue about whethe the consdeed cluste can be meged nto ts adjacent cluste; e the agglomeaton pocess When the obust cadnalty s less than a pe-specfed constant, cluste s ε dscaded Assume that Gaussan membeshp functons ae used n the pemse pats, (e, x k d j ( x ) l θcl Al ( θcl, θ) = exp ), whee θ cl and θ ae θ two adjustable paametes of the l-th membeshp functon of the -th fuzzy ules To combne those two pocesses, a good choce of these two paametes s a weghted aveage of them, that s, the new paametes ae u% = λu + ( λ) u (6) k k k w% k = λwk + ( λ) wk (7) whee λ [0,] s a weghted facto that s selected by desgne To get much accuacy, a easonable choce s λ > 05 Then, we have two update equatons as follows k= θ cl = k= θ = ( u ) k = k k l % w% x ( k), (8) % w% ( u ) k ( u% k ) w% k ( xj( k) θcl ) k= ( u ) k k % w% k (9) The poposed RPSFR Algothm s descbed n the followng [Step ]: Set ntal condtons, step-sze and the stop cteon [Step ]: ompute Eucldean dstance measue d j by usng () [Step 3]: Update the weghts w j, α () t and u j [Step 4]: ompute the consequent paamete sets a and j [Step 5]: Update the weghts w j α () t and u j [Step 6]: Update the cente θ cl and vaance θ [Step 7]: ompute the obust cadnalty = w u Decdng whethe cluste s dscaded o k k = k not [Step 8]: Update the tunng paamete [Step 9]: If the stop cteon s satsfed, then stop; othewse go to [Step ] And ts flow chat s descbed as Fg

4 algothms of constuctng TSK fuzzy models ae also mplemented n ou study One s the FRM clusteng algothm wth BP leanng algothm, Self-onstuctng eual Fuzzy Infeence etwok (SOFI) and the othe s the RFRA clusteng algothm wth obust BP leanng algothm These thee algothms ae selected fo compason because they ae all TSK fuzzy modelng appoaches and possess the stuctue leanng capablty, whch means the ules ae dynamcally geneated n the tanng pocess Afte tanng wth fne-tunng, the RPSFR s the best one n RMSE that can efe to Table I and Fg ~Fg5, and the stuctue s a pope clusteng stuctue, the othes ae not, as shown n Fg 6 and Fg7 TABLE I onlnea functon(5 ules) Algothm RMSE RPSFR 0446 RPSFR wth ARBP RFRA 030 RFRA wth ARBP FRM FRM wth BP 0358 SOFI SOFI wth BP 0985 Fg The poposed RPSFR algothm IV SIMULATIO EXAMPLE A nonlnea functon s defned as 5 y = ( + x + x ) x x 5 89 nput-output data ae used The goss eo model s used fo modelng outles The goss eo model s defned as F = ( ε ) G + εh (0) The values used n the goss eo model ae ε = 0 05, G ~ (0,005) and H ~ (0,) The numbe of the used test data s 089 They ae shown n Fg By the Unvesal appoxmaton, the moe ules the moe accuacy But the appoxmated output wll ovefttng afte fnetunng pocess, f the outle exsts Fo compason, thee Fgue The appoxmated esults by fou vaous algothms

5 Fg 3 The appoxmated esult by FRM and wth BP Fg 5 The appoxmated esults by RPSFR and wth ARBP Fg 4 The appoxmated esults of Example by SOFI and wth BP Fg6 The fng stength of each ule fo RFRA

6 Fg7 The fng stength of each ule fo RPSFR V OLUSIOS Vaous TSK modelng appoaches have been poposed n the lteatue Howeve, all of them only consde the modelng eo and esult an mpope clusteng stuctue In ths pape, a obust TSK fuzzy modelng appoach temed as Robust Pope Stuctue Fuzzy Regesson (RPSFR) Algothm s poposed It s poposed to smultaneously defne fuzzy subspaces and fnd the paametes n the consequent pats of TSK ules Ths clusteng algothm not only fnds egesson nstead of clusteng fo ules, but also has obust capablty aganst outles and wth a pope clusteng stuctue that s vey sutable to descbe a dynamc system The poposed obust TSK fuzzy modelng appoach s tested fo example and ndeed showed supeo pefomance n ou smulaton REFEREES [] H O Wang, K Tanaka, and M F Gffn, An appoach to fuzzy contol of nonlnea system: Stablty and desgn ssues, IEEE Tans Fuzzy Syst, vol 4, pp 4-3, 996 [] T Tanguch, K Tanaka, H Ohtake and H O Wang, Model constucton, ule educton, and obust compensaton fo genealzed fom of Takag-Sugeno fuzzy systems, IEEE Tan Fuzzy Syst, vol 9, pp , 00 [3] Koll, Identfcaton of functonal fuzzy models usng mult-dmensonal efeence fuzzy sets, Fuzzy Sets Syst, vol 80, pp 49 58, 996 [4] F Klawonn and R Kuse, onstuctng a fuzzy contolle fom data, Fuzzy Sets Syst, vol 85, pp 77 93, 997 [5] E Km, M Pak, S J and M Pak, A new appoach to fuzzy modelng, IEEE Tans Fuzzy Syst, vol 5, no 3, pp , 997 [6] F Juang, and T Ln, An On-lne Self- onstuctng eual Fuzzy Infeence etwok and Its Applcatons, IEEE Tans on Fuzz Systems, vol 6, no, pp-3, 998 [7] P P Angelov and D P Flev, An Appoach to Onlne Identfcaton of Takag-Sugeno Fuzzy Models, IEEE Tans Syst, Man, yben-pat B, vol 34, no, pp , 004 [8] I Rojas, H Pomaes, J Otega and A Peto, Self- Oganzed Fuzzy System Geneaton fom Tanng Examples, IEEE Tans on Fuzz Systems, vol 6, no, pp3-36, 000 [9] H Pomaes, I Rojas, J Otega and A Peto, A Systematc Appoach to a Self-Geneatng Fuzzy Rule- Table fo Functon Appoxmaton, IEEE Tans Syst, Man, yben-pat B, vol 30, no 3, pp43-44, 000 [0] S J Lee and S Quyang, A euo-fuzzy System Modelng Wth Self-onstuctng Rule Geneaton and Hybd SVD-Based Leanng, IEEE Tans on Fuzz Systems, vol, no 3, pp34-354, 003 [] W L Tung and Quek, Falcon: eual Fuzzy ontol and Decson Systems Usng FKP and PFKP lusteng Algothms, IEEE Tans Syst, Man, yben-pat B, vol 34, no, pp , 004 [] J González, I Rojas, H Pomaes, J Otega, and A Peto, A ew lusteng Technque fo Functon Appoxmaton, IEEE Tans eual etwoks, vol 3, no, pp 3-4, 00 [3] R Dave and R Kshnapuum, Robust clusteng methods: A unfed vew, IEEE Tans Fuzzy Systems, pp 70-93, 997 [4] H Fgu and R Kshnapuam, A obust compettve clusteng algothm wth applocatons n compute vson, IEEE Tans Patten Analyss and Machne Intellgence, vol, no 5, 999 [5] huang, S F Su and S S hen, Robust TSK Fuzzy Modelng fo Functon Appoxmaton wth Outles, IEEE Tans Fuzzy Systems, pp 80-8, vol 9, no 6, 00 [6] huang, S F Su, and Hsao, The Annealng Robust Backpopagaton (ARBP) Leanng Algothm, IEEE Tans On eual etwoks, vol, no 5, pp , 000

A SAS Macro for Finding Optimal k-means Clustering in One Dimension with Size Constraints

A SAS Macro for Finding Optimal k-means Clustering in One Dimension with Size Constraints Pape SD-02 A SAS Maco fo Fndng Optmal k-means Clusteng n One Dmenson wth Sze Constants Fengao Hu, Geoga Regents Unvesty; Robet E. Johnson, Vandeblt Unvesty ABSTRACT Wang and Song (2011) poposed a k-means