Tunng of Fuzzy Inference Systems Through Unconstraned Optmzaton Technques ROGERIO ANDRADE FLAUZINO, IVAN NUNES DA SILVA Department of Electrcal Engneerng State Unversty of São Paulo UNESP CP 473, CEP 733-36, Bauru, São Paulo BRAZIL raflauzno@ladaps.feb.unesp.br, van@feb.unesp.br Abstract: - Ths paper presents a new methodology for the adjustment of fuzzy nference systems. A novel approach, whch uses unconstraned optmzaton technques, s developed n order to adjust the free parameters of the fuzzy nference system, such as ts own parameters of the membershp functon, and the weght of the nference rules. Ths methodology s nterestng, not only for the results presented and obtaned through computer smulatons, but also for ts generalty concernng to the knd of fuzzy nference system used. Therefore, ths methodology s expandable ether to the Mandan archtecture or also to that suggested by Takag-Sugeno. The valdaton of the presented methodology s accomplshed through estmaton of tme seres. More specfcally, the Mackey-Glass chaotc tme seres s used for the valdaton of the proposed methodology. Keywords: - Fuzzy systems, optmzaton problems, parameters estmaton, tme seres analyss, neural systems. Introducton The fuzzy nference systems desgn comes along wth several decsons taken by the desgners snce t s necessary to determne, n a coherent way, the number of membershp functons for the outputs and nputs, and also the specfcaton of the fuzzy rules sets of the system, besdes defnng the strateges of rules aggregaton and defuzzfcaton of output sets. The need to develop systematc procedures to help the desgners has been wde, snce very often the tral and error technque s the only one avalable []. At present tme, there are several researchers engaged n studes related to the desgn technques nvolvng fuzzy nference systems. A bref resume about the dfferent approaches for tunng of fuzzy nference systems may be found n []. Ths paper presents a methodology of tunng fuzzy nference systems based on unconstraned optmzaton technques. Ths methodology has the objectve of mnmzng an energy functon assocated to the fuzzy nference system. The defnton of the energy functon must obey the performance requrements of the fuzzy system. Ths way, the energy functon may be explcted as the mean squared error between the output of the fuzzy system and the desred results, whch are provded by the tunng set, smlar to the artfcal neural networks wth supervsed tranng. The energy functon may be also defned through the performance parameters desred to the fuzzy system behavor, as t happens n the adjustment of the process controllers actng n a determned plant. It may be observed that the correct defnton of energy functon s fundamental to the success of the desred adjustment [3]. Generally, n applcatons nvolvng the dentfcaton and fuzzy modelng, t s convenent to use energy functons that express the error between the desred results and the results provded by the fuzzy system. An example s the use of the mean squared error and normalzed mean squared error as energy functons. In the context of systems dentfcaton, besdes the mean squared error, data regularzaton ndcators may be added to the energy functon n order to mprove the system response n presence of noses (from tranng data) or when the tunng set has a constraned data quantty. In the absence of a tunng set, as t happens n parameters adjustment of a process controller, the energy functon can be defned through a functon that consders the desred requrements of a desgn [4]. The example of tunng a fuzzy controller, such requrements may be, for nstance, maxmum overshoot sgnal, settng tme, rse tme, or less usual ones, lke the undamped natural frequency of the system among others. Therefore, the defnton of the energy functon n the context of tunng fuzzy nference systems becomes part of ther specfcatons, as t occurs wth the nputs number, the membershp functons, and the
choce of the used technques n fuzzfcaton processes, rules nference and defuzzfcaton. Besdes addng a new task to the process of fuzzy nference system creaton, the defnton of the energy functon assocated to the system, jontly wth the technques underlned n ths paper, saves the desgners efforts n steps of tunng the membershp functons, ncludng the creaton of the fuzzy nference rules. Usng ths new approach from the defnton pont of vew, the fuzzy system becomes defned as a three layers model. Each one of these layers represents the tasks performed by the fuzzy nference system, such as fuzzfcaton tasks, fuzzy rules nference and defuzzfcaton. The fuzzy nference system adjustment proposed n ths paper s performed through the adaptaton of the free parameters from each one of these layers, wth the objectve of mnmzng the energy functon prevously defned. In prncple, the adjustment can be made layer by layer separately, nevertheless not preventng t to be made n all layers at each teraton. The operatonal dfferentaton of each layer, where the parameters adjustment of a layer doesn t nfluences the performance of the others, allows the ndvdual adjustment of the layers. Thus, the routne of fuzzy nference system tunng acqures a larger flexblty when compared to the tranng process used n artfcal neural networks. To valdate the proposed methodology, t s developed a fuzzy nference system to predct the Mackey-Glass tme seres. Due to ts chaotc nature, ths estmaton problem offers an adequate applcaton to valdaton of the hereby-underlned approach [5]. Ths artcle s organzed as follows. In Secton, a revew about fuzzy nference systems s presented to elaborate all needed consderatons to the tunng methodology comng on Secton 3. In Secton 4 s made a bref resume about the Glass-Glass chaotc tme seres. In Secton 5, the smulaton results are presented. Fnally, conclusons are descrbed n Secton 6. Fuzzy Inference Systems The fuzzy nference systems may be treated as systems that use the concepts and operatons defned by the fuzzy set theory, snce they use the fuzzy nference process to perform ther operatonal functons. Bascally, these operatonal functons nclude the nputs fuzzfcaton of the system, the nference rules assocated to t, the aggregaton of rules and the later defuzzfcaton of the aggregaton results, whch represent the outputs of the fuzzy system [6]. Ths way, t may be observed that the fuzzy nference systems have dfferent functons clearly defned, allowng those systems the functons nterpretaton through the representaton of a multlayer model. Consderng the operatonal functons performed by the fuzzy nference systems, t s convenent to represent them by a three layers model. Thus, a fuzzy nference system may be gven by the sequental composton of the nput layer, by the nference layer of the fuzzy rules and by the output layer. The nput layer has functonaltes of connectng the nput varables (comng n from outsde) wth the fuzzy nference system and also ther fuzzfcatons through respectve membershp functons. In the nference layer of the fuzzy rules or just nference layer, the nput fuzzfed varables are combned among them, accordng to defned rules, usng as support the operatons defned n the theory of fuzzy sets. The set resultng of the aggregaton process s then defuzzfed, resultng n the fuzzy nference system output. The aggregaton process and the defuzzfcaton process of the fuzzy set output are made by the output layer. It s mportant to observe, concernng to the output layer, that although t performs the two processes above descrbed, t s also responsble for storng the membershp functons of the output varables. In the followng sub-sectons further detals wll be presented about how fuzzy nference systems can be represented through a three layers model.. Input Layer As prevously presented, the fuzzy nference system nput has the purpose of connectng the nputs comng from the envronment wth the fuzzy system, as well as the fuzzfcaton of those accordng to the membershp functons assocated to the fuzzy system. The system nputs fuzzfcaton has the purpose of determnng the degree of each nput related to the fuzzy sets assocated to each nput varable. To each nput varable of the fuzzy system can be assocated as many fuzzy sets as necessary. Ths way, gven a fuzzy system wth one only nput and, to ths nput, assocated wth N functons, that s N fuzzy sets whch defne that nput, the output of the nput layer wll be a column vector wth N elements representng the degrees of the nput membershp n relaton to those fuzzy sets.
If we defne the nput of ths fuzzy system wth one only nput, by the scalng of x, then the nput layer output of the fuzzy system wll be the vector y, that s: y ( x ) = [ p ( x ) p ( x ) p ( )] T N x where (.) p s the defned membershp functon for k the x nput, referrng to the k -th fuzzy set assocated to ths nput. The generalzaton of the nput layer concept for a fuzzy system ownng m nput varables s acheved f each nput of ths fuzzy system s modeled as a sublayer of the nput layer. Thus, n equaton () x s the -th nput of the p k. s the k -th vector of the fuzzy system, ( ) membershp functons assocated to the x k nput and to the yk vector. Each sub-layer has ts own fuzzy sets defned by the fuzzy nference functons vector p k (.). The output vector of the nput layer Y(x) may be gven as presented n (), that s: Y ( x) y y = = ym p p p m ( x ) ( x ) ( x ) There are several membershp functons that can be defned. One of the necessary requstes for those functons s that they must be normalzed n closed doman [,]. However, t s convenent that the membershp functons are defned n a smple and convenent way, amng ther computatonal mplementaton, wth objectve of a hgher processng speed and ratonal use the memory. Besdes provdng benefts from the computatonal pont of vew, t s convenent that the membershp functons may have a reduced number of free parameters n order that the tunng algorthm performs the tunng task n an adequate way.. Inference Layer The nference layer of a fuzzy system has the functonalty of processng the fuzzy nference rules defned for t. Another functonalty of the nference layer s to provde a knowledge base for the process. The nference rules are processed n parallel, the same way as the sub-layers of the nput layers are. Insde ths context, ths set of rules has fundamental mportance to the correct functonng of the fuzzy nference system. There are several methods for the extracton of fuzzy rules from the tunng set. m () () In ths artcle, ntally, the fuzzy nference system has all the possble nferred rules. Therefore, the tunng algorthm has the task of weghtng the nference rules. Weghtng the nference rules s an adequate way to represent the most mportant rules n the fuzzy system, or even to allow that conflctng rules are related to each other wthout any verbal completeness loss. Thus, t s possble to express the -th fuzzy rule as n (3), that s: R ( Y ( x) ) = w r ( Y ( x) ) (3) where (.) R s the functon representng the fuzzy weght value of the -th fuzzy rule, w s the weght factor of the -th fuzzy rule and r (.) represents the fuzzy value of the -th fuzzy rule..3 Output layer The output layer of the fuzzy nference system ams to aggregate the nference rules, as well as the defuzzfcaton of the fuzzy set generated by the aggregaton of nference rules. In the fuzzy nference systems desgn, the choce of not only the aggregaton method but also the defuzzfcaton method consttutes a very mportant decson. The aggregaton method of the fuzzy nference rules must be n such a way that the fuzzy set resultng from aggregaton s capable of adequately representng the knowledge explcted by ths set of fuzzy rules. By analogy, the method chosen for the defuzzfcaton must be capable of expressng, n a crsp value, the fuzzy set resultng from the fuzzy aggregaton. Besdes the operatonal aspects, the aggregaton and defuzzfcaton methods must attend the requstes of computatonal performance n order to reduce the computatonal effort needed n the fuzzy system processng. In ths paper, the output layer of the nference system s also adjusted. The adjustment of ths layer occurs n a smlar way to what occurs wth the nput layer of the fuzzy system. 3 Adjustment of the Fuzzy Inference System The formalzaton of a fuzzy nference system n the form of a multlayer system, as presented n Secton, can be justfed not only by the dfferent operatonal dvson of each one of these layers, but
also by the presence n each of the dfferent free parameters. For example, let a fuzzy system wth two nputs, each one wth three gaussan membershp functons, wth a total of fve nference rules, and havng an output defned wth two gaussan membershp functons. It s known that, for each gaussan membershp functon, two free parameters exst: the mean and the standard devaton. Ths way, the number of free parameters of the nput layer wll be. For each nference rule has been assocated a weghtng factor, so, there are fve free parameters n the nference layer. In relaton the to output layer, the same consderatons made for the nput layer are vald. Therefore, four free parameters are n the output layer. Ths way, the mappng f between the nput space x and the output space y may be defned as n (4) y = f ( x, w, w, w3) (4) where w, w and w3 respectvely represent the vectors of the nput membershp functons parameters, the weght of the nference rules and the output membershp functons parameters. The defnton of the energy functon to be mnmzed remans n functon of the fuzzy mappng. Consderng that the tunng set { x, d} s fxed durng the whole adjustment process, t may be wrtten: ξ ( x, y) = ξ( x,y)( w,w,w3) (5) where ξ represents the energy functon assocated to the fuzzy nference system f. y = f, ( x w*,w*,w3* ) * * * where w, w and w3 are the free parameters values of the fuzzy nference systems after the adjustment process. 3. Unconstraned Optmzaton Technques ξ w, w,w3 n the Taken an energy functon ( )( ) form prevously defned for a fuzzy nference system, n the assumpton that a functon mght be dfferentable n relaton to a determned nterval by the w, w and w3 vectors, that s, dfferentable n relaton to free parameters of the fuzzy nference system. Therefore, t s desred to fnd an optmum soluton that may fulfll the followng condtons: x,y * * * ( w,w,w ) ξ ( w, w, ) ξ 3 w 3 Wth the fnalty of smplfyng the notaton, these three vectors may be represented by one unque vector, resultng from the vector concatenaton of w, w and w3, that s: w = T T T [ w w w3 ] T (6) (7) (8) Thus, the expresson (7) may be rewrtten by: ξ * ( w ) ξ ( w) Therefore, t may be observed that, to attend the condton expressed n (9), t s necessary to solve an unconstraned optmzaton problem. Then, the w vector may be obtaned by the followng equaton: w * = arg mn ξ w ( w) The condton that expresses the optmum soluton n () must attend to the followng soluton: ξ * ( w ) = where s the gradent operator, that s: ξ ξ ξ ξ ( w ) =,,, () w w w m In problems lke ths, nvolvng the mnmzaton of energy functons, t s desred that to each teraton, the energy functon value would be less than the energy functon value of the prevous teraton. There are several technques used n solvng the unconstraned optmzaton problems. A detaled descrpton of the unconstraned optmzaton technques may be found n [7]. The choce of the most adequate technque to be used s condtoned to the form by whch the energy functon s defned. For example, the Gauss-Newton method for the unconstraned optmzaton may be more applcable n problems where the energy functon s defned as: where ( ) ξ ( w) = e ( ) n = e s the absolute error n relaton to the -th tunng pattern. In ths paper, a dervaton of the Gauss-Newton method s used for the fuzzy nference system tunng. The optmzaton algorthm used was the Levenberg- Marquardt method [8]. The calculaton of dfferental equatons was performed wth the help of the fnte dfferences method. 4 Mackey-Glass Chaotc Tme Seres The research around chaotc seres created new paradgms about the exstent modelng technques. In ths way, the present research appeals to new fundaments for the seres predcton. On the other hand, the determnsm nherent to chaos mples that many phenomenons, formerly seen as random, may be treated n a predctve way. T (9) () () (3)
The predcton of the Markey-Glass chaotc tme seres s a classc estmaton problem. Generally, ths problem s used to test the generalzaton capacty of such as systems comng from the computatonal ntellgence, lke neural networks and fuzzy nference systems. The dynamc propertes of Mackey-Glass tme seres are rch n complexty. The Mackey-Glass dfferental equaton may be expressed by: dx( t) ax ( ) ( t τ ) = bx t + (4) dt + x t τ ( ) c The Mackey-Glass tme seres was one of the frst models for the tme quantzaton of producng whte cells n the human organsm [9]. In general, and n ths work too, the values of the constants n (4) are adopted as beng a =., b =. and c =. The value for the delay constant τ s chosen as beng 7. The Mackey-Glass tme seres may be obtaned by the ntegraton of the equaton n (4). More specfcally, t has been used the second order Runge- Kutta method wth ntegraton step equal to.. The result of ths ntegraton s shown n Fgure. ξ L ( w,w,w3) = [ x ( t + 6) x ( t + 6) ] = where L s the number of data used n the tunng process (L=5). After mnmzaton of (4), the membershp functons of the fuzzy nference system were adjusted as llustrated n Fgures from to 5. mf mf..4.6.8 x(t-8) Fg. - Input membershp functons to ( t 8) mf4 mf3 mf mf mf3 mf4 (4) Ampltude.6.4...4.6.8 x(t-) Fg. 3- Input membershp functons to ( t ) 4 6 8 Tme (seconds) Fg. - Mackey-Glass tme seres. mf mf mf3 mf4 5 Methodology and Results Usng the methodology presented n ths paper for the fuzzy nference systems tunng and, based on the Mackey-Glass tme seres, a fuzzy nference system of Mandan type was developed wth the objectve to predct ths seres. The tunng set was consttuted by 5 patterns. The nput varables of the fuzzy nference system were four, correspondng to values x(t 8), x(t ), x(t 6) and x(t). As an output varable was adopted x(t + 6). The fuzzy nference system was defned havng four fuzzy sets attrbuted to each nput varable and also to the output varable. A total of 64 nference rules have been used n the nference process. The energy functon of the system was defned as beng the mean squared error between the desred t + 6 x t + 6 values x ( ) and the values ( )..4.6.8 x(t-6) Fg. 4 - Input membershp functons to ( t 6) mf mf mf3 mf4..4.6.8 x(t) Fg. 5 - Input membershp functons to ( t)
In Fgure 6 s presented the output membershp functons of the fuzzy nference system. mf mf mf3 mf mf4..4.6.8 x(t+6) Fg. 6 - Output membershp functons ( t + 6) mf mf3 mf4 In Fgure 7 s presented the result of predcton provded by the fuzzy nference system for ponts. The mean squared error of estmaton for the proposed problem was.598 wth standard devaton of.448..4.3...9.7.5 real values estmated values 3 4 5 6 7 8 9 Fg. 7 - Estmaton of the fuzzy nference system for the Mackey-Glass seres. For comparson, t was developed a fuzzy nference system tunng wth ANFIS (Adaptve Neural-Fuzzy Inference System) method. Ths fuzzy nference system was made wth membershp functons for each nput, beng the knowledge base consttuted by rules. The mean squared error of estmaton for the proposed problem was.65 wth standard devaton of.4. 6 Conclusons In ths paper was underlned the basc foundatons around the fuzzy nference systems tunng process, from the unconstraned optmzaton technques. In order that the tunng may be effcent t s necessary that the energy functon s perfectly underlned for the adjustment process. For the valdaton of the proposed methodology, t was studed the estmaton of the Mackey-Glass chaotc tme seres. The comparson of the results was made wth those results provded from the ANFIS methodology. Although the results provded by the ANFIS methodology were better, the amount of fuzzy sets assocated to each nput was superor to the number of fuzzy sets used n the fuzzy system developed along ths work. Ths approach offers new perspectves of research related to the fuzzy nference systems, allowng thus that problems prevously treated only wth the help of artfcal neural networks may now be treated through fuzzy nference systems. As future works, t s ntended to develop effcent technques for the constructon of nference rules bases wth the objectve of optmzng ther structures n the whole. References: [] M. Fgueredo and F. Gomde, Adaptve Neuro Fuzzy Modelng, Proceedngs of the Sxth IEEE Internatonal Conference on Fuzzy Systems, Vol. 3, 997, pp. 567-57. [] S. Gullaume, Desgnng Fuzzy Inference Systems from Data: Na Interpretablty - Orented Revew, IEEE Trans. Fuzzy Systems, Vol. 9, No. 3,. [3] R. Kammura, T. Takag and S. Nakansh, Improvng Generalzaton Performance by Informaton Mnmzaton, IEEE World Congress on Computatonal Intellgence, Vol., 994, pp. 43-48. [4] S. Becker, Unsupervsed Learnng Procedures for Neural Networks, Internatonal Jornal of Neural Systems, Vol., 99, pp. 7-33. [5] W. Wan, K. Hrasawa, J. Hu and J. Murata, Relaton Between Weght Intalzaton of Neural Networks and Prunng Algorthms: Case Study on Mackey-Glass Tme Seres, Internatonal Jont Conference on Neural Networks, Vol. 3,, pp. 75-755. [6] J. R. Jang, ANFIS: Adaptve-Network-Based Fuzzy Inference System, IEEE Transactons on Systems, Man, and Cybernetcs, Vol. 3, 993, pp. 665-685. [7] D. P. Bertsekas, Nonlnear Programmng, Athenas Centfc, 995. [8] D. Marquardt, An Algorthm for Least Squares Estmamaton of NonLnear Parameters, J. Soc. Ind. Appl. Math, 963, pp. 43-44. [9] M. C. Mackey and L. Glass, Oscllatons and Chaos n Physologcal Control Scences, Scence, pp. 87-89, 997.