FUZZY LOGIC AND NEURO-FUZZY MODELING

Transcription

1 , pp Avalable onlne at FUZZY LOGIC AND NEURO-FUZZY MODELING NIKAM S.R.*, NIKUMBH P.J. AND KULKARNI S.P. RAIT College of Enggneerng, Nerul, Nav Mumba, Inda. *Correspondng Author: Emal- Receved: February, 0; Accepted: March 5, 0 Abstract- Fuzzy logc and fuzzy systems have recently been recevng a lot of attenton, both from the meda and scentfc communty, yet the basc technques were orgnally developed n the md-sxtes. Fuzzy logc provdes a formalsm for mplementng expert or heurstc rules on computers, and whle ths s the man goal n the feld of expert or knowledge-based systems, fuzzy systems have had consderably more success and have been sold n automobles, cameras, washng machnes, rce cookers, etc. Ths report wll descrbe the theory behnd basc fuzzy logc and nvestgate how fuzzy systems work. Ths leads naturally on to neuro fuzzy systems whch attempt to fuse the best ponts of neural and fuzzy networks nto a sngle system. Throughout ths report, the potental lmtatons of ths method wll be descrbed as ths provdes the reader wth a greater understandng of how the technques can be appled. Keywords- Fuzzy logc, Neural networks, fuzzy modelng, neuro-fuzzy systems, neuro-fuzzy modelng, ANFIS. Ctaton: Nkam R.S., Nkumbh P.J. and Kulkarn S.P. (0) Fuzzy Logc and Neuro-Fuzzy Modelng., ISSN: & E-ISSN: , Volume 3, Issue, pp Copyrght: Copyrght 0 Nkam S.R., Nkumbh P.J. and Kulkarn S.P. Ths s an open-access artcle dstrbuted under the terms of the Creatve Commons Attrbuton Lcense, whch permts unrestrcted use, dstrbuton, and reproducton n any medum, provded the orgnal author and source are credted. Introducton Neuro Fuzzy (NF) computng s a popular framework for solvng complex problems. If we have knowledge expressed n lngustc rules, we can buld a FIS, and f we have data, or can learn from a smulaton (tranng) then we can use ANNs. For buldng a FIS, we have to specfy the fuzzy sets, fuzzy operators and the knowledge base. Smlarly for constructng an ANN for an applcaton the user needs to specfy the archtecture and learnng algorthm. An analyss reveals that the drawbacks pertanng to these approaches seem complementary and therefore t s natural to consder buldng an ntegrated system combnng the concepts. Whle the learnng capablty s an advantage from the vewpont of FIS, the formaton of lngustc rule base wll be advantage from the vewpont of ANN. Hayash et al. showed that a feed forward neural network could approxmate any fuzzy rule based system and any feed forward neural network may be approxmated by a rule based fuzzy nference system. Fuson of Artfcal Neural Networks (ANN) and Fuzzy Inference Systems (FIS) have attracted the growng nterest of researchers n varous scentfc and engneerng areas due to the growng need of adaptve ntellgent systems to solve the real world problems. A neural network learns from scratch by adjustng the nterconnectons between layers. Fuzzy nference system s a popular computng framework based on the concept of fuzzy set theory, fuzzy f-then rules, and fuzzy reasonng. The advantages of a combnaton of neural networks and fuzzy nference systems are obvous. An analyss reveals that the drawbacks pertanng to these approaches seem complementary and therefore t s natural to consder buldng an ntegrated system combnng the concepts. The arrangement of ths artcle s as follows: In part, an ntroducton to the basc concepts of fuzzy sets, fuzzy reasonng, fuzzy f-then rules are gven. In part 3, Fuzzy nference System s Descrbe. In part 4, s devoted to the Neuro-Fuzzy systems. In part 5, a number of desgn technques for fuzzy and neural controllers s descrbed. In part 6, concludes the paper by pontng current problems and future drectons. Bonfo Publcatons 74

2 Fuzzy Logc and Neuro-Fuzzy Modelng Fuzzy Sets, Fuzzy Rules, Fuzzy Reasonng Ths secton provdes a concse ntroducton to and a summary of the basc concepts central to the study of fuzzy sets. A. Fuzzy sets Fuzzy logc starts wth the concept of a fuzzy set. A fuzzy set s a set wthout a crsp, clearly defned boundary. It can contan el ments wth only a partal degree of membershp. A classcal set s a contaner that wholly ncludes or wholly excludes any gven element. Another verson of ths law s: Of any subject one thng must be ether asserted or dened That s, the transton from belongng to a set to not belongng to a set s gradual, and ths smooth transton s characterzed by membershp functon that gve fuzzy sets flexblty n modelng commonly used lngustc expressons. B. Membershp Functon A membershp functon (MF) s a curve that defnes how each pont n the nput space s mapped to a membershp value (or degree of membershp) between 0 and. The nput space s sometmes referred to as the unverse of dscourse, a fancy name for a smple concept. The output-axs s a number known as the membershp value between 0 and. The curve s known as a membershp functon and s often gven the desgnaton of µ. The smplest membershp functons are formed usng straght lnes. Of these, the smplest s the trangular membershp functon, and t has the functon name trmf. Ths functon s nothng more than a collecton of three ponts formng a trangle. The trapezodal membershp functon, trapmf, has a flat top and really s just a truncated trangle curve. These straght lne membershp functons have the advantage of smplcty. Fg..- Trangular & Trapezodal MF Two membershp functons are bult on the Gaussan dstrbuton curve: a smple Gaussan curve and a two-sded composte of two dfferent Gaussan curves. The two functons are gaussmf and gaussmf. The generalzed bell membershp functon s specfed by three parameters and has the functon name gbellmf. Fg..3- Sgmodal MF C. Fuzzy If-Then Rules A fuzzy f-then rule (fuzzy rule, fzzy mplcaton or fuzzy condtonal statement) assumes the form If x s A then y s B, Where A and B are lngustc values defned by fuzzy sets on unverse of dscourse X and Y, respectvely. Often x s A s called the antecedent or premse whle y s B s called the consequence or concluson. Examples of fuzzy f-then rules n our daly lngustc expressons are as follows: If the road s slppery the drvng s dangerous. If a tomato s red then t s rpe. Before we can employ fuzzy f-then rules to model and analyze a system, we frst have to formalze what s meant by the expresson f x s A then y s B, whch s sometmes abbrevated as A B. n essence, the expresson descrbes a relaton between two varables x and y; ths suggests that a fuzzy f-then rule be defned as a bnary fuzzy relaton R on the product space X Y. a bnary relaton R s an extenson of the classcal Cartesan product, where each element (x,y) X Y s assocated wth a membershp grade denoted by µr(x,y). Interpretng an f-then rule nvolves dstnct parts: frst evaluatng the antecedent (whch nvolves fuzzfyng the nput and applyng any necessary fuzzy operators) and second applyng that result to the consequent (known as mplcaton). In the case of two-valued or bnary logc, f-then rules do not present much dffculty. If the premse s true, then the concluson s true. If you relax the restrctons of two-valued logc and let the antecedent be a fuzzy statement, how does ths reflect on the concluson? The answer s a smple one. f the antecedent s true to some degree of membershp, then the consequent s also true to that same degree. D. Fuzzy Reasonng Fuzzy reasonng also known as approxmate reasonng s an nference procedure used to derve concluson from a set of fuzzy fthen rules and one or more condtons. Before ntroducng fuzzy reasonng, we wll dscuss the compostonal rule of nference. The compostonal rule of nference s a generalzaton of the followng noton. Suppose that we have a curve y = f(x) that regulates the relaton between x and y. when we are gven x=a, then from y=f(x) we can nfer that y = b = f(a)whch s shown below fg. Fg..- Gaussan & Generalzed Bell MF Also we can defne the sgmodal membershp functon, whch s ether open left or rght. Asymmetrc and closed (.e. not open to the left or rght) membershp functons can be syntheszed usng two sgmodal functons, so n addton to the basc sgmf, you also have the dfference between two sgmodal functons, dsgmf, and the product of two sgmodal functons psgmf. Fg..4- Compostonal Rule (a) a & b are ponts (b) a & b are ntervals Usng the compostonal rule of nference, we can formulze an nference procedure, called fuzzy reasonng, upon a set of fuzzy f- Bonfo Publcatons 75

3 Nkam S.R., Nkumbh P.J. and Kulkarn S.P. then rules. The basc rule of nference n tradtonal two-valued logc s modus ponens, accordng to whch we can nfer the truth of a proposton B from the truth of A and the mplcaton A B. for nstance, f A s dentfed wth the tomato s red and B wth the tomato s rpe, then f t s true that the tomato s red, t s also true that the tomato s rpe. Premse (fact)- x s A, Premse (rule)- f x s A then y s B, Consequence(concluson)- y s B. However, n much of human reasonng, modus ponens s employed n an approxmate manner. Usng ths compostonal rule we can perform fuzzy reasonng. The varous types of reasonng are: Based on max-mn composton Sngle rule wth sngle antecedent Sngle rule wth two antecedents Multple rules wth two antecedents Fuzzy Inference Systems Fuzzy nference s the process of formulatng the mappng from a gven nput to an output usng fuzzy logc. The mappng then provdes a bass from whch decsons can be made, or patterns dscerned. The process of fuzzy nference nvolves all of the peces that are descrbed n the prevous sectons: Membershp Functons, Fuzzy set theory, and If-Then Rules and Fuzzy reasonng. Because of ts multdscplnary nature, fuzzy nference systems are assocated wth a number of names, such as fuzzy-rule-based systems, fuzzy expert systems, fuzzy modelng, fuzzy assocatve memory, fuzzy logc controllers, and smply (and ambguously) fuzzy systems. Bascally a fuzzy nference system s composed of fve functonal blocks: A rule base contanng a number of fuzzy f-then rules; A database whch defnes the membershp functons of the fuzzy sets used n the fuzzy rules; A reasonng mechansm whch performs the nference procedure upon the rules and a gven condton to derve a reasonable output; A fuzzfcaton nterface whch transforms the crsp nputs nto degrees of match wth lngustc values; A defuzzfcaton nterface whch transform the fuzzy results of the nterface nto a crsp output. Usually, the rule base and the database are jontly referred as knowledge base. transformng a fuzzy output nto a crsp one. Wth crsp nputs and outputs, a fuzzy nference system mplements a non-lnear mappng from ts nput space to output space. Ths mappng s done by a no of fuzzy f-then rules, each of whch descrbes the local behavor of the mappng. Now, we wll frst ntroduce three types of fuzzy nference systems that have been wdely employed n varous applcatons. The dfference between these three fuzzy nference systems le n the consequents of ther fuzzy rules, and ther aggregaton and defuzzfcaton procedures. A. Mamdan Fuzzy Model The Mamdan Fuzzy Model was proposed to control a steam e gne and boler combnaton by a set of lngustc control rules. Followng fg. shows how a two-rule fuzzy nference system of the Mamdan type derves the overall output z when subjected to two crsp nputs x and y. Fg. 3.- Max-Mn composton If we adopt product and max as our choce for the fuzzy AND and OR operators and use max-product composton nstead of maxmn composton, then the resultng fuzzy reasonng s shown below: Fg Max-Product composton Snce the plant takes only crsp values as nputs, we have defuzzfer to convert a fuzzy set to a crsp value. The most frequently used defuzzfcaton strategy s the centrod of area, whch s defne as Azzdz COA Z Az Z where, µa(z) s the aggregated output MF. Other defuzzfcaton strateges arse for specfc applcatons, whch nclude bsector of area, mean of maxmum, largest of maxmum, and smallest of maxmum and so on. dz Fg.3.- Block dagram for fuzzy nference system The dashed lne ndcates a basc fuzzy nference system wth fuzzy output and the defuzzfcaton block serves the purpose of Fg 3.4- Defuzzfcaton Strateges Bonfo Publcatons 76

4 Fuzzy Logc and Neuro-Fuzzy Modelng B. Sugeno Fuzzy Model The sugeno fuzzy model also known as TSK fuzzy model was proposed to develop a systematc approach to generatng fuzzy rules from a gven nput-output data set. A typcal fuzzy rule n a sugeno fuzzy model has the form If x s A and y s B the z = f(x,y) Where A and B are fuzzy sets n the antecedent and z = f(x,y) s a crsp functon n the consequent. when f(x,y) s a frst order polynomal, the resultng fuzzy nference system s called a frst order sugeno fuzzy model. When f s constant, then we have zero-order sugeno fuzzy model, whch can be vewed as a specal case of the mamdan fuzzy nference system. Fg Frst Order Sugeno Fuzzy Model fgure 3.5 shows the reasonng procedure for a frst-order sugeno fuzzy model. The aggregator and defuzzfer blocks n fg 3. are replaced by the operaton of weghted average, thus avodng the tme consumng procedure of defuzzfcaton. Sometmes the weghted average operator s replaced wth the weghted sum operator. However, ths smplfcaton could lead to the loss of MF lngustc meanngs unless the sum of frng strength s close to unty. C. Tsukamoto Fuzzy Model In the Tsukamoto fuzzy model, the consequent of each fuzzy fthen rule s represented by a fuzzy set wth a monotoncal MF as shown n fg. The nferred output of each rule s defned as a crsp value nduced by the rule s frng strength. The overall output s taken as the weghted average of each rule s output. Fg Tsukamoto Fuzzy Model Snce each rule nfers a crsp output, the Tsukamoto fuzzy model aggregates each rule s output by the method of weghted average and also avods the tme consumng process of defuzzfcaton. Neuro-Fuzzy Systems Neuro Fuzzy (NF) computng s a popular framework for solvng complex problems. If we have knowledge expressed n lngustc rules, we can buld a FIS, and f we have data, or can learn from a smulaton (tranng) then we can use ANNs. For buldng a FIS, we have to specfy the fuzzy sets, fuzzy operators and the knowledge base. Smlarly for constructng an ANN for an applcaton the user needs to specfy the archtecture and learnng algorthm. An analyss reveals that the drawbacks pertanng to these approaches seem complementary and therefore t s natural to consder buldng an ntegrated system combnng the concepts. Whle the learnng capablty s an advantage from the vewpont of FIS, the formaton of lngustc rule base wll be advantage from the vewpont of ANN. The process for constructng a fuzzy nference system s usually called fuzzy modelng, whch has followng features: Due to the rule structure of a fuzzy nference system, t s easy to ncorporate human expertse about the target system drectly nto the modelng process. Fuzzy modelng takes advantage of doman knowledge that mght not be easly or drectly employed n other modelng approaches. When the nput-output data of a system to be modeled s avalable, conventonal system dentfcaton technques can be used for fuzzy modelng. The term neuro-fuzzy modelng refers to the way of applyng varous learnng technques developed n the neural network lterature to fuzzy nference systems. Now, we present cooperatve NF system and concurrent NF system followed by the dfferent fused NF models. A. Cooperatve And Concurrent Neuro-Fuzzy Systems In the smplest way, a cooperatve model [][][3]can be consdered as a preprocessor wheren artfcal neural network (ANN) learnng mechansm determnes the fuzzy nference system (FIS) membershp functons or fuzzy rules from the tranng data. Once the FIS parameters are determned, ANN goes to the background. The rule based s usually determned by a clusterng approach or fuzzy clusterng algorthms. Membershp functons are usually approxmated by neural network from the tranng data. Fg. 4.- Cooperatve NF model In a concurrent model[][], neural network asssts the fuzzy system contnuously (or vce versa) to determne the requred parameters especally f the nput varables of the controller cannot be measured drectly. Such combnatons do not optmze the fuzzy system but only ads to mprove the performance of the overall system. Learnng takes place only n the neural network and the fuzzy system remans unchanged durng ths phase. In some cases the fuzzy outputs mght not be drectly applcable to the process. In that case neural network can act as a postprocessor of fuzzy outputs. Fgure 4. depcts a concurrent neuro-fuzzy model where n the nput data s fed to a neural network and the output of the neural network s further processed by the fuzzy system. Fg. 4.- Concurrent NF model Bonfo Publcatons 77

5 Nkam S.R., Nkumbh P.J. and Kulkarn S.P. B. Fused Neuro-Fuzzy Systems In an ntegrated model[][][3], neural network learnng algorthms are used to determne the parameters of fuzzy nference systems. Integrated neuro-fuzzy systems share data structures and knowledge representatons. A fuzzy nference system can utlze human expertse by storng ts essental components n rule base and database, and perform fuzzy reasonng to nfer the overall output value. The dervaton of f-then rules and correspondng membershp functons depends heavly on the a pror knowledge about the system under consderaton. However there s no systematc way to transform experences of knowledge of human experts to the knowledge base of a fuzzy nference system. There s also a need for adaptablty or some learnng algorthms to produce outputs wthn the requred error rate. On the other hand, neural network learnng mechansm does not rely on human expertse. However, n realty, the a pror knowledge s usually obtaned from human experts, t s most approprate to express the knowledge as a set of fuzzy f-then rules, and t s very dffcult to encode nto a neural network. Table 4.- Comparson between neural networks and fuzzy nference systems Artfcal Neural Network Dffcult to use pror rule knowledge Learnng from scratch Black box Complcated learnng algorthms Dffcult to extract knowledge Fuzzy Inference System Pror rule-base can be ncorporated Cannot learn (lngustc knowledge) Interpretable (f-then rules) Smple nterpretaton and mplementaton Knowledge must be avalable Table 4. summarzes the comparson between neural networks and fuzzy nference system. To a large extent, the drawbacks pertanng to these two approaches seem complementary. Therefore, t seems natural to consder buldng an ntegrated system combnng the concepts of FIS and ANN modelng.. Ths problem can be tackled by usng dfferentable functons n the nference system or by not usng the standard neural learnng algorthm. Some of the major woks n ths area are GARIC, FAL- CON, ANFIS, NEFCON, FUN, SONFIN,FINEST, EFuNN, dmefunn, evolutonary desgn of neuro fuzzy systems, and many others. C. Fuzzy Adaptve learnng Control Network (FALCON) FALCON [][][9] has a fve-layered archtecture and mplements a Mamdan type FIS. There are two lngustc nodes for each output varable. One s for tranng data (desred output) and the other s for the actual output of FALCON. The frst hdden layer s responsble for the fuzzfcaton of each nput varable. Each node can be a sngle node representng a smple membershp functon (MF) or composed of multlayer nodes that compute a complex MF. The Second hdden layer defnes the precondtons of the rule followed by rule consequents n the thrd hdden layer. FALCON uses a hybrd-learnng algorthm comprsng of unsupervsed learnng and a gradent descent learnng to optmally adjust the parameters to produce the desred outputs. The hybrd learnng occurs n two dfferent phases. In the ntal phase, the centers and wdth of the membershp functons are determned by selforganzed learnng technques analogous to statstcal clusterng technques. Once the ntal parameters are determned, t s easy to formulate the rule antecedents. A compettve learnng algorthm s used to determne the correct rule consequent lnks of each rule node. After the fuzzy rule base s establshed, the whole network structure s establshed. The network then enters the second learnng phase to adjust the parameters of the (nput and output) membershp functons optmally. The back propagaton algorthm s used for the supervsed learnng. Hence FALCON algorthm provdes a framework for structure and parameter adaptaton for desgnng neuro-fuzzy systems. Fg Archtecture of FALCON D. Generalzed Approxmate Reasonng based Intellgent Control (GARIC) GARIC [][][9] s an extended verson of Berenj s Approxmate Reasonng based Intellgent Control (ARIC) that mplements a fuzzy controller by usng several specalzed feed forward neural networks. Lke ARIC, t conssts of an Acton state Evaluaton Network (AEN) and an Acton Selecton Network (ASN). Archtecture of the GARICASN s depcted n Fg.4.4. ASN of GARIC s feedforward network wth ASN of GARIC s feed forward network wth fve layers. Fg ASN of GARIC The frst hdden layer stores the lngustc values of all the nput varables. Each nput unt s only connected to those unts of the frst hdden layer, whch represent ts assocated lngustc values. The second hdden layer represents the fuzzy rules nodes, whch determne the degree of fulfllment of a rule usng a softmn operaton. The thrd hdden layer represents the lngustc values of the control output varable η. Conclusons of the rule are computed dependng on the strength of the rule antecedents computed by the rule node layer. GARIC makes use of local mean-of-maxmum method for computng the rule outputs. Ths method needs a crsp output value from each rule. Therefore, the conclusons must be defuzzfed before they are accumulated to the fnal output value of the controller. GARIC uses a mxture of gradent descent and renforcement learnng to fne-tune the node parameters. The hybrd learnng stops f the output of the AEN ceases to change. The relatvely complex learnng procedure and the archtecture of GARIC can be seen as a man dsadvantage of GARIC. Bonfo Publcatons 78

6 Fuzzy Logc and Neuro-Fuzzy Modelng E. Neuro-Fuzzy Control (NEFCON) The learnng algorthm defned for NEFCON[][][9] s able to learn fuzzy sets as well as fuzzy rules mplementng a Mamdan type FIS [][]. Ths method can be consdered as an extenson to GARIC that also use renforcement learnng but need a prevously defned rule base. Fgure 4.5 llustrates the basc NEFCON archtecture wth nputs and fve fuzzy rules [][]. The nner nodes R,..., R5 represent the rules, the nodes ξ, ξ, and η the nput and output values, and μr, Vr the fuzzy sets descrbng the antecedents and consequents. In contrast to neural networks, the connectons n NEFCON are weghted wth fuzzy sets nstead of real numbers. Rules wth the same antecedent use so-called shared weghts, whch are represented by ellpses drawn around the connectons as shown n the fgure 4.5. They ensure the ntegrty of the rule base. The knowledge base of the fuzzy system s mplctly gven by the network structure. The nput unts assume the task of fuzzfcaton nterface, the nference logc s represented by the propagaton functons, and the output unt s the defuzzfcaton nterface. The learnng process of the NEFCON model can be dvded nto two man phases. Incremental rule learnng s used when the correct output s not known and rules are created based on estmated output values. As the learnng progresses, more rules are added accordng to the requrement. For decremental rule learnng, ntally rules are created due to fuzzy parttons of process varables and unnecessary rules are elmnated n the course of learnng. Decremental rule learnng s less effcent compared to ncremental approach. F. Fuzzy Inference Envronment Software wth Tunng (FINEST) FINEST[][][9] s desgned to tune the fuzzy nference tself. FINEST s capable of two knds of tunng process, the tunng of fuzzy predcates, combnaton functons and the tunng of an mplcaton functon []. The three mportant features of the system are: The generalzed modus ponens s mproved n the followng four ways:. aggregaton operators that have synergy and cancellaton nature. a parameterzed mplcaton functon 3. a combnaton functon, whch can reduce fuzzness 4. backward channg based on generalzed modus ponens. Aggregaton operators wth synergy and cancellaton nature are defned usng some parameters, ndcatng the strength of the synergc affect, the area nfluenced by the effect, etc., and the tunng mechansm s desgned to tune these parameters also tune the mplcaton functon and combnaton functon. The software envronment and the algorthms are desgned for carryng out forward and backward channg based on the mproved generalzed modus ponens and for tunng varous parameters of a system. Fg Archtecture of FINEST Fg Archtecture of NEFCON Due to the complexty of the calculatons requred, the decremental learnng rule can only be used, f there are only a few nput varables wth not too many fuzzy sets. For larger systems, the ncremental learnng rule wll be optmal. Pror knowledge whenever avalable could be ncorporated to reduce the complexty of the learnng. Membershp functons of the rule base are modfed accordng to the Fuzzy Error Back propagaton (FEBP) algorthm. The FEBP algorthm can adapt the membershp functons, and can be appled only f there s already a rule base of fuzzy rules. The dea of the learnng algorthm s dentcal: ncrease the nfluence of a rule f ts acton goes n the rght drecton (rewardng), and decrease ts nfluence f a rule behaves counter productvely (punshng). If there s absolutely no knowledge about ntal membershp functon, a unform fuzzy partton of the varables should be used. FINEST make use of a back propagaton algorthm for the fnetunng of the parameters. Fgure4.6 shows the layered archtecture of FINEST and the calculaton process of the fuzzy nference. The nput values (x) are the facts and the output value (y) s the concluson of the fuzzy nference. Layer s a fuzzfcaton layer and layer aggregates the truth-values of the condtons of Rule. Layer 3 deduces the concluson from Rule I and the combnaton of all the rules s done n Layer 4. Referrng to Fg. 4.6, the functon and, I and comb respectvely represent the functon characterzng the aggregaton operator of rule, the mplcaton functon of rule, and the global combnaton functon. The functons and, I, comb and membershp functons of each fuzzy predcate are defned wth some parameters. Back propagaton method s used to tune the network parameters. It s possble to tune any parameter, whch appears n the nodes of the network representng the calculaton process of the fuzzy data f the dervatve functon wth respect to the parameters s gven. Thus, FINEST framework provdes a mechansm based on the mproved generalzed modus ponens for fne tunng of fuzzy predcates and combnaton functons and tunng of the mplcaton functon. Bonfo Publcatons 79

7 Nkam S.R., Nkumbh P.J. and Kulkarn S.P. G. Self Constructng Neural Fuzzy Inference Network (SONFIN) SONFIN [][][9] mplements a Takag-Sugeno type fuzzy nference system. Fuzzy rules are created and adapted as onlne learnng proceeds va a smultaneous structure and parameter dentfcaton. In the structure dentfcaton of the precondton part, the nput space s parttoned n a flexble way accordng to an algned clusterng based algorthm. As to the structure dentfcaton of the consequent part, only a sngleton value selected by a clusterng method s assgned to each rule ntally. Afterwards, some addtonal sgnfcant terms (nput varables) selected va a projecton-based correlaton measure for each rule wll be added to the consequent part (formng a lnear equaton of nput varables) ncrementally as learnng proceeds. For parameter dentfcaton, the consequent parameters are tuned optmally by ether Least Mean Squares [LMS] or Recursve Least Squares [RLS] algorthms and the precondton parameters are tuned by back propagaton algorthm. To enhance knowledge representaton ablty of SONFIN, a lnear transformaton for each nput varable can be ncorporated nto the network so that much fewer rules are needed or hgher accuracy can be acheved. Proper lnear transformatons are also learned dynamcally n the parameter dentfcaton phase of SONFIN. (fuzzy-and) are calculated. Membershp functons of the output varables are stored n the thrd hdden layer. Ther actvaton functon s a fuzzy-or. Fnally, the output neurons contan the output varables and have a defuzzfcaton actvaton functon. The rules and the membershp functons are used to construct an ntal FUN network. The rule base can then be optmzed by changng the structure of the net or the data n the neurons. To learn the rules, the connectons between the rules and the fuzzy values are changed. To learn the membershp functons, the data of the nodes n the frst and three hdden layers are changed. FUN can be traned wth the standard neural network tranng strateges such as renforcement or supervsed learnng. Fg Archtecture of the FUN showng the mplementaton of a sample rule Fg llustrates the 6-layer structure of SONFIN and the parameters n the precondton part are adjusted by the backpropagaton algorthm. SONFIN can be used for normal operaton at anytme durng the learnng process wthout repeated tranng on the nput-output pattern when onlne operaton s requred. In SONFIN rule base s dynamcally created as the learnng progresses by performng the followng learnng processes: H. Fuzzy Net (FUN) In FUN[][][9] n order to enable an unequvocal translaton of fuzzy rules and membershp functons nto the network, specal neurons have been defned, through ther actvaton functons, can evaluate logc expressons. The network conssts of an nput, an output and three hdden layers. The neurons of each layer have dfferent actvaton functons representng the dfferent stages n the calculaton of fuzzy nference. The actvaton functon can be ndvdually chosen for problems. The network s ntalzed wth a fuzzy rule base and the correspondng membershp functons. Fgure 4.8 llustrates the FUN network. The nput varables are stored n the nput neurons. The neurons n the frst hdden layer contan the membershp functons and ths performs a fuzzfcaton of the nput values. In the second hdden layer, the conjunctons I. Evolvng Fuzzy Neural Networks (EFuNN) EFuNNs [][][9] and dmefunns [][][9] are based on the ECOS (Evolvng Connectonst Systems) framework for adaptve ntellgent systems formed because of evoluton and ncremental, hybrd (supervsed/unsupervsed), onlne learnng. They can accommodate new nput data, ncludng new features, new classes, etc. through local element tunng. In EFuNNs all nodes are created durng learnng. EFuNN has a fve-layer archtecture as shown n Fgure 4.9. The nput layer s a buffer layer representng the nput varables. The second layer of nodes represents fuzzy quantfcaton of each nput varable space. Each nput varable s represented here by a group of spatally arranged neurons to represent a fuzzy quantzaton of ths varable. The nodes representng membershp functons (trangular, Gaussan, etc) can be modfed durng learnng. The thrd layer contans rule nodes that evolve through hybrd supervsed/unsupervsed learnng. Fg Archtecture of EFuNN Bonfo Publcatons 80

8 Fuzzy Logc and Neuro-Fuzzy Modelng The rule nodes represent prototypes of nput-output data assocatons, graphcally represented as an assocaton of hyper-spheres from the fuzzy nput and fuzzy output spaces. Each rule node r s defned by two vectors of connecton weghts: W(r) and W(r), the latter beng adjusted through supervsed learnng based on the output error, and the former beng adjusted through unsupervsed learnng based on smlarty measure wthn a local area of the nput problem space. The fourth layer of neurons represents fuzzy quantfcaton for the output varables. The ffth layer represents the real values for the output varables. In the case of oneof-n EFuNNs, the maxmum actvaton of the rule node s propagated to the next level. In the case of many-of-n mode, all the actvaton values of rule nodes that are above an actvaton threshold are propagated further n the connectonst structure. J. Dynamc Evolvng Fuzzy Neural Networks (dmefunns) Dynamc Evolvng Fuzzy Neural Networks (dmefunn) model[] [][9] s developed wth the dea that not just the wnnng rule node s actvaton s propagated but a group of rule nodes s dynamcally selected for every new nput vector and ther actvaton values are used to calculate the dynamcal parameters of the output functon. Whle EFuNN make use of the weghted fuzzy rules of Mamdan type, dmefunn uses the Takag-Sugeno fuzzy rules. The archtecture s depcted n Fgure 4.0 means that f there are suffcent tranng data vectors and suffcent rule nodes are created, a satsfyng accuracy can be obtaned. K. Evolutonary and Neural Learnng of Fuzzy Inference System (EvoNF) In an ntegrated neuro-fuzzy model there s no guarantee that the neural network learnng algorthm converges and the tunng of fuzzy nference system wll be successful. Natural ntellgence s a product of evoluton. Therefore, by mmckng bologcal evoluton, we could also smulate hgh-level ntellgence. Evolutonary computaton works by smulatng a populaton of ndvduals, evaluatng ther performance, and evolvng the populaton a number of tmes untl the requred soluton s obtaned. The drawbacks pertanng to neural networks and fuzzy nference systems seem complementary and evolutonary computaton could be used to optmze the ntegraton to produce the best possble synergetc behavor to form a sngle system. Adaptaton of fuzzy nference systems usng evolutonary computaton technques has been wdely explored. EvoNF s an adaptve framework based on evolutonary computaton and neural learnng wheren the membershp functons, rule base and fuzzy operators are adapted accordng to the problem. The evolutonary search of MFs, rule base, fuzzy operators etc. would progress on dfferent tme scales to adapt the fuzzy nference system accordng to the problem envronment. Membershp functons and fuzzy operators would be further fnetuned usng a neural learnng technque. Optmal neural learnng parameters wll be decded durng the evolutonary search process. Fg Archtecture of dmefunn The frst, second and thrd layers of dmefunn have exactly the same structures and functons as the EFuNN. The fourth layer, the fuzzy nference layer, selects m rule nodes from the thrd layer whch have the closest fuzzy normalzed local dstance to the fuzzy nput vector, and then, a Takag Sugeno fuzzy rule wll be formed usng the weghted least square estmator. The last layer calculates the output of dmefunn. The number m of actvated nodes used to calculate the output values for a dmefunn s not less than the number of the nput nodes plus one. Lke the EFuNNs, the dmefunns can be used for both offlne learnng and onlne learnng thus optmzng global generalzaton error, or a local generalzaton error. In dmefunns, for a new nput vector (for whch the output vector s not known), a subspace conssted of m rule nodes are found and a frst order Takag Sugeno fuzzy rule s formed usng the least square estmator method. Ths rule s used to calculate the dmefunn output value. In ths way, a dmefunn acts as a unversal functon approxmator usng m lnear functons n a small m dmensonal node subspace. The accuracy of approxmaton depends on the sze of the node subspaces, the smaller the subspace s, the hgher the accuracy. It Fg. 4.- Interacton of evolutonary search mechansms n the adaptaton of fuzzy nference system Fgure 4. llustrates the general nteracton mechansm of the EvoNF framework wth the evolutonary search of fuzzy nference system (Mamdan, Takag -Sugeno etc.) evolvng at the hghest level on the slowest tme scale. For each evolutonary search of fuzzy operators (best combnaton of T-norm and T-conorm, defuzzfcaton strategy etc), the search for the fuzzy rule base progresses at a faster tme scale n an envronment decded by the problem. In a smlar manner, evolutonary search of membershp functons proceeds at a faster tme scale (for every rule base) n the envronment decded by the problem. Herarchy of the dfferent adaptaton procedures wll rely on the pror knowledge. L. Adaptve Network Based Fuzzy Inference System (ANFIS) In ths secton we wll dscuss the archtecture and learnng procedure of the adaptve network whch s n fact a superset of all knds of feed forward neural networks wth supervsed learnng capablty. As ts name mples, adaptve network structure con- Bonfo Publcatons 8

9 Nkam S.R., Nkumbh P.J. and Kulkarn S.P. sstng of nodes and drectonal lnks though whch the nodes are connected. Also, part or all of the nodes are adaptve, whch means each output of these nodes depends on the parameters of ths node, and learnng rule specfes how these parameters should be changed to mnmze a prescrbed error measure. Snce the basc learnng rule s based the gradent method whch s notorous for ts slowness and tendency to become trapped n local mnma, here we propose a hybrd learnng rule whch can speed up the learnng process. ANFIS Archtecture Functonally, there are almost no constrants on the node functons of an adaptve network except pecewse dfferentablty. Structurally, the only lmtaton of network confguraton s that t should be feed-forward type. Due to ths restrctons, the adaptve networks applcatons are mmedate and mmense n varous areas.n ths secton, we descrbe a class of adaptve network whch are functonally equvalent to fuzzy nference systems. For smplcty, assume the fuzzy nference system under consderaton has two nputs x and y and one output z. suppose that the rule base contans two fuzzy f-then rules of Takag and Sugeno type: Rule : If x s A and y s B, then f = px+qy + r Rule : If x s A and y s B, then f = px + qy +r. Then type-3 fuzzy reasonng s gven n belo fg 4. Fg ANFIS- type 3 And equvalent ANFIS archtecture s, Layer Every node n ths layer s a crcle node label as II whch multples the ncomng sgnals and sends the product out. =, Each node output represents the x A x c a b x exp x c a frng strength of a rule. Also T-norm operators that perform generalzed AND can be used as a node functon. Layer 3 Every node n ths layer s a crcle node label N. the -th n o d e x rule s rules frng strength. A B A y calculates the rato of the -th frng strength to the sum of all Outputs of ths layer,, s called as normalzed frng strength. Layer 4 Every node I n ths layer s a square node wth a node functon Where w s the output of layer 3 and {p, q, r} s the parameter set. Param- 4 eters n ths layer f p x q y r wll be referred as consequent parameters. Layer 5 the sngle node n ths layer s a crcle node labeled that computs the overall output as summaton of all ncomng sgnals f 5 overalloutput f Thus we have constructed adaptve network whch s functonally equvalent to type-3 fuzzy nference system. For type- fuzzy nference systems the extenson s qute straghtforward and type- ANFIS s shown n below fg. Where the output of each rule s nduced jontly by the output membershp functon and the frng strength. For type- fuzzy nference systems, f we replace the centrod defuzzfcaton operator wth a dscrete verson whch calculates the centrod of area, then type-3 ANFIS can stll be constructed accordngly. A x b Fg Equvalent ANFIS archtecture Layer Every node I n ths layer s a square node wth a node functon Where x s the nput to node I, and A s the lngustc label (small,large, etc) assocated wth ths node functon. In other words, O s the membershp fucton of A and t specfes the degree to whch the gven x satsfes the quantfer A. Usually µa (x) s choose as bell-shaped wth maxmum equal to and mnmum equal to 0, such as, Where {a, b, c} s parameter set. As the values of these parameters change, the bell-shaped fuctons vary accordngly. Parameters n ths layer are reffered to as premse parameters. Fg 4.5- (a)type Fuzzy Reasonng (b) Equvalent ANFIStype Below fgure shows a -nput, type-3 ANFIS wth 9 rules. Three membershp functons are assocated wth each nput, so the nput space s parttoned nto 9 fuzzy subspaces, each of whch s governed by a fuzzy f-then rules. The premse part of a rule delneates a fuzzy subspaces, whle the consequent part specfes the output wthn ths fuzzy subspace. Bonfo Publcatons 8

10 Fuzzy Logc and Neuro-Fuzzy Modelng all parameters. The choce of above methods should be based on the trade-off between computaton complexty and resultng performance. Fg (a) ANFIS type-3 wth nput and 9 rules (b) Correspondng fuzzy Subspaces Hybrd Learnng Algorthm From the type-3 anfs archtecture t s observed that gven values of premse parameters, the output can be expressed as a lnear combnatons of the consequent parameters. The output f n fg. 4 can be rewrtten as, = Whch s lnear n consequent parameters (p, q, r, p, q, and r). S = set of total parame- ters S = set of premse pa- f f rameters S = set of consequent parameters In forward pass of hybrd learnng algorthm, functonal sgnals go forward tll layer 4 and the consequent parameters are dentfed by the least squares estmate. In the backward pass, the error rates propagate backward and the premse parameters are updated by the gradent descent. Below table summarzes the actvtes n each pass. Table 4.- Summarzng the actvtes n each pass The consequent parameters dentfed are optmal under the condton that premse parameters are fxed. Accordngly the hybrd approach s much faster than the strct gradent descent. It should be noted that computaton complexty of the least squares estmate s hgher than that of the gradent descent. There are four methods to update the parameters,. Gradent descent only:- all parameters are updated by the gradent descent.. x) p ( y) f f f Gradent descent and one pass of LSE:- the LSE s appled only once at the very begnnng to get the ntal values of the consequent parameters and then gradent descent takes over to update all parameters.. Gradent descent and LSE:- ths s proposed hybrd learnng rule. v. Sequental LSE only:- the ANFIS s lnearzed w.r.t all parameters and the extended kalman flter algo s employed to update r xp yq ( q r Forward pass Backward pass Premse parameters Fxed Gradent descent Consequent parameters Least squares estmates Fxed Sgnals Node outputs Error rates = Concluson and Future scope We presented the dfferent ways to learn fuzzy nference systems usng neural network learnng technques. As a gudelne, for neurofuzzy systems to be hghly ntellgent some of the major requrements are fast learnng (memory based - effcent storage and retreval capactes), on-lne adaptablty (accommodatng new features lke nputs, outputs, nodes, connectons etc), acheve a global error rate and computatonally nexpensve. The data acquston and preprocessng tranng data s also qute mportant for the success of neuro-fuzzy systems. Many neuro-fuzzy models use supervsed/unsupervsed technques to learn the dfferent parameters of the nference system. The success of the learnng process s not guaranteed, as the desgned model mght not be optmal. Emprcal research has shown that gradent descent technque (most commonly used supervsed learnng algorthm) s trapped n local optma especally when the error surface s complcated. Global optmzaton procedures lke evolutonary algorthms, smulated annealng, tabu search etc. mght be useful for adaptve evoluton of fuzzy f-then rules, shape and quantty of membershp functons, fuzzy operators and other node functons, to prevent the network parameters beng trapped n local optma due to relance on gradent nformaton by most of the supervsed learnng technques. Sugeno-type fuzzy systems are hgh performers (less RMSE) but often requres complcated learnng procedures and computatonal expensve. However, Mamdan-type fuzzy systems can be modeled usng faster heurstcs but wth a compromse on the performance (accuracy). Hence there s always a compromse between performance and computatonal tme. ANFIS mplements a Takag-Sugeno fuzzy system and apples a mxture of back propagaton and least mean squares procedure to tran the system. The adaptaton process s only concerned wth parameter level adaptaton wthn fxed structures. For large scale problems, t wll be too much complcated to determne the optmal premse consequent structures, rule numbers etc. the structure of ANFIS ensures that each lngustc term s represented by only one fuzzy set. The learnng procedure of ANFIS does not provde the means to apply constrants that restrct the knd of modfcaton appled to membershp functons. Due to the hgh flexblty of adaptve networks, the anfs can have number of varants, for nstance, the membershp functons can be changed to L-R representaton whch could be asymmetrc, also we can replace II nodes n layer wth parameterzed T-norm and the learnng rule to decde the best T-norm operator for a specfc applcaton. By employng the adaptve network as a common framework, we have proposed other adaptve fuzzy models for data classfcaton and feature extracton purposes. FUN system s ntalzed by specfyng a fxed number of rules and a fxed number of ntal fuzzy sets for each varable and there after uses a stochastc procedure that randomly changes parameters of membershp functons and connectons wthn the network structure. The learnng process s drven by a cost functon, whch s evaluated after random modfcaton. NEFCON makes use of an ncremental or decremental learnng Bonfo Publcatons 83

11 Nkam S.R., Nkumbh P.J. and Kulkarn S.P. algorthm for learnng the rule base and back propagaton algorthm for learnng the fuzzy sets. NEFCON system s capable of ncorporatng pror knowledge as well as learnng from scratch. The performance of the system wll very much depend on heurstc factors lke learnng rate, error measure etc. FINEST provdes a mechansm based on the mproved generalzed modus ponens for fne tunng of fuzzy predcates & combnaton functons and tunng of an mplcaton functon. FINEST uses a gradent descent technque to tune the varous parameters. Parameterzaton of the nference procedure s very much essental for proper applcaton of the tunng algorthm. SONFIN learns from scratch and the rules are created and adapted as onlne learnng proceeds va smultaneous structure and parameter dentfcaton. As the learnng proceeds, rules wll get modfed ncrementally. References [] Ajth Abraham (00) Sxth nternatonal work conference on Artfcal and Natural Neural Networks, IWANN, Granada, [] Ajth Abraham and Bakunth Nath (000) School of computng & nformaton technology, Monash Unversty, Australa, techncal report seres, -55. [3] Abraham A. (005) Computer Scence Department, Oklahoma State Unversty. [4] Hekk Kovo (000) ANFIS (Adaptve Neuro-Fuzzy Inference System). [5] Jang R. (99) Neuro-Fuzzy Modellng: Archtectures, Analyses and Applcatons. [6] Jyh-Shng Roger Jang (993) IEEE transacton on systems, Man, and Cybernetcs, 3(03): [7] Jyh-Shng Roger Jang and Chuen-Tsa Sun (995) Proceedngs of IEEE, 83, [8] Jan Jantzen (995) An computatonal approach to ntellgence. [9] Fernando J.V., Morgado Das, Alexandre Mota, (004) 5 th WSEAS NNA Internatonal Conference on Neural Networks and Applcatons, Udne, Itala. Bonfo Publcatons 84