FTSM FAST TAKAGI-SUGENO FUZZY MODELING. Manfred Männle

Transcription

1 FTS FAST TAKAGI-SUGENO FUZZY ODELING anfre ännle Institute for Computer Design an Fault Tolerance University of Karlsruhe, D-7628 Karlsruhe, Germany Abstract Takagi-Sugeno type fuzzy moels are wiely use for moel-base control an moel-base fault iagnosis. They provie high accuracy with relatively small an easy to interpret moels. The problem that we aress in this paper is that ata riven ientification of such fuzzy moels is computationally costly. Whereas most ientification algorithms for Takagi-Sugeno moels restrict the moel s generality in orer to simplify the ientification, a ifferent approach is taken here we apply resilient propagation (POP), an efficient nonlinear optimization technique, for parameter ientification in orer to achieve a fast Takagi- Sugeno moeling (FTS) that is suite to moel high-imensional ata sets containing a large number of ata. Keywors fuzzy moeling, ientification algorithms, nonlinear optimization, fault etection. INTODUCTION Takagi-Sugeno type fuzzy moels (Takagi an Sugeno, 985; Sugeno an Kang, 986; Sugeno an Kang, 988; Sugeno an Tanaka, 99; Sugeno an Yasukawa, 993), also being referre to as TSKmoels (after Takagi, Sugeno, an Kang), are wiely use for moel-base control an moel-base fault iagnosis. This is ue to the moel s properties of, on one han being a general nonlinear approximator that can approximate every continuous mapping, an on the other han being a piecewise linear moel that is relatively easy to interpret (Johansen an Foss, 995) an whose linear submoels can be exploite for control an fault etection (Füssel et al., 997; Ballé et al., 997). The generality of TSK-like moels makes the ata riven ientification of such moels very complex. A fuzzy moel consists of multiple rules, each rule containing a premise part an a consequence part. The premise part specifies a certain input subspace by a conjunction of fuzzy clauses that contain the input variables. The consequence part is a linear regression moel. The ientification task inclues two subtasks structure ientification, like etermination of the number of rules an the etermination of the variables involve in the rule premises, an parameter ientification, the estimation of the membership function parameters an the estimation of the consequence regression coefficients. There is a possibility to aress the three ientification tasks separately When fixing the structure an the premise parameters, consequence parameter ientification becomes a linear optimization problem an can therefore be solve by a linear least mean squares optimization like singular value ecomposition (SVD) (Klema an Laub, 98; ännle, 999). Premise parameter optimization remains in any case a nonlinear optimization problem. Finally, structure ientification, when solve exhaustively, is a combinatorial search problem. Because of this complexity, most TSK-ientification algorithms simplify the moel structure or apply heuristics or so-calle meta-optimization techniques like genetic algorithms for structure ientification an (at least for the nonlinear part of) parameter optimization The algorithms LOLIOT an Prouct Space Clustering, as escribe in (Nelles, 999; Nelles et al., 999), o not compute a premise parameter optimization but etermine the structure an premise parameters by either heuristic search or fuzzy clustering

2 of the input-output space. In ANFIS (Jang, 993; Jang an Sun, 995), the moel structure is preefine as a gri partition. Parameter optimization is then performe by a hybri learning rule, a combination of least squares estimate an a graient metho (backpropagation). In (Yen et al., 998), the number of rules create by the gri partition is finally reuce by a pruning algorithm. The NFIN algorithm (Lin et al., 999) in a first step performs a heuristic input space partition (similar to clustering) an then optimizes parameters using backpropagation. Preefining the structure by a gri partition is not suite for high-imensional input spaces, since for a N-imensional input space partitione into K parts at each imension the initial partition contains K N rules which cannot be hanle any more alreay for meium-size problems with, e. g., k 3 an N. We therefore use a bottom up approach yieling a tree partition as escribe in (Sugeno an Kang, 988; Jang an Sun, 995; Nelles et al., 999; ännle, 999). The main problem when applying the heuristic search is its computational cost because a lot of moels must be evaluate. So, the main iea of this work is to apply a sophisticate optimization technique for parameter ientification which enables the use of the heuristic search even when being applie to high-imensional problems. For this purpose, we chose resilient propagation (POP) (iemiller an Braun, 993; Braun an iemiller, 993; Zell et al., 994) because it is easy to apply (only nees first erivatives) but has a performance like secon orer methos as for example the Levenberg-arquart algorithm (Hagan an enhaj, 994). The following sections provie a escription of the fuzzy moel, the ientification proceure (incluing the erivatives necessary to apply POP), the ientification of a nonlinear ynamical process, an a fault etection experiment. 2. FUZZY IDENTIFICATION Fuzzy ientification is one for ISO systems (multiple input single output) system, i. e., the moel performs a mapping ŷ from an N-imensional input vector u (u u N ) 2 U U N I N to an output value ŷ 2 Y I. 2. The Fuzzy oel embership functions of TSK-moels as use here have a trapezoial shape F(u) max(;min(;5 + σ(u ; µ))) () with the parameters µ an σ to be optimize. The parameter µ escribes the location an σ escribes the steepness of the membership function. This type of membership function is piecewise erivable, which is necessary for applying POP. It is also possible to use sigmoial shape membership functions (ännle, 996), yieling comparable results. A fuzzy moel contains D fuzzy sets F ; D. The inex [] 2f Ng enotes the input space imension in which the fuzzy set F is vali. The inex set I r contains the inices of all fuzzy sets that appear in rule r. A fuzzy set is vali in exactly one input space imension an may occur in several rule premises. The inex set J of the fuzzy set F ; D J f j 2 I j ; j g (2) contains the inices of all rules that have F in in their premise. Fuzzy rules may contain full consequences (C N), i. e., a linear equation of the input variables (Takagi- Sugeno type) or simple consequences (C ), i. e., only a constant (Sugeno-Yasukawa type). The consequence parameter vector is either c (c c c N ) or c (c ). The fuzzy rule r has for the empty premise I r / the general form if TUE then f r c r +c r u + + c Nr u {z N } (3) optional an for I r 6 / the form if u ir isf r ananu inrr isf nr r then f r c r +c r u + + c Nr u {z N } (4) optional where f r enotes the consequence of rule r. Finally, the fuzzy moel consists of a set of fuzzy rules r ; r,i.e. f g (5) The membership w r of u m to the rule r is given by ^ w r (u m ) F ir (u [i]m ) (6) i2i r an by choosing the prouct as t-norm we obtain w r (u m ) i2i r F ir (u [i]m ) (7) The normalize membership v r (u) be v r (u) w r(u) (8) w k (u) k Finally, the moel output ŷ(u) is calculate via prouct inference (Larsen) an weighe average by w k (u) f k (u) k ŷ(u) v k (u) f k (u) k w k (u) (9) k

3 2.2 Structure Ientification 3. Symptom Generation Problems of imension N make a bottom up approach necessary. In this paper, a bottom up tree partition algorithm is applie. The optimal structure is etermine by a heuristic search. The structure moeling starts with a one rule moel that is further refine at each epoch by aing one rule, i. e., partitioning one of the moels subspaces. At each epoch, the best partitioning (i. e., the rule to split an the imension where to split) is etermine by evaluation of all possibilities. The best performing moel is then use as starting point for the next epoch. For further etails on the heuristic search the reaer may refer to (Sugeno an Kang, 988; Jang an Sun, 995; Nelles, 999; ännle, 999). 2.3 Parameter Ientification Parameter optimization minimizes the error E 2 which is efine by the Eucliean norm L 2 for the moel output (9) as E 2 2 kε k2 2 2 m y m ; q v q (u m ) f q (u m ) 2 () We also investigate the use of the L norm. This usually results in slightly worse moels, but the moeling is more robust in presence of outliers in the training ata. Ientification of premise an consequence parameters is achieve through POP, a graient escent algorithm that was initially evelope for neural network training. It has a resilient parameter upate step which is base on a local aaption to the topology of the target function (E 2 ). Further etails on POP can be foun in (iemiller an Braun, 993; Braun an iemiller, 993; Zell et al., 994; ännle, 996). In orer to apply POP, one nees to compute the erivatives of all parameters to be optimize, namely the consequence parameters c for all rules ir r an the premise parameters an σ for all fuzzy sets F. The erivatives are given in appenix A.The resulting formulae (A.3), (A.2), an (A.3) show many equal terms which allows an efficient implementation The complexity of one POP iteration for input imension N, rules an patterns is O(N), the same as for a feeforwar step of all patterns! 3. FAULT DETECTION Fault etection is performe in two steps symptom generation an symptom evaluation. There are ifferent ways to gererate symptoms base on TSK-like moels, see for example (Füssel et al., 997; Ballé et al., 997). In this paper, the output errors between moel an process (resiuals) uring (close loop) operation are use as fault symptoms. In orer to isolate single faults in a set of multiple faults it may be necessary to esign symptoms. For this purpose exist several methos, as for example the parity space approach (Füssel et al., 997; Ballé et al., 997). 3.2 Symptom Evaluation The easiest way to etect faults is to efine borers for the resiuals which can be tolerate an to fire an alarm if such a borer is exceee. In orer to make the etection more sensitive while still keeping a low false alarm rate, methos for on-line etection of jumps in means as a moving average filter or a Page/Hinkley etector (Basseville, 986) can be applie. Then, the tolerable eviations can be choosen consierably smaller. To apply a Page/Hinkley etector, one must first efine the two parameters µ inc an µ ec of tolerable eviations (of resiual increase an ecrease). Deviations of an greater than µ will be etecte. The sensitivity of the etector is ajuste by choosing the parameter λ. A bigger λ makes the etection more robust, but also yiels a bigger etection elay. See (Basseville, 986) for further etails. 4. EXAPLE 4. Tank System Ientification In this section a simple nonlinear process is use as an example to show the moeling capabilities of the algorithm an how the ientifie moel can be use for fault etection. valve tank Q in q in phi control Figure. A simple tank system. h pressure Figure epicts a simple tank system. The flui pours out a the rate q out epening on the height h an the (constant) outlet plan a out. The amount q in of flui flowing in epens on the angle ϕ of the valve, which is controlle epening on the flui height, an the (constant) maximal flui Q in. The flui height epens q out

4 on the in/out streams an the tank plane A. The process can be escribe by the following equations The simulation of the training ata is given in figure 4. q in (t) Q in sin(ϕ(t)) ϕ 2 [ π2] () p q out (t) a out 2gh(t) g 98ms ;2 (2) Z t h(t) h()+ (q in (τ) ; q out (τ))τ A (3) For ientification an test, iscrete time ata series were generate with a sampling time of one secon, h() 2m, A m 2, a out m 2, Q in 2m 3 s ;, an an activation ϕ shown in figure Fault Detection Experiment In orer to investigate the fault etection capabilities, a fault injection experiment with two faults is presente () suen harware fault valve leakage of.3 % of Q in from k 25, an (2) rift sensor fault rifting from k 25 to k 35 the pressure sensor measures.75 % less than the real h training test no fault fault fault 2 phi [ra] h [m] Figure 2. Activation ϕ (training an test ata). SE number of rules S Figure 3. oeling error uring ientification. h [m] process moel error Figure 4. Simulation of tank system (training ata). The flui level h at iscrete time k is to be ientifie by the moel, i. e. y(k) h(k). Ientification is one using a series-parallel moel ŷ(k) f (ϕ(k ; ) y(k ; )) (4) Figure 3 epicts the root mean square error of the first five moels, starting with a one-rule linear moel. The accuracy of the five-rule moel is sufficient an ientification can be stoppe there. The simulation is performe with the parallel moel ŷ(k) f (ϕ(k ; ) ŷ(k ; )) (5) Figure 5. Simulation in presence of fault an 2 (test ata). resiual fault Figure 6. esiual of fault (test ata). resiual fault Figure 7. esiual of fault 2 (test ata). Figure 5 epicts the simulation of the test ata uner presence of fault an fault 2. The resiuals shown in the figures 6 an 7, i. e. the eviations from the moel preiction an the real process, are small but still big enough to be reliably etectable through a Page/Hinkley etector. During ientification an tests it is foun that the moeling error remains smaller than 5m uner absence of faults. Therefore, a robust Page/Hinkley

5 Page/Hinkley fault fault 2 Figure 8. Detection of fault an 2 (test ata). etector can be built by choosing the eviations to etect as for example µ inc µ ec 5m. Figure 8 shows the the result of the Page/Hinkley etection. The choice of a borer λ woul reliably etect the faults with a elay of about 2s. 5. CONCLUSIONS TSK-like moels combine the avantages of being general approximators that can reach high accuracy an being easy to interpret, since they are piecewise linear moels that are represente in a quite natural way. Owing to the generality of such moels, the computational complexity of ata-riven ientification is very high. Therefore, the iea was to apply one of the recent powerful nonlinear optimization techniques for parameter optimization. In the presente approach, POP, a sophisticate nonlinear optimization technique, is successfully applie to the problems of premise an consequence parameter optimization. The high efficiency of this parameter optimization allows to apply the original heuristic search for structure ientification, first propose in (Sugeno an Kang, 988). This heuristic is relatively costly with respect to the number of moels to be optimize an evaluate, but is well suite for high-imensional an large problems, since it automatically etermines the most important input variables an yiels well-performing moels with a low number of rules (bottom up approach). Even more exhaustive search strategies show only marginally better results (Johansen an Foss, 995) an o not justify the aitional computational costs. We evelope the ientification proceure FTS to automatically buil TSK-moels base on large an high imensional ata sets. The application to fault etection by the use of resiuals is briefly escribe. The usage of a Page/Hinkley etector in orer to achieve a more sensitive etection is presente an shown by a fault injection experiment with a simple tank system. In our current work we further evaluate the capability of FTS to moel nonlinear processes an investigate methos for fault etection an isolation of multivariable processes. Appenix A. DEIVATIVES FO PAAETE OPTIIZATION From () we get with u i for all consequence parameters c ir, r an i C with c ir yieling m c ir (y m ; ŷ m )(;) m q v q (u m ) f q(u m ) c ir (A.) f q (u m ) c ir u im (A.2) ;ε(u m ) k w k (u m ) w r (u m ) u im (A.3) With () we obtain the partial erivations of the fuzzy set parameters µ an σ for all fuzzy sets F, D as ; ε(u m ) f r (u v r(u m ) m ) m r (A.4) Derivating (8) for all examples u m m, all rules r,r, an all fuzzy sets F, D yiels for r 62 J v r (u m ) an for r 2 J v r (u m ) ;w r (u m ) w j(u m)! 2 j2j (A.5) w q (u m ) q w q (u m ) q! w r (u m ) ; w r (u m ) w q (u m ) q Combining (A.5) an (A.6) we get f r (u v r(u m ) m ) r w r (u m ) r w r(u m) ; ŷ(u m ) r2j w j (u m) j2j! 2 (A.6) f r (u w r(u m) m ) r2j (A.7) Furthermore, from (7) we get for all r an all D w r (u m ) F (u []m ) F i (u [i]m ) i2ir i6 With w r(u m ) F (u []m ) F (u []m ) (A.8) F (u []m ) ;σ u l < u []m < u r else (A.9)

6 an F (u []m ) u[]m ; µ u l < u []m < u r σ else (A.) with the limits u 5 l µ; jσj an we finally obtain for all µ, D m r ε(u m ) w r (u m ) ; r2j f r (u m )w r (u m ) m r w r (u m ) u r µ+ 5 jσj (A.) ŷ(u m ) r2j w r (u m ) ;σ F (u []m ) (A.2) an corresponingly for all σ, D ε(u m ) ŷ(u m ) σ w r (u m ) r2j ; r2j f r (u m )w r (u m ) u []m ; µ Appenix B. EFEENCES F (u []m ) (A.3) Ballé, P., O. Nelles an D. Füssel (997). Fault etection for nonlinear processes base on local linear fuzzy moels in parallel an series-parallel moe. In Proceeings of 4th IFAC Symposium on Fault Detection, Supervision an Safety for Technical Processes (SAFEPOCESS 97). Basseville, ichèle (986). On-Line Detection of Jumps in eans. In Detection of Changes in Signals an Dynamical Systems. pp. 26. Number 77 In Lecture Notes in Control an Information Science. Springer Verlag. Braun, H. an. iemiller (993). prop A fast an robust backpropagation learning strategy. In Proceeings of the ACNN. Füssel, D., P. Ballé an. Isermann (997). Close loop fault iagnosis base on a nonlinear process moel an automatic fuzzy rule generation. In Proceeings of 4th IFAC Symposium on Fault Detection, Supervision an Safety for Technical Processes (SAFEPOCESS 97). Hagan,. an. enhaj (994). Training feeforwar networks with the arquart algorithm. IEEE Transactions on Neural Networks 5(6), Jang, J.-S.. (993). ANFIS Aaptive-networkbase fuzzy inference system. IEEE Transactions on Systems, an, an Cybernetics 23(3), Jang, J.-S.. an C. Sun (995). Neuro-fuzzy moeling an control. In Proceeings of the IEEE. Vol. 83(3). pp Johansen, T. an B. Foss (995). Ientification of non-linear system structure an parameters using regime ecomposition. Automatica 3(2), Klema, V. an A. Laub (98). The singular value ecomposition Its computation an some applications. IEEE Transactions on Automatic Control AC-25(2), Lin, C.-T., C.-F. Juang an C.-P. Li (999). Temperature control with a neural fuzzy inference network. IEEE Transactions on Systems, an, an Cybernetics 29(3), ännle,. (999). Ientifying ule-base TSK Fuzzy oels. In Proceeings of Seventh European Congress on Intelligent Techniques an Soft Computing (EUFIT 99). ännle,.; ichar, A.; Dörsam T. (996). Ientification of ule-base Fuzzy oels Using the POP Optimization Technique. In Proceeings of Fourth European Congress on Intelligent Techniques an Soft Computing (EU- FIT 96). pp Nelles, O., A. Fink,. Babuška an. Setnes (999). Comparison of two construction algorithms for Takagi-Sugeno fuzzy moels. In Proceeings of Seventh European Congress on Intelligent Techniques an Soft Computing (EUFIT 99). Nelles, Oliver (999). Nonlinear System Ientification with Local Linear Fuzzy oels. Dissertation. Techn. Univ. Darmstat, Shaker Verlag, ISBN iemiller,. an H. Braun (993). A irect aaptive metho for faster backpropagation the POP algorithm. In Proceeings of the IEEE Int. Conf. on Neural Networks (ICNN). pp Sugeno,. an G. Kang (986). Fuzzy moeling an control of multilayer incinerator. Fuzzy Sets an Systems 8, Sugeno,. an G. Kang (988). Structure ientification of fuzzy moel. Fuzzy Sets an Systems 26(), Sugeno,. an K. Tanaka (99). Successive ientification of a fuzzy moel an its application to preiction of a complex system. Fuzzy Sets an Systems 42, Sugeno,. an T. Yasukawa (993). A fuzzy-logicbase approach to qualitative moeling. IEEE Transactions on Fuzzy Systems (), 7 3. Takagi, T. an. Sugeno (985). Fuzzy ientification of systems an its application to moeling an control. IEEE Transactions on Systems, an, an Cybernetics 5(), Yen, J., L. Wang an Gillespie C. W. (998). Improving the interpretability of TSK fuzzy moels by combining global learning an local learning. IEEE Transactions on Fuzzy Systems 6(4), Zell, A., N. ache, T. Sommer et al. (994). Stuttgart Neural Network Simulator, Users anual, Version 3.2. University of Stuttgart.