A MODIFIED FUZZY C-REGRESSION MODEL CLUSTERING ALGORITHM FOR T-S FUZZY MODEL IDENTIFICATION

Transcription

1 20 8th International Multi-Conference on Systems, Signals & Devices A MODIFIED FUZZY C-REGRESSION MODEL CLUSTERING ALGORITHM FOR T-S FUZZY MODEL IDENTIFICATION Moêz. Soltani, Borhen. Aissaoui 2, Abdelader. Chaari 3, Faycal. Ben Hmida 4, and Moncef. Gossa 5,2 3,4,5 (moez soltani, borhen.issaoui)@yahoo.fr, (assil.chaari, fayçal. benhmida, moncef.gossa)@esstt.rnu.tn ABSTRACT In this paper, a modified fuzzy c-regression model (FCRM) clustering algorithm for identification of Taagi-Sugeno (T-S) fuzzy model is proposed. The FCRM clustering algorithm has considerable sensitive to noise. To overcome this problem, a modified FCRM clustering algorithm is presented. This latter is based to adding a second regularization term in the alternative optimization process of FCRM. This regularization term is introduce in objective function in order to tae in account the data are noisy. The parameters of the local linear models are identified based on orthogonal least squares (OLS). The proposed approach is demonstrated by means of the identification of nonlinear numerical examples. Index Terms Taagi-Sugeno fuzzy model, Fuzzy c- regression model, Noise clustering algorithm, Fuzzy modeling, Orthogonal least squares.. INTRODUCTION The fuzzy c-regression models algorithm has shown good effectiveness in handling various problems in industry because of their excellent capability of describing complex systems in a human intuitive way. However, the FCRM clustering algorithms in literature are too complicated and suffer from two major difficulties: (a) the algorithm is time-consuming, while the format is complicated and (b) obtaining cluster representatives with weighted recursive least squares (WRLS) requires initials and suffering from no convergence []. In addition, the FCRM is very sensitive to noise, and has considerable trouble in noisy environments. In practice there is noise in the data. To overcome this problem, many studies on the robust fuzzy modeling technique to apply when the noise exist have been reported [2], [3], [4], [5], [6], [7] among them noise clustering (NC) algorithm [8]. Noise clustering algorithm is a prototype-based clustering algorithm and it has a robust capability against Noise. In this paper, we proposed a novel approach termed as the modified fuzzy c-regression model (MFCRM) algorithm for T-S fuzzy modeling. The MFCRM algorithm is modified from the FCRM clustering algorithm proposed by [] but it is also taes into account the data are noisy. On other word, an additional regularization term is used //$ IEEE to reduce the influence of noise in the clustering process. Based on MFCRM, we develop fuzzy modeling approach for unnown nonlinear systems with given samples data, in order to provide good approximation to unnown parameters of the assumed model. The optimal consequent parameters of local linear T-S model are estimated by using orthogonal least squares (OLS) method. This paper is organized as follows: Section 2 recalls the necessary bacground about FCRM clustering algorithm. In Section 3, a modified fuzzy c-regression modeling algorithm considering the noise of input data is proposed. Numerical examples and conclusions are given in section 4 and section 5 respectively. 2. FUZZY C-REGRESSION MODELS CLUSTERING ALGORITHM The Fuzzy c-regression Models clustering algorithm, was introduced by Hathaway and Bezde [9] and belongs to the range of clustering algorithms with linear prototype. The FCRM approach can be viewed as an extension of the fuzzy c-means (FCM) approach. While the FCM algorithm develops hyper-spherical-shaped clusters, the FCRM algorithm develops hyper-plane-shaped clusters. Assume the data S = (x, y ),, (x, y ), ( =,, N) are drown from c different fuzzy regression models, the hyper-plane-shaped cluster representative will be adopted as follows: y = f i (x, θ i ) = a i x a i2 x 2... a im x M b i0 = [x ]. θ T i, i =, 2,..., c () where x = [x,..., x M ] is the system input, M dimension of input vector, c is the number of rules and θ i = [a i,..., a im, b i0 ] is the parameter vector of the corresponding local linear model. The FCRM clustering algorithm is based on the minimization of the sum of weighted squared distances (measure of error) between the data set S and the parameter vector θ i. The square of the distances Ei 2 (θ i) are weighted with the membership values µ i in the objective function that is minimized by the clustering algorithm and formulated as: J(S; U, θ) = (µ i ) m (E i (θ i )) 2 (2)

2 with E i (θ i ) = y [x ]. θ T i (3) where m is the fuzzy weighting exponent and µ i is the fuzzy membership degree the i th data pair belonging to i th cluster.the membership values µ i have to satisfy the following conditions: J NC (U, V ) = (µ i ) m (D i ) 2 δ 2 ( µ i ) m (9) µ i, [0 ]; i =, 2,..., c ; =, 2,..., N (4) < µ i < N; i =, 2,..., c (5) µ i = ; =, 2,..., N (6) The objective function (Eq. (2)) to be minimized subject to constrains defined by Eqs. (4-6), can be solved by using an alternating optimization (AO), which is formulated as follows [9]: Fuzzy c-regression models (FCRM) clustering algorithm: Given data S, set m > and specify regression models (Eq. ()), choose a measure of error (Eq. (3)). Pic a termination threshold ɛ > 0 and an initial partition U (0). Repeat for l =, 2,... Step. Calculate values for c model parameters θ (l) i in (Eq. ()) that globally minimize the restricted function (Eq. (2)). Step 2. Update U (l) with E i (θ (l) i ), to satisfy U (l) i = { c ( E i E ) j m if E i > 0 for i c ; 0 otherwise (7) Until U (l) U (l) ɛ, then stop; otherwise set l = l and return to step THE MODIFIED FCRM CLUSTERING ALGORITHM In this section, a modified clustering algorithm for fuzzy model identification will be discussed in details. Many studies on the robust fuzzy modeling technique have been applied when the noise exist among them noise clustering (NC) algorithm [8]. Noise clustering algorithm is a prototype-based clustering algorithm and it has a robust capability against Noise. In this approach, noise is considered to be a separate class, and is represented by a fictitious prototype that a constant distance δ from all the data point. The membership µ of point x in the noise cluster is given by: µ = µ i (8) Dave s objective function is given by: Here N is the number of data points, m > is weighting exponent and µ i is the membership of x in subspace i. The formulas (Eq. (9)) are derived under the constraint: µ i µ = (0) Now, we combines NC algorithm with FCRM algorithm, then the objective function described by Eq. (2) can be modified as: J new (S; U, θ) = (µ i ) m (E i (θ i )) 2 δ 2 ( µ i ) m () Where δ is a scale parameter and may be used based on the idea presented in [8] as: δ 2 = γ c.n ( N (E i (θ i )) 2 ) (2) where γ is a user-defined parameter depending on the type of data in the example. The success of MFCRM depends on the appropriate choice of the noise distance δ. If δ is large, the number of data points in the noise cluster becomes small. On the other hand, if δ is too small, a lot of number of data points can be considered as noise and misplaced into the noise cluster. To minimize J new (Eq. ()) an alternative optimization scheme can be designed as FCM and fuzzy NC algorithm. To solve constrained problem J new with respect to µ i, we introduce n Lagrange multipliers λ, =,..., N. The minimization of J new, it follows as: F (µ i, λ ) = J new λ ( µ i µ ) (3) By differentiating the Lagrangian with respect to the µ i, µ and λ and setting the derivates to zero, it follows as: F µ i = m (µ i ) m (E i ) 2 λ = 0 (4) F µ = m δ 2 (µ ) m λ = 0 (5) F λ = µ i µ = 0 (6)

3 From Eq. (4), we get: And from Eq. (5) µ i = [ λ m ] m [ (E i ) 2 ] m (7) µ = [ λ m ] m [ δ 2 ] m (8) Using Eq. (6), Eq. (7) and Eq. (8), we get: [ λ m ] m = c j= ( E j ) 2 m ( δ ) 2 m (9) And then substituting it into Eq. (7), the following equation can be obtained: µ i = c j= ( E i E j ) 2 m ( E i δ ) 2 m (20) From Eqs. () and (), the objective function of modified FCRM is defined as: J new (S; U, θ) = (µ i ) m (y [x ]. θi T ) 2 = δ 2 ( µ i ) m M (µ i ) m (y θ ij ˆx j ) 2 δ 2 ( j= µ i ) m (2) where ˆx = [x ]. The partial derivation of Eq. (2) is: J new θ ij M = 2 (µ i ) m (y θ it ˆx t )ˆx j (22) t= And then, θ ij = (µ i) m (y t j θ it ˆx t )ˆx j (µ i) m ˆx 2 j i =, 2,..., c; j =, 2,..., M. (23) Based on the optimization conditions (20) and (23), the identification algorithm for MFCRM via iterative optimization is now given as follows: Modified Fuzzy c-regression models (MFCRM) clustering algorithm: Given data S, set m > and specify regression models (Eq. ()), choose a measure of error (Eq. (3)) and distance δ > 0. Pic a termination threshold ɛ > 0 and an initial partition U (0). Repeat for l =, 2,... Step. Compute measure of error E i (θ i ) via Eq. (3). Step 2. Compute µ (l) i and θ(l) ij via Eqs. (20) and (23) respectively. Step 3. Compute err = U (l) U (l). Until err ɛ, then stop; otherwise set l = l and return to step. 3.. Identification of premise and consequent parameters We use the novel fuzzy c-regression models for decomposition of the input-output space into multiple linear structures. Gaussian membership functions are usually chosen to represent the fuzzy sets in the premise part of each fuzzy rule. As mentioned in [0] and [], the premise parameters can be easily obtained using µ i. The fuzzy sets centers ν i and the standard derivation σ i are calculated as follows: ν ij = µ i x j µ, i =, 2,..., c; j =, 2,..., M. i (24) and σ ij = 2 µ i (x j ν ij ) 2 µ (25) i Once the premise parameters have been fixed, the consequent parameters for each rule can be obtained using orthogonal least squares (OLS) [], [2], [3] and []. Using OLS, the consequent parameters are estimated by transforming model (Eq. ()) into an equivalent auxiliary model: y = M p j ()θ j e(),, =, 2,..., N (26) where y is th system output, p j () and θ j are nown as regressors and parameters respectively. We arrange Eq. (26) in the following matrix form: Y = P Θ E (27) where Y = [y,..., y N ] T, P = [p,..., p M ] with p i = [p i (x ),..., p i (x N )], Θ = [θ,..., θ M ] T and E = [e,..., e N ] T. The OLS algorithm is described as follows [2]: Step. For i N, compute w (i) = p i, g (i) = Find (w (i) )T Y (w (i) )T w (i) [err] (i) = (g(i) )2 (w (i) )T w (i) Y T Y (28) (29) [err] (i) = max([err] (i), i N) (30) and select w = w (i) = p (i) ; g = g (i) (3) Step 2. For 2 M, for i N,i i,..., i i, compute α (i) j = wj T p i, j < (32) w T j w j

4 [err] (i ) Find = p i α (i) j w j (33) w (i) j= g (i) = (w (i) )T Y (w (i) )T w (i) [err] (i) = (g(i) )2 (w (i) )T w i Y T Y (34) (35) = max([err] (i), i N, i i,..., i i ) (36) and select w = w (i ) ; g = g (i ) (37) Step 3. Solve the triangular system A Θ = g, where α i2 2 α i3 3 α i M M 0 α i2 23 α i M 2M A = (38).... α i MM,M and g = [g,, g M ] T (39) 4. NUMERICAL EXAMPLES In this section, we will present the simulation results to examine the performance of the MFCRM clustering algorithm and to verify the algorithm developed above. In this paper, root mean square error (RMSE), mean square error (MSE) and the computational time (CT ) of all algorithms were used as the performances index (PI), with RMSE and MSE are defined as: RMSE = (y ŷ ) 2 (40) N MSE = N (y ŷ ) 2 (4) All algorithms have been implemented on a PC Core Duo, 2.2 GHz with 2 Giga RAM, using MATLAB 7. as a programming language. 4.. A bench mar problem We consider the nonlinear system given in equation Eq. (40) [7] which used as a test for identification techniques proposed in this paper, to demonstrate the effectiveness of the proposed algorithm in a noisy environment. y = y (y 2) (y 2.5) 8.5 y 2 y2 u v (42) where y is the output, u is the input which is uniformly bounded in the region [ 2, 2] and v is a white noise with zero mean and unit variance, which is added to the process output at different SNR levels (SNR=0 and 30 db). The noise influence is analyzed according to the difference between the outputs of the fuzzy models, obtained from the noisy experimental data, and the output of the plant without noise for the different algorithms. We simulated two experimental cases: case and case 2. In case, the 500 data pairs are used as training data and 000 data pairs testing data for the plant without noise. The input applied for the test data is: u = sin( 2 pi 250 ) if <= sin( 2 pi 2 pi 250 ) 0.2 sin( 25 ) otherwise (43) We choose y(-), y(-2), u(), u(-) as the variables of the fuzzy model, while the rule number of fuzzy model is four. Table compares our result with result obtained with different algorithms such as: Gustafson-Kessel (GK) [4], new FCRM (NFCRM) [], FCM [5] and fuzzy model identfication (FMI) clustering algorithm [6]. Table. Comparison Results (Case ). PI RMSE T r RMSE T s CT T r (s) Algorithms FCM GK FMI NFCRM MFCRM with RMSE T r and CT T r are the RMSE and the CT for training data and RMSE T s is the RMSE for test data. The comparison results demonstrate that the computational time of the proposed method is reasonable taing into account the best RMSE obtained by proposed method. In the absence of noise, the positive scalar parameter δ can be regarded as a regulatory factor to reduce the sensitivity of the model to the identification data. We can also note that the addition of the positive scalar parameter δ has reduced the computational complexities of algorithm NFCRM. In case 2, the noise influence is analyzed with different SNR levels (SNR=0 and 30 db). Table 2. Comparison Results with SNR=30 db (Case 2). PI RMSE Algorithms T r RMSE T s CT T r (s) FCM GK FMI NFCRM MFCRM

5 Table 3. Comparison Results with SNR=0 db (Case 2). PI Algorithms RMSE T r RMSE T s CT T r (s) FCM GK FMI NFCRM MFCRM output y is the CO2 concentration in the outlet gases. In order to tae all mentioned issues into account, we simulated all the 296 data pairs as training data and y(-), u(-4) are selected as input variable to MFCRM. After extended run, the mean square error equal to was achieved. Table 4 provides a comparison analysis of the performance obtained in this case with the respective performances of other models [3]. It can be seen from Table 2 and 3 that compared with these existing techniques, not only is the MFCRM algorithm gives the best performance for different noise contents, but it is also has the smallest RMSE. The second regularization term adding in the objective function of FCRM mae it less sensitive to noise data sets. Thans to the MFCRM algorithm, the problem of time-consuming of FCRM algorithm is reduced. Fig. (a) shows original data and the identified data obtained using MFCRM for the testing data set, and Fig. (b) exhibits the respective errors. The corresponding RMSE for the training data was equal to Fig. (c) shows original data and the identified data obtained using NFCRM algorithm for the testing data set, and Fig. (d) exhibits the respective errors. Table 4. Comparison results for the Gaz Furnance process identification. No. of No. of MSE Inputs rules Tong (80) Pedrycz (84) Xu (87) Yoshinari (93) Joo (97) Chen (98) Delgado (99) Liu (02) Zhang (06) Glowaty (08) Soltani et al. (0) [3] MFCRM without noise MFCRM with SNR=30db MFCRM with SNR=20db MFCRM with SNR=0db MFCRM with SNR=5db MFCRM with SNR=db (a) (b) As seen from table 4, our method gives the best MSE performance of all compared techniques. We can also note that the addition of the positive scalar parameter δ in the objective function of FCRM has reduce the sensitivity of the model to the identification data especially in absence of measurement noise and provided an acceptable accuracy. (c) Figure. MFCRM and NFCRM performances for test data with SNR=0 db. From Figure we can see that our algorithm gives the best accuracy model against other existing algorithms in the literature Box-Jenins system We consider the Box-Jenins gas furnace data set [8] which used as a standard test for identification techniques. The data set consists of 296 pairs of input-output measurements. The input u is the gas flow rate into a furnace; the (d) 5. CONCLUSIONS In this paper, modified fuzzy c-regression model clustering algorithm has been presented. This new algorithm allows the identification of the premise and consequence parameter via iterative minimization using new objective function. The proposed approach is not only to simultaneously identify fuzzy the premise and consequent parameters but also have robust learning effects when the noise exist. The results show the high accuracy of our algorithm for the problem of identification of nonlinear complex systems. The proposed approach provide a good performance modeling and is able to handle various types of modeling problems without and with noisy data.

6 6. REFERENCES [] C. Li, J. Zhou, X. Xiang, Q. Li, and X. An, T-S fuzzy model identification based on a novel fuzzy c- regression model clustering algorithm, Engineering Applications of Artificial Intelligence, vol. 22, pp , [2] R. N. Dave and R. Krishnapuram, Robust clustering methods: a unified view, in IEEE Transactions On Fuzzy Systems, vol. 5, no. 2,pp , 997. [3] S. Moez, C. Lotfi, C. Abdelader, B. faycal and M. Gossa, Identification of nonlinear complex systems using uncoupled state fuzzy model for liquid level process, in International Review of Automatic Control, vol. 3, no. 5, pp , 200. [4] H. Ichihashi, K. Honda, and W. Noboru, Robust PCA with intra-sample outlier process based on fuzzy Mahalanobis distances and noise clustering, in IEEE International Conference On Fuzzy Systems, Reno, Ny, 2005, pp [5] H. Ichihashi and K. Honda, On parameter setting in applying Davé s noise fuzzy clustering to Gaussian Mixture models, in Proc. IEEE International Conference On Fuzzy Systems, Budapest, Hungary, 2004, vol. 3, pp [3] X. F. Wu, Z. Q. Lang, and S. Billings, An orthogonal least squares based approach to FIR designs, International Journal of Automation and Computing, vol. 2, no. 2, pp , [4] D. E. Gustafson, and W. C. Kessel, Fuzzy clustering with a fuzzy covariance matrix, in Proc. IEEE Conference On Decision Control, vol. 2, pp , 979. [5] F. Hoppner, F. Klawonn, R. Kruse, and T. Runler, Fuzzy cluster analysis, methods for classification, data analysis and image recognition, John Wiley and Sons, NJ, 999. [6] J. Q. Chen, Y. G. Xi, and Z. J. Zhang, A clustering algorithm for fuzzy model identification, Fuzzy Sets and Systems, vol. 98, pp , 998. [7] B. Subudhi, and D. Jena, A differential evolution based neural networ approach to nonlinear identification, Applied Soft Computing, vol., pp , 20. [8] G. E. P Box, G. M. Jenins, Times series analysis, Forecasting and Control, Holden Day, San Francisco, 970. [6] X. H. Wu, and J. J. Zhou, Alternative noise clustering algorithm, in 8th International Conference On Signal Processing, Beijing, 2006, vol. 3. [7] S. Moez, C. Abdelader, B. Fayçal, and G. Moncef, Modified fuzzy model identification clustering algorithm for liquid level process, in 8th Mediterranean Conference on Control and Automation, Marraech, Maroc, 200, pp [8] R. N. Dave, Characterization and detection of noise in clustering, Pattern Recognition Letters, vol. 2, no., pp , 99. [9] R. J. Hathaway, J. C. Bezde, Switching regression models and fuzzy clustering, IEEE Transactions On Fuzzy Systems, vol., no. 3, pp , 993. [0] E. Kim, M. Par, S. Ji, and M. Par, A new approach to fuzzy modeling IEEE Transactions on Fuzzy Systems, vol. 5, no. 3,pp , 997. [] S. Chen, S. A. Billings, and W. Luo, Orthogonal least squares methods and their application to nonlinear system identification International Journal of Control, vol. 50, no. 5, pp , 989. [2] L. X. Wang, J. M. Mendel, Fuzzy basis functions, universal approximation, and orthogonal least-squares learning IEEE Transactions on Neural Networs, vol. 3, no. 5, pp , 992.