Cluster Centric Fuzzy Modeling

Transcription

1 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 1 Cluster Centri Fuzzy Modeling Witold Pedryz, Fellow, IEEE, and Hesam Izakian, Student Member, IEEE Abstrat In this study, we propose a luster oriented development of fuzzy models. An overall design proess is foused on an effiient usage of fuzzy lustering, Fuzzy C Means (FCM), in partiular, to form information granules lusters used in the onstrution of the fuzzy model. Fuzzy models are regarded as mappings from information granules expressed in the input and output spaes. This position motivates us to look at the development of the models through the perspetive of the onstrution and effiient usage of information granules. The study diretly assoiates fuzzy lustering with fuzzy modeling both in terms of oneptual and algorithmi linkages. The augmented FCM method is formed predominantly for modeling purposes so that a balane between the strutural ontent present in the input and output spaes is ahieved and in this way the performane of the resulting fuzzy model is optimized. It is shown that the luster oriented modeling gives rise to the Mamdani like fuzzy rules and a zero order Takagi Sugeno model (under a ertain deoding sheme). We identify an interesting and diret linkage between the developed fuzzy models and a fundamental idea of enoding deoding (or granulation degranulation) enountered in proessing fuzzy sets and Granular Computing, in general. Further refinements of zero order fuzzy models are investigated leading to first order fuzzy models with linear funtions standing in the onlusions of the rules. A series of experiments is reported where we used syntheti and real world data in whih an issue of generalization apabilities is elaborated in detail. Index Terms fuzzy modeling, information granules, Fuzzy C Means, augmented lustering, rule based model F I. INTRODUCTION UZZY models and fuzzy modeling dwell upon a onept of information granules fuzzy sets. Fuzzy sets form a blueprint of any fuzzy model and ontribute to the key features of fuzzy models suh as their interpretability. Fuzzy lustering as an essential mehanism of building information granules plays a pivotal role in fuzzy modeling. Fuzzy C Means (FCM) [4] is well doumented as a design vehile of fuzzy sets. In fuzzy modeling, espeially in rule based models, fuzzy sets appear always in the ondition and Manusript reeived June 10, 013. This work was supported in part by Natural Sienes and Engineering Researh Counil of Canada (NSERC), the Canada Researh Chair Program, the Alberta Innovates Tehnology Futures and Alberta Advaned Eduation & Tehnology. W. Pedryz is with the Department of Eletrial and Computer Engineering, University of Alberta, Edmonton, AB, Canada, TG V4, with the Department of Eletrial and Computer Engineering Faulty of Engineering, King Abdulaziz University, Jeddah 159, Kingdom of Saudi Arabia, and with the System Researh Institute, Polish Aademy of Sienes, Warsaw 00-71, Poland ( wpedryz@ualberta.a). H. Izakian is with the Department of Eletrial and Computer Engineering, University of Alberta, Edmonton, AB, Canada, TG V4 (e -mail: izakian@ualberta.a). sometimes in the onlusion parts of the rules. When being onsidered diretly in the setting of fuzzy models, fuzzy lustering leads to the formation of fuzzy sets in the input and output spae. A onstrution of these fuzzy sets has to be ompleted having in mind their role being played in fuzzy models. In other words, one has to be aware that there is a mapping from the input to the output spae and fuzzy lusters are involved in the realization of this granular mapping. The input and output spae (and fuzzy sets therein) are to be onsidered together with some provisions to inorporate some flexibility to lustering being ognizant of the diretionality of the mapping under onstrution. There are several essential ontributions of this study and those along with well-motivating fators all together bring forward a ertain faet of originality: We learly identify a role of information granules fuzzy sets in the onstrution of fuzzy models. In this ontext, it beomes imperative to assoiate a way in whih information granules are onstruted with an integral role they play in the formation of the model. This linkage is revealed and diretly exploited in the re-formulation of the objetive funtion used in lustering. The modified objetive funtion brings an important feature of diretionality to the lustering proess. Clustering, in its virtue, is inherently diretion-free, viz. no distintion is being made among input and output variables when forming lusters. By introduing a arefully strutured distane funtion, we artiulate relationships between information granules formed in the spaes of input and output variables. We reall a fundamental onept of information granulation-degranulation [5][][7] and demonstrate that this idea is exploited in Mamdani-like fuzzy models. In essene, fuzzy models an be sought as onstruts that are formed diretly on a basis of linkages between information granules. Further refinements of fuzzy models are realized through a Taylor-like expansion where onlusions of the rules are ompleted around linearization points being the prototypes of the lusters. This makes the resulting refined onstruts easily interpretable. Proeeding with more details, we propose a luster entri development methodology of fuzzy models. We develop a detailed design proess involving an augmented FCM algorithm endowed with an ability to onstrut interrelated lusters that effetively realize granular mapping. From a funtional point of view, the luster entri models realize a Mamdani like fuzzy models or are equivalent to the zero order Takagi Sugeno (T S) fuzzy models (assuming a ertain format of the deoding defuzzifiation sheme). From the design perspetive, the proess revolves around () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

2 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 fuzzy lusters and the augmented lustering tehnique is optimized so that the resulting lusters lead to enhaned approximation and generalization features of the ensuing rules. The proposed design approah promotes a luster based approah. The model beomes ompleted one the lusters have been formed and, in priniple, no further optimization is required. If neessary, the model ould be refined the study elaborates on a suessive generalization of the model where the onstant onlusion parts are replaed by linear funtions or polynomials. Interestingly, the linear (or polynomial) relationships are positioned around the prototypes of the lusters onstruted in the input spae (meaning that the onlusion parts are easy to interpret) [9]. A suite of experiments reported in the study omes with two objetives. First, our objetive is to present step by step the design proess and quantify the performane of the model. Seond, we ontrast the harateristis of the proposed fuzzy model with other models espeially with regard to their generalization abilities (one may antiipate that the generalization aspets ould be substantial onsidering that the design is onentrated at the level of information granules rather than individual numeri data). This study is organized as follows. We start with a brief review of the pertinent literature. The overall struture of fuzzy models onsidered in this study is disussed in Setion 3. In Setion 4, a luster driven development of fuzzy models is proposed and extended. In Setion 5, two design perspetives of fuzzy models have been elaborated on. In Setion, experimental results dealing with syntheti and real world data are reported, and Setion 7 onludes this paper. II. CLUSTERING IN FUZZY MODELING AN OVERVIEW Clustering and fuzzy lustering play an important role in fuzzy modeling [9][11][1][9][44][45] and fuzzy ontrol [4][47]. The results of lustering information granules (fuzzy sets) are instrumental in the formation of a blueprint (oneptual skeleton) of the model. Fuzzy lusters ould be further refined as funtional modules of the model and in this way are essential to the improvement of the performane of the model. There are literally many variants of lustering algorithms and to a signifiant extent they are analyzed as assessed as standalone onstruts aimed at the analysis of data rather than being evaluated in the ontext of their funtionality as omponents of fuzzy models. Having this in mind, we identify a series of main features of lustering and their results ast in the setting of fuzzy models so that it beomes more apparent to understand role and ontributions to building fuzzy models. Fuzzy lusters in arhitetures of fuzzy models arhitetural omponent: In fuzzy models information granules are inherently visible in the input spae as well as in ase of Mamdani-type models in the output spae. In other words, we are onerned about fuzzy rule-based models where the rules are of the form if x is A i then y = f i (x, a i ) (Takagi- Sugeno model with a funtional form of the rules) and if x is A i then y is B i. Here A i and B i are the orresponding information granules (fuzzy sets) formed in the input and output spaes, respetively. The evident riterion using whih ertain taxonomy is being built revolves around the aspet of loalization of information granules. There ould be alternatives where input data are subjet to lustering. There are ases where the data in the onatenated input-output spae are lustered. There are options where separate lustering is realized for the input and output data. In this ase the number of lusters formed in the input and output spae ould be different. TABLE I A FOCUSED OVERVIEW OF FUZZY CLUSTERING, DESIGN ASPECTS, AND KEY FEATURES STUDIED IN THE CONTEXT OF FUZZY MODELING Arhitetural omponent Geometry of fuzzy lusters Constrution of fuzzy lusters Optimization and further refinements Clusters formed individually in the input and output spae [][5][31][33] Clusters formed in the output spae and lusters in the input spae are developed by projeting them onto individual input variables [3] Clusters formed in the input spae [4][41][43] Clustering realized in the ombined (onatenated) input - output data [7][] [30][34][35][3][4] Clusters onstruted using Fuzzy C-Means equipped with the Eulidean distane funtion; spherial shape of lusters [][3][4][7][35][37][41]; sometimes post-proessing is realized suh as Karhunen-Loeve [31] resulting in modified ondition part Gustafson-Kessel modifiation of the FCM; hyper-ellipsoidal shape of lusters [5][4] Fuzzy linear varieties [] and -hyperplanes [33][39] ART-like lustering with its ensuing geometry of lusters [31] Gath-Geva lustering method [33]; use of the Mahalanobis distane Geometry affeted by the type of fuzzy funtion used in the objetive funtion [3] Hierarhial lustering [41][43] The hoie of the number of fuzzy lusters by the objetive funtion [][4] [43][][30][31][33][3][37][39][4] Cluster validity riterion defined in the input spae [5][41] Cluster validity index defined in the input-output spae [35] No further expliitly artiulated optimization tasks assoiated with fuzzy lustering [][3][4][5][7][][30][31][33] [34][35][39][41][4][43] Optimization linked with the struture in the joint input-output spae [3] or the spae of [37] Geometry of fuzzy lusters: In light of the diversity of geometry of lusters (quite often resulting from the use of more advaned objetive funtions, say those enountered in kernel-based lustering), the role of this more advaned geometry of lusters in the formation of rules and assessing their usefulness beomes important. Along this line, questions about the usefulness or relevane of suh geometry of onditions of the rules in fuzzy models are worth posing. Evaluating the linkages between the sophistiation of the ondition part and the funtional part (loal funtional onlusions) is an important design aspet of the overall modeling proess. In other words, one may study how more advaned geometry of lusters in the input spae may help redue the omplexity of the loal models (onlusions), so instead of polynomials one an onsider linear or onstant funtions. Constrution of fuzzy lusters along with assoiated evaluation mehanisms: Fuzzy lusters are results of the lustering algorithm, namely, a minimization of a ertain objetive funtion. The objetive funtion is not diretly linked to the optimization riterion of the fuzzy model () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

3 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 3 Likewise we enounter a large number of luster validity indexes those ome with some well defined rationale however none of them assoiates diretly with fuzzy models. The questions essential to be asked here are about the relationship of the objetive funtion used in lustering and the performane index of the fuzzy model are those two related, ompletely disjoint or somewhat related? Optimization and further refinements of fuzzy lusters in the framework of fuzzy models: Here a list of items under onsideration is revolved about optimization mehanisms employed to refine fuzzy lusters one they have been inorporated as the building bloks of the fuzzy model. Are the fuzzy lusters left intat? Are they subjet to additional refinements/adjustments in order to improve the performane of the fuzzy model? If these optimization ativities are sought, how are they realized? The above features are benefiial to look at the plethora of algorithms of fuzzy lustering in a systemati way when studying and assessing their role in fuzzy models and fuzzy modeling. Table 1 has been onstruted by taking the above features into aount. This perspetive helps us build a systemati and oherent piture on the dominant diretions already established in the existing studies. III. AN OVERALL ARCHITECTURE OF THE FUZZY MODEL AND ITS UNDERLYING CONCEPTS In this study, we onsider a lass of Takagi Sugeno (T S) fuzzy models [1]. The struture of the rules forming this model omes in the form: R i : If x 1 is A 1 i, x is A i,, xd is Adi then y is f i ( x1, x,..., xd ), i=1,,, (1) where x 1, x,, x d are input variables, y is output variable, A 1 i, A i,, A di are fuzzy sets formed in the spae of input variables, stands for the number of rules in the system, and f i is a funtion of input variables. As fuzzy lustering is being used to onstrut onditions of the rules, the form of the rules (1) is slightly different as no fuzzy sets are expliitly present there; instead we enounter membership funtions of the lusters A i defined in the multidimensional input spae. When the onlusion parts f i, i=1,,,, are onstant ( viz. fi w i ), the model is referred to as a zero order fuzzy model. One an show that it is equivalent to the Mamdani fuzzy rule based model [40] assuming a ertain deoding (defuzzifiation) sheme. For some input x [ x1, x,..., xd ], the output of the model is determined by taking a weighted average of the onlusions (loal funtions) as follows: Ai ( x) fi ( x) yˆ i1 () Ai ( x) i1 where Ai (x) is the ativation level of the ith rule, R i, omputed in the form Ai ( x) min( Aji ( x j )), j 1,,..., d. Introduing j the notation Aˆ i( x ) Ai ( x) A i 1 i( x), one rewrites () in a more onise way as yˆ ˆ A i( x) f i ( x). (3) i1 Assuming that there is a set of input output data, to establish a fuzzy model, one has to deal with identifiation of struture and estimation of parameters of premise and onlusion parts of rules [1][][3]. IV. CLUSTER DRIVEN DEVELOPMENT OF FUZZY MODELS Let us onsider a set of input output data oming in d form ( x k, y k ), k 1,,... N, x k R, and y k R. The objetive is to onstrut a fuzzy model produing an aurate estimate of the output. Assuming that ŷk is the result produed by the fuzzy rule based model for kth output, the performane index is expressed as a mean squared (MSE): N 1 y k yˆ k. (4) N k 1 We briefly review a standard Least Square Error (LSE) tehnique along with fuzzy lustering to onstrut a zero order Takagi Sugeno fuzzy model. Next, we propose a luster based fuzzy model, and finally expand the onstruted zero-order models to the first order models. A. Least Square Error (LSE) in the onstrution of the zero order T S fuzzy models In this method, the input data xk, k 1,,... N are lustered using a Fuzzy C Means (FCM) lustering algorithm. This lustering tehnique reveals a struture in the input data by forming a set of prototypes v 1, v,..., v and a fuzzy partition matrix U [ u ik ], i=1,,,, k=1,,,n, where N u ik [0,1], u k i ik 1, and 0 u N i 1 k ik. This 1 struture arises through the minimization of the following objetive funtion N m J u ik vi xk (5) i1 k 1 where m ( m 1) stands for a fuzzifiation oeffiient. This objetive funtion is minimized in an iterative fashion through suessive updates of the prototypes and the partition matrix. One lustering of input data has been ompleted, we obtain a set of prototypes assoiated with the orresponding loal models. The aggregation of the loal models is realized in the form: yˆ k Aik. wi, k=1,,,n () i1 m m where Aik uik u i 1 ik. The onstants standing in the onlusion parts w 1, w,..., w need to be estimated to minimize a ertain predefined performane index ; see (4) () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

4 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 4 Let us organize the ativation levels of the rules in a matrix form A11 A1 A1 A A A A 1 (7) AN1 AN AN Minimizing the sum of the squared s, we estimate the weights w 1, w,..., w by using the well known expression W ( T 1 T A A ) A Y () T where Y [ y 1, y,..., y N ], and T W [ w1, w,..., w ]. For the first order Takagi Sugeno model, the estimation proedure is the same as for the zero order with an exeption that instead of onstants w 1, w,..., w, the linear funtions f 1, f,..., f have to be onsidered in the onlusion parts. B. An augmented fuzzy lustering in the onstrution of the zero order T S fuzzy models While the LSE tehnique desribed in the previous subsetion uses the already formed results of fuzzy lustering to partition the spae of the input variables, the method proposed here is onerned with the formation of lusters in the omposite input output spae. One the lustering of this nature has been realized, the essene of the approah is to estimate the output data using the available struture disovered within the input part. For this purpose, the strutures in the input spae and the output spae are balaned and arefully adjusted so that the mapping from information granules (lusters) in the input spae to the lusters in the output spae attains the highest approximation apabilities (the lowest approximation ). We luster the input output data using an augmented version of the FCM through the minimization of the following objetive funtion: N m J uik v i xk wi yk, 0. (9) i1 k 1 The objetive funtion in (9) uses a omposite (augmented) Eulidean distane, wherev i is ith luster prototype orresponding to the input part of the data, and wi stands for ith luster enter formed for the output part. The parameter ontrols the effet of eah part of data in the formation of information granules lusters. Setting =0, removes the impat of output data in the formation of the struture and only the input data are onsidered in lustering. Assigning higher values to this ontrol oeffiient inreases the effet of the output data (and the struture therein) in the lustering proess. Consider a ertain predetermined value of the oeffiient. As usual, the objetive funtion in (9 ) is minimized by determining luster enters (prototypes) and the partition matrix. The optimization is arried out in an iterative fashion by using the following formulas N m u k ik xk v 1 i (10) N m u k 1 ik N k m u ik yk w 1 i (11) N m u k 1 ik 1 uik. (1) 1/( m1) vi xk wi yk j1 v j xk w j yk At the next step (when forming the fuzzy model itself), for the input data xk and the already determined prototypes in the input spae, the entries of the partition matrix are expressed in the form u ~ 1 ik. (13) 1/( m1) vi xk j1 v j xk Note that the distane shown in (13) is omputed for the vetors present in the input spae only. Then using the luster enters positioned in the output spae, w 1, w,..., w, and the new partition matrix U ~, the output is estimated by minimizing the following sum of squared distanes: N ~ m F uik wi yˆ k (14) i1 k 1 where ŷ k is the estimated value of y k. By zeroing the gradient of F with respet to ŷ k, we have: m u ~ ik wi yˆ i1 k. (15) m u ~ ik i1 Evidently plays a pivotal role in realizing lustering of the input output data by balaning the struture in the input and output spae. The value of this parameter is optimized by minimizing the performane index (4). C. The design of first order T S model: further refinements of the model Zero order fuzzy models are readable (interpretable) and diretly assoiated with the linguisti interpretation [15]. In general, higher auray is gained by forming higher order loal models standing in the onlusion parts of the rules. Typially, we an talk about linear models or polynomials. Our objetive is to refine the zero order fuzzy model by raising its order. In the realization of the onstrut, we follow a generi idea of a Taylor expansion of a funtion around a given point speified in the input spae. Let us reall the following well known linearization relationship: f i ( x) fi ( x 0 ) fi ( x x 0) (1) x0 T where x [ x1, x,..., xd ]. The loal linear funtion omes in the form f i ( x1, x,..., xd ) = wi a1i x1 aix... adixd where a1i, a i,, a di, i=1,, are the parameters of the () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

5 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 5 model whose values need to be estimated. In the above expression the point around whih we expand the funtion is v i, namely x 0 = v i. The value of the funtion at this point is equal to w i, whih has been determined when designing the zero order fuzzy model. In light of these observations we rewrite (1) as follows x1 v1i x v f i i(x ) wi [ a1 i, ai,..., adi ] (17) xd vdi The objetive is to estimate the values of the vetor T a [ a1 i, ai,..., adi ], i=1,,, to minimize the performane index (4). By onsidering the partition matrix U ~ and defining m m Aik u ~ ik u i 1 ik, the output of the fuzzy model for input x k is estimated using yˆ k Aik fi( xk ). (1) i1 Defining T yˆ [ ˆ1, ˆ,..., ˆN ] and introduing some auxiliary notation T T T z z z T T T Z z z 1 z, T T T z z z 1N N N (19) where T z Aik ( xk vi) and ik [ 1,,..., N ] hk Aik wi, we have i1 yˆ h Z a (0) Minimizing the standard squared we have T -1 T aopt ( Z Z) Z ( y h), (1) T where y [ y 1, y,..., y N ]. V. A DESIGN PERSPECTIVE: OPTIMIZATION CONSIDERATIONS The lustering method inorporating input output data with the augmented distane endowed with the adjustment weight,, forms a rux of the overall design proess. By starting with the zero order fuzzy model, we build the first order model. Subsequently higher order models an be formed in the same manner by onsidering polynomials of higher order. When working with the first order fuzzy models, two design strategies are envisioned as shown in Fig. 1. In the first approah, Fig. 1, we onstrut the zero order model and at this stage the value of the parameter is optimized (see the feedbak loop shown in the figure). In the sequel, first order fuzzy model is formed. In ontrast, as illustrated in Fig. 1, the zero order model is formed, afterwards refined by building the first order model and at this stage the value of is optimized; note a feedbak loop shown in Fig. 1. Data Data FCM FCM v i w i v i w i Fuzzy model of zero order Fuzzy model of zero order Fuzzy model of first order Fuzzy model of first order Fig. 1. Two design strategies in the onstrution of first order fuzzy models We have proposed an augmented fuzzy lustering of input output spae of data to onstrut zero order and first order fuzzy models. The approah advoated here leads to the generation of a set of luster enters in the input spae and a set of weights in the output spae. One may onstrut a set of rules with the expliitly enumerated input variables by projeting eah prototype of the luster on the individual input variables. By doing this, the projeted prototypes distributed for eah input variable an be ordered in a linear fashion and by doing this we an assoiate with some linguisti terms suh as low, medium, high, et. VI. EXPERIMENTAL STUDIES In this setion, we present a series of experiments involving syntheti low dimensional data as well as real world data. We also report on the detailed omparative studies by ontrasting the performane of the proposed model with the performane of the fuzzy model disussed in the literature. A. Syntheti data A 00 two dimensional input data set x 1, x,..., x00 is generated over [ 0,10] [0,10] using a uniform distribution. Five prototypes in the input spae and their orresponding onstant values positioned in the output spae are set as v 1 [1.5, 0.5], v [1, 4], v 3 [4, ], v 4 [,.5], v 5 [,.5], and w 1 1, w 0. 5, w 3 3, w 4, and w 5 5, respetively. Fig. visualizes the input data along with the luster enters. Using the prototypes in the input spae and the output onstants, we generate the output data as follows m uik wi y i1 k () m uik i () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

6 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 where data. m=.0. Fig. 3 illustrates the generated input output Fig.. Two dimensional input data of the generated syntheti dataset Fig. 3. Generated syntheti input output data Given the data generated in this manner, we form a olletion of data sets that are affeted by noise of different intensity (standard deviation). The output is expressed as yk yk fk, k=1,,.00, and f k ~ (0, ), for 0,0.1,0.,...,1. As a result, 11 input output syntheti data sets are formed. In the experiments, eah data set is split into training and testing subsets, where 0% of data is viewed as the training data, while the rest is used as the testing set. The split was repeated 40 times for eah data set. For all experiments, the value of the fuzzifiation oeffiient was set to m=. To arry out optimization, different values of loated in the range of [0, 100] were onsidered. Table shows the results in the form of the average and standard deviation of training and testing (performane index expressed by ( 4)) for zero order LSE tehnique, zero order augmented FCM tehnique, expanded first order model using strategy, and expanded first order model using strategy ; see Fig. 1. Moreover, the average and the standard deviation of the optimal value of obtained in the proposed models are reported. In all ases the proposed zero order fuzzy model produes lower values of the training and testing in omparison with the produed by the zero order LSE tehnique. The t test with (95% onfidene) was employed to assess whether the differenes between the ahieved testing s produed by the two zero order models are statistially signifiant. The results show that in all ases (for the testing data), the differenes are statistially signifiant. The reason for the inreased auray of the proposed method is that we try to strike a sound balane between the effet of the input part and the output part of data when forming lusters. Then, by the use of the revealed struture in the input part of data, U ~, and the estimated parameters of the model, w i, i=1,,, we determine the output of the model. There is an interesting and intuitively aeptable trend: with the inrease of the standard deviation of noise, the values of get smaller in order to diminish (absorb) an impat of the noisy output on the determination of the struture of the data. This effet ould be sought as a remedy against assigning too high relevane to the noisy data. The omplexity of the proposed zero order model is higher than the standard LSE tehnique as for eah value of we need to luster the training data and finally selet the struture orresponding to the optimal value of this parameter. Nevertheless there are lear advantages by running this balaning proess. As shown in Table, expanding the zero order model to the first order redues the values of the training and testing s. When omparing the first and the seond strategy used to the design of first order models, in most ases, the differenes between the results are negligible. In omparing the omplexities of the two strategies (refer to Fig. 1), one may realize that in the first method the parameters of the first order model are alulated only one, but for the seond strategy, these parameters should be estimated for all values of. However, this differene is negligible beause for eah value of we need to luster the data set for both strategies that is muh more time onsuming than alulating the parameters of the first order model. As it an be seen in Table, for all models developed in the presene of the inreasing the level of noise, the performane index obtained for both training and testing sets ahieves higher values. For eah noise level, =0, 0.5, and 1, we seleted one of the generated training testing data sets. Fig. 4 shows the performane index for different values of in range [0, 100] for the proposed zero order fuzzy model over eah seleted data set. As shown there, with the inrease of the noise level, the values of both training and testing s, inreased. For all values of noise, the plot of the training and testing for =0 exhibits its maximum. The reason behind this behavior is that we have not onsidered output data in the lustering proess. When inreasing the value of (taking into onsideration the output data in the lustering proess), both the training and testing s derease until they reah their minimal value. This gives rise to establishing a sound balane between the struture of data present in the input and output spaes. However, the optimal value of depends on the dimensionality of the input spae and the mutual struture of data in the input and output spaes as well as the relationships among information granules formed in the input and output spaes () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

7 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 7 TABLE II EXPERIMENTAL RESULTS OBTAINED FOR SYNTHETIC DATA SETS AFFECTED BY DIFFERENT LEVELS OF NOISE. AVERAGE AND STANDARD DEVIATION OF PERFORMANCE INDEX (EXPRESSED BY (4)) FOR THE GENERATED TRAINING AND TESTING DATA ARE REPORTED. Opt STANDS FOR THE OPTIMAL VALUE OF first order fuzzy model LSE (zero order) zero order model first order fuzzy model using strategy using strategy Training Testing Training Testing Training Training Testing Opt Testing Opt ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ±0.1 1.± ± ± ± ± ± ± ± ± ± ±0. 0.4± ± ± ± ± ± ±0.07.9± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ±0.1.05± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ±0.3.5± ± ±0.5 3.± ± ± ± ± ± () Fig. 4. Performane index of onstruted zero order fuzzy models vs. different values of for three seleted data sets for noise levels: =0, =0.5, and () =1 10 x 4 0 Data points =0 =14 = x x 4 0 Data points =0 = =100 Data points =0 =35 = x1 10 Fig. 5 shows a distribution of the prototypes in the input spae of the seleted data sets for =0, =100, and optimal value of. As shown in this figure, for different values of we enounter with different positions of prototypes in the input spae. Finally, Fig. shows the plot of the performane index of the onstruted first order fuzzy model using the expansion strategy for different values of over the seleted data sets. As shown in this figure, similar to the zero order fuzzy models, the plot of testing usually is similar to the plot of training. However, first order models generate lower values of the training and testing, espeially for lower levels of noise. B. Real world data Eight real world data sets inluding Housing, Auto MPG, Forest fires, Red wine quality, Servo, Yaht hydrodynamis, Conrete ompressive strength oming from the UCI mahine learning repository ( and PM10 from CMU StatLib library ( have been onsidered in this experiment. For eah data set, the experiment was ompleted 40 times with the 0 40 split of the data into the training and testing set, respetively. The number of lusters varied from to x x1 10 () Fig. 5. Distribution of prototypes for =0, optimal value of, and =100, for seleted data sets for noise levels: =0, =0.5, and () = () Fig.. Performane index of onstruted first order fuzzy models using the seond expansion strategy vs. different values of for three data sets from noise levels: =0, =0.5, and () = () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

8 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 TABLE III COMPARISON BETWEEN THE MODELS CONSTRUCTED WITH THE USE OF LSE AND THE PROPOSED METHOD FOR EIGHT REAL DATA SETS. RESULTS ARE REPORTED IN THE FORM OF THE AVERAGE AND STANDARD DEVIATION OF TRAINING AND TESTING ERROR (4), OPTIMAL VALUE OF m, AND OPTIMAL VALUE OF LSE (zero order) Proposed zero order model Training Testing mopt Training Testing Opt mopt Housing Auto MPG Forest fires Red wine quality Servo Yaht hydrodynamis Conrete ompressive strength PM10 =* 3.10±. 4.7± ± ± ± ± ±0.19 =3* 59.± ± ± ± ± ± ±0.15 =4* 59.0± ± ± ± ± ± ±0.1 =5* 51.44± ± ± ± ±5.13.7± ±0.17 =* 40.35± ± ±0.1.05± ± ± ±0.1 =7* 37.37± ± ±0.0.4± ± ± ±0.09 = 1.53± ±.53 3.± ±1.39.±.1 59.± ±0.5 =3* 1.3±1.4.5±.5.7± ± ± ± ±0.09 =4* 17.4± ±.39.09± ± ±1.9 4.± ±0.0 =5* 13.7± ± ± ± ± ± ±0.1 =* 13.3± ± ± ± ±1.9 9.± ±0.17 =7* 13.19± ± ± ± ± ±. 1.±0.1 = 1.9± ± ± ±0.1 ±0.1.75± ±0.9 =3 1.9± ± ± ±0.1.01± ± ±0.1 =4 1.9±0.1 ±0.1.5± ±0.1.0± ± ±0.1 =5 1.± ±0.1.7± ±0.1.03±0.1 1.±0. 1.±0.1 = 1.7±0.11 ±0.1.53± ±0.1.05± ± ±0.15 =7 1.7± ±0.1.4± ±0.1.09± ± ±0.13 =* 0.5± ±0.04.4± ± ± ± ±0.04 =3 0.5± ± ± ± ± ± ±0.11 =4* 0.49± ± ± ± ± ± ±0.07 =5* 0.49± ± ± ± ± ± ±0.15 =* 0.4± ± ± ± ± ± ±0.14 =7 0.4± ± ± ± ± ± ±0.19 = 1.9± ±0.3.55± ±0.1.0± ± ±0.3 =3* 1.94±0..± ± ± ±0.3.4±5.53.7±0.4 =4* 1.3±0.4.± ± ± ±0.4.9± ±0.34 =5* 1.73±0.4.43±1.3.9± ± ± ± ±0.11 =* 1.7±0.5.1±0. 3.0± ± ± ± ±0.0 =7* 1.54±0.7 ±0.5.34± ±0.1 1.± ± ±0.1 =* 0.± ± ± ± ±13.4.1±1.5 1.±0.0 =3* ± ± ± ± ± ± ±0.0 =4* ± ±40.9.1±0..3± ±0..53± ±0.09 =5* 1.73± ± ± ± ±4.1.5± ±0.05 =* 11.1± ± ± ±. 1.7± ± ±0.04 =7* 153.±1.7.±349..3± ±9.9.1± ± ±0.0 =* 1.7± ±17.9.9± ± ±. 93.3± ±0.04 =3* 15.1± ±4.35.3± ± ± ±0 1.±0.05 =4* 194.1± ± ± ± ±7.9 3.± ±0.0 =5* 15.09± ± ± ± ±.05.± ±0.0 =* 173.1± ± ± ± ±. 75.± ±0.05 =7* 17.± ± ± ± ± ± ±0.1 = 0.71± ± ± ± ± ± ±0.14 =3 0.7± ± ± ± ± ±4.1 1.±0.3 =4 0.7± ± ± ± ± ± ±0.0 =5* 0.9± ± ±0.3 0.± ± ± ±0.0 =* 0.± ± ±0. 0.5± ±0.0.7± ±0.05 =7* 0.7± ±0.0.01± ± ± ± ±0.0 Housing Auto-MPG Forest fires Red wine quality Servo 30 Yaht hydrodynamis Conrete ompressive strength 0.7 PM Fig. 7. Training and testing vs. for one of the training testing data generated from eight real word data sets for = and optimal value of m () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

9 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 9 We also experimented with different values of the fuzzifiation oeffiient m ranging from 1.1 to 5 with a step of 0.1. The range of possible values of was set to [0, 100]. The results are provided by reporting the average and standard deviation of the training and testing, and showing optimal values of m and ; refer to Table 3. In most ases the values of the training and testing produed by the proposed method is lower than the one produed by the LSE method. The t test with the onfidene level of has been applied and entries marked with asterisk (*) positioned after the number of lusters indiate that the differenes between the ahieved testing produed by the proposed method and the ahieved testing by LSE are statistially signifiant. Note that for a few entries of this table, the LSE method leads to a lower value of the testing, but in this ase the differene is not statistially signifiant. As shown in Table 3, in both models the inrease of the number of lusters leads to the derease of the training. Having more lusters, the model beomes apable of apturing more detailed struture of the data, ausing a redution of the training. However, we do not observe the same tendeny with respet to the testing and in some data (e.g. Servo) inreasing the number of lusters, inreases the values of the testing. This omes as a manifestation of an overfitting effet. Figure 7 shows the performane index vs. different values of for one of the training testing data generated from eight real word data sets for = and optimal value of m. As shown in this figure in most ases the plot of the performane index for the testing data is usually similar to the plot obtained for the training data set. Let us onsider in more detail the Auto MPG data set. We look at one of the generated training testing pair of data sets and look at the fuzzy model for =, and = 3 and optimal values of m and. By projeting eah prototype of the lusters on the individual input variables, we form a family of rules with expliitly spelled out input variables. Table 4 provides the rules onstruted in this way. Figure visualizes the values of the training and testing obtained for different values of for the Auto MPG data set for =, 3 and optimal value of m. As shown there, for number of lusters =3, the performane index attains lower values. Moreover, in this ase the optimal value of is 100, whih indiates that the lowest value of the training ours when one puts more emphasis on the output part of data. Table 5 shows the omparison between the two strategies used to onstrut first order models. Note that for strategy, the optimal value of m and the optimal value of is the same as the onstruted zero order model reported in Table 3. As shown in Table 5, in all ases strategy produes lower training, but strategy in most ases omes with a lower testing. Comparing Table 3 and Table 5, one may observe that the proposed zero order model is more robust against potential overfitting. The first order models yield higher auray while they may suffer from eventual overfitting. TABLE IV CONSTRUCTED CLUSTER CENTERS, WEIGHTS, OPTIMAL VALUES OF m AND, TRAINING ERROR, TESTING ERROR, AND INDUCED RULES FOR THE AUTO MPG DATA SET FOR C= AND 3. =: v1 [.9, 3.3, 13.01, 39.0, 14.51, 74.59, 1.1] v [4.1, 115.7, 79.9, 331.0, 1.17, 77.31, 1.9 ] w 1 1.7, w 9., m Opt, Opt 1, training =1.53, testing =3.01 Rule 1: If the number of ylinders is high, displaement is high, horsepower is high, weight is high, aeleration is low, ar is old, and is Amerian then miles per gallon= 1.7 Rule : If the number of ylinders is low, displaement is low, horsepower is low, weight is low, aeleration is high, ar is new, and is European then miles per gallon = 9. =3: v1 [4.1, 104.0, 7.34, 1., 1.43, 7.5,.19] v [7., 304.1, 139.7, 30.1, 14.17, 74.1, 1.05] v3 [4.1, 143., 90.4, 11.5, 1.01, 75., 1.] w , w 15., w m Opt 1. 4 Opt 100, training =14.9, testing =15.99 Rule 1: If the number of ylinders is low, displaement is low, horsepower is low, weight is low, aeleration is high, ar new, and is European or Japanese then miles per gallon = 34.3 Rule: If the number of ylinders is high, displaement is high, horsepower is high, weight is high, aeleration is low, ar is old, and is Amerian then miles per gallon = 15. Rule3: If the number of ylinders is medium, displaement is medium, horsepower is medium, weight is medium, aeleration is medium, age of ar is medium, ar is European or Amerian then miles per gallon = 4.35 C. Comparison between the expanded model and other fuzzy models Here we ompare the proposed expanded model with a number of first order fuzzy models found in the literature. For this purpose, two well known real data sets namely the Box Jenkins gas furnae [0] and Auto MPG are onsidered. Box Jenkins gas furnae modeling: this data set onsists of 9 input output measurements of a gas furnae proess olleted using a sampling of 9 seonds. In eah sampling instant k, the input xk is the gas flow into the furnae and the output y k is CO onentration. We employed the proposed tehnique to onstrut a fuzzy model with six inputs x k, x k 1, x k, y k 1, y k, and y k 3, and a single output y k. The same struture of the model has been onsidered in [17], [31], [3], and [39]. We investigate two senarios when onstruting fuzzy models. In the first senario (senario 1), the objetive is to evaluate the ability of the proposed method to fit the training data. In this ase all the data were used as the training data set. For onsistent omparison, the fuzzifiation oeffiient in all experiments in this subsetion was set to m=. Table shows the training produed by our method for different number of lusters and ontrasts it with the results formed by some other methods available in the literature () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

10 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 10 TABLE V COMPARISON OF THE TWO DEVELOPMENT STRATEGIES WITH REGARD TO THE PERFORMANCE INDEX (4) WHEN CONSTRUCTING FIRST ORDER MODELS. FOR STRATEGY (A), THE OPTIMAL VALUE OF m AND THE OPTIMAL VALUE OF IS THE SAME AS FOR CONSTRUCTED ZERO ORDER MODEL REPORTED IN TABLE 3. Expanded model using strategy Expanded model using strategy Training Testing Training Testing Opt = 13.59± ±3. 1.3± ±19.0.3± ±0. =3 10.7± ± ± ±. 1.77± ±0.47 Housing =4 9.7± ±.1.4±1.3.95±1.4.04± ±0.9 =5.± ±7.11.7± ± ± ±0.19 =.11±0.1.51± ± ± ± ±0.5 =7 5.13± ± ± ± ± ±0.0 = 7.7± ± ± ± ± ±0.93 =3.1±0.51.± ± ± ± ±0.11 Auto MPG =4 5.± ± ± ± ± ±0.11 =5 5.49± ± ± ± ±.37 1.±0.1 = 4.3± ± ± ± ± ±0. =7 4.39± ± ± ± ± ±0.13 = 1.73±0.1.3± ±0.11.4±0. 7.± ±0.1 =3 1.5±0.1.5±1.1 1.± ± ±..15±1.3 Forest fires =4 1.57±0.11.1±0. 1.5±0.1.4± ± ±0.33 =5 1.4± ± ± ± ± ±0.1 = 1.4±0.11 5± ± ± ± ±0. =7 1.3± ± ± ± ±.1 1.±0.17 = 0.43± ± ± ± ± ±0.4 =3 0.41± ± ± ± ± ±0.19 Red wine quality =4 0.4± ± ± ±0.0 0.±. 1.5±0.33 =5 0.39± ± ± ± ± ±0.3 = 0.3± ± ± ± ± ±0.1 =7 0.37± ± ± ± ±0. 1.3±0.1 = 0.55± ± ± ± ± ±0.4 =3 0.47± ± ± ± ± ±0.15 Servo =4 0.35± ±1.9 0.± ± ± ±0.34 =5 0.31± ± ± ± ± ±0. = 0.± ± ±0.05.4± ± ±0. =7 0.± ± ±0.05.1± ± ±0.13 = 10.± ±5.51.1±1.0.1±.7 74.± ±0 =3 13.0± ±9.7.15± ±.9 54.± ±0 Yaht = ± ±.5.5± ± ± ±0 hydrodynamis =5.9± ± ±.3 14.± ± ±0.03 = 7.4± ± ± ± ± ±0.03 Conrete ompressive strength PM10 mopt =7 7.3± ± ± ±9.5.03± ±0.05 = 90.49± ± ±. 0.53± ± ±0 =3 3.0± ± ± ±5..3± ±0 =4.± ± ±.74.11± ± ±0.07 =5.05± ± ± ± ± ±0.09 = 59.9± ±.3 4.9±.4 91.± ± ±0.04 =7 55.4± ± ± ± ± ±0. = 0.3± ± ± ± ± ±1.13 =3 0.55± ± ± ± ± ±0.4 =4 0.49± ± ± ± ± ±1.01 =5 0.47± ± ± ±0.0.5± ±0.7 = 0.44± ± ± ± ± ±0.54 =7 0.43± ± ± ±0.0.1± ± Fig.. Training and testing for different values of for Auto MPG data set. =, m=, =3, m= As shown in this table, with the inrease of the number of lusters the proposed expanded models fit the training data more aurately. The seond strategy (strategy ) yields lower training in omparison with the one produed by the first strategy. Furthermore, as shown in this table, the proposed models surpass most of the reported approahes in the literature. Fig. 9 shows the output and the predited output along with resulting using strategy for = () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

11 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 11 TABLE VI COMPARISON OF THE RESULTS PRODUCED BY THE PROPOSED METHOD AND THE RESULTS REPORTED IN THE LITERATURE FOR BOX JENKINS DATA (SCENARIO 1) Number of Number of Training Model parameters rules (MSE) Box and Jenkins [0] Tong [19] Xu and Lu [1] Pedryz [] Chen et al. [14] Sugeno and Yasukawa [3] Emani et al. [13] Kroll [1] Oh and Pedryz [11] Evsukoff et al. [10] Tsekouras [3] Chiu [] Kim et al. [39] Kim et al. [31] Expanded fuzzy model using strategy Expanded fuzzy model using strategy Output Error Real output Predited output Sample Sample Fig. 9. Results obtained for the Box Jenkins data (Senario 1) using the first expanding strategy for =: data versus predition results, and In the seond senario (Senario ), we onsider the first 14 samples as a training set while the remaining data form the testing set. The results of omparative analysis are given in Table 7. The disussed fuzzy models outperform most of the models in terms of the testing. As antiipated, by inreasing the number of rules, we redue the training however only to some extent; an exessively large number of lusters lead to the overfitting problem. Moreover, the expanded fuzzy model using strategy usually omes with the lower training, while in most ases strategy exhibits lower testing. The results produed for = using strategy are visualized in Fig. 10. TABLE VII PROPOSED METHOD AND THE MODELS REPORTED IN THE LITERATURE FOR BOX JENKINS DATA SET (SCENARIO ): A COMPARATIVE ANALYSIS Model Number of Number Training Testing parameters of rules (MSE) (MSE) Kim et al. [17] Lin and Cunnigham [3] Tsekouras [3] Li et al. [33] Expanded fuzzy model using strategy Expanded fuzzy model using strategy Auto MPG data set: after removing the data with missing values, there are 39 samples left in the data set. It was used by Abonyi et al. [34] to evaluate their proposed fuzzy model. The input variables seleted in their experiments were x 1 : displaement; x : horsepower; x 3 : weight; x 4 : aeleration; and x 5 : model year. Moreover, the output of the model was y: miles per gallon. In this experiment we onsider the same input output data and ompare our method with the methods disussed in [34], [1], and the standard LSE tehnique. All these models use a lustering tehnique for model identifiation. [34] uses a modified version of Gath Geva lustering (MGG) realized in the spae of input output data. In [1] ( with the model alled FMID toolbox) a Gustafson Kessel lustering [] was employed, and in the LSE tehnique the FCM used in the spae of input variables was onsidered. Sine the data set oming from the UCI mahine learning repository is sorted based on the model year variable, to have data from various years in both training and testing sets we seleted samples 1, 3, 5,, 391 as the training data and samples, 4,,, 39 as the testing data. Table summarizes the results. Some general tendenies similar to the ones reported in the previous experiments are observed. The training lowers its value while the number of rules (lusters) inreases. Expanded fuzzy model using the strategy usually surpasses the strategy over trainin g data, but this may lead to overfitting problem. An interesting feature of the expanded model using strategy for this data set omes with its robustness to overfitting as demonstrated in ase when =10. All other methods exhibit a signifiant jump in the values of the testing (doubling its value vis à vis the value reported for the training ). VII. CONCLUSION A luster entri fuzzy modeling tehnique is proposed in this study. An augmented version of Fuzzy C Means is employed to luster the input output spae of data to ahieve () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

12 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 1 a sound balane between the effet of input spae and output spae in the lustering proess through minimizing a ertain performane index. Using the revealed struture in the input spae and the estimated parameters of the fuzzy model, an effiient zero order fuzzy model has been onstruted. Although the zero order fuzzy models are more readable and interpretable, higher order models are more interesting for approximation purposes. Therefore, the developed zero order fuzzy model was expanded to order one using a Taylor expansion tehnique leading to the inrease of its effiieny. Two key design strategies are investigated. When omparing the two arhitetures studied here, experimental results reveal that the zero order model beomes more robust against potential overfitting. The seond one (first order model) yields higher auray (whih is not surprising) however it may suffer from possible overfitting. Experimental results reported over both syntheti and real data sets show the effiieny of the proposed approahes in omparison with several tehniques reported in the literature. The main reason behind high auray of the proposed zero and the first order fuzzy models is that in the identifiation proedure we take into onsideration the struture of the data available in both input spae and output spaes and develop a struture (olletion of lusters) guided by the auray riterion of the fuzzy model rather than the objetive funtion navigating fuzzy lustering and as suh not neessarily assoiated diretly with the auray requirement of fuzzy models. Output Output Real output Predited output 0 50 Sample Real output Predited output Sample Fig. 10. Results obtained for the Box Jenkins data (Senario ) using the first expanding strategy: training data, and testing data. REFERENCES [1] T. Takagi, M. Sugeno, Fuzzy identifiation of systems and its appliation to modeling and ontrol, IEEE Trans, IEEE Trans. Syst., Man, Cybern., vol. SMC 15, no.1, pp.11 13, Jan. Feb [] M. Sugeno, G.T. Kang, Struture identifiation of fuzzy model, Fuzzy Sets Syst., vol., no. 1, pp , Ot. 19. [3] M. Sugeno, K. Tanaka, Suessive identifiation of a fuzzy model and its appliations to predition of a omplex system, Fuzzy Sets Syst., vol. 4, no. 3, pp , Aug [4] J.C. Bezdek, Pattern Reognition with Fuzzy Objetive Funtion Algorithms, New York, Plenum, 191. [5] W. Pedryz and A. Bargiela, Fuzzy lustering with semantially distint families of variables: Desriptive and preditive aspets, Pattern Reognition Letters, vol. 31, no. 13, pp , Ot [] W. Pedryz and J. Valente de Oliveira, A development of fuzzy enoding and deoding through fuzzy lustering, IEEE Trans. Instrum. Meas, vol. 57, no. 4, pp. 9 37, Apr. 00. [7] H. Izakian, W. Pedryz, I. Jamal, Clustering spatio temporal data: An augmented fuzzy C Means, IEEE Trans. Fuzzy Syst., vol. 1, no. 5, pp. 55, Ot [] S. Chiu, Seleting input variables for fuzzy models, Journal of Intelligent & Fuzzy Systems, vol. 4, no. 4, pp. 43 5, 199. [9] W. Pedryz, M. Reformat, Evolutionary fuzzy modeling, IEEE Trans. Fuzzy Syst., vol. 11, no. 5, pp. 5 5, Ot [10] A. Evsukoff, A. C. S. Brano, and S. Galihet, Struture identifiation and parameter optimization for non linear fuzzy modeling, Fuzzy Sets Syst., vol. 13, no., pp , De. 00. [11] S.K. Oh, W. Pedryz, Identifiation of fuzzy systems by means of an auto tuning algorithm and its appliation to nonlinear systems, Fuzzy Sets Syst., vol. 115, no., pp , Ot [1] A. Kroll, Identifiation of funtional fuzzy models using multidimensional referene fuzzy sets, Fuzzy Sets Syst., vol. 0, no., pp , Jun [13] M.R. Emani, I.B. Turksen, A.A. Goldenberg, Development of a systemati methodology of fuzzy logi modeling, IEEE Trans Fuzzy Syst., vol., no. 3, pp , Aug [14] J.. Chen, Y.G. Xi, Z.J. Zhang, A lustering algorithm for fuzzy model identifiation, Fuzzy Sets Syst., vol. 9, no. 3, pp , Sep [15] J. Abonyi, R. Babuska, and F. Szeifert, Fuzzy modeling with multivariate membership funtions: Gray box identifiation and ontrol design, IEEE Trans. Syst. Man, Cybern. B, Cybern., vol. 31, no. 5, pp , Ot [1] C.W. Xu, Y.Z. Lu, Fuzzy model identifiation and self learning for dynami systems, IEEE Trans. Syst. Man, Cybern., vol. 17, no. 4, pp. 3 9, Jul [17] E. Kim, H. Lee, M. Park, M. Park, A simply identified Sugeno type fuzzy model via double lustering, Information Sienes, vol. 110, no. 1, pp. 5 39, Sept [1] R. Babuska, Fuzzy Modeling for Control. Boston, MA: Kluwer, 199. [19] R.M. Tong, The evaluation of fuzzy models derived from experimental data, Fuzzy Sets Syst., vol. 4, no. 1, pp. 1 1, Jul [0] G.E.P. Box, G.M. Jenkins, Time series analysis, foreasting and ontrol, seond ed., Holden Day, San Franiso, CA, [1] T. W. Liao, A. K. Celmins, R. J. Hammell, A fuzzy -means variant for the generation of fuzzy term sets, Fuzzy Sets Syst., vol. 135, no., pp , Apr [] W. Pedryz, An identifiation algorithm in fuzzy relational systems, Fuzzy Sets Syst., vol.13, no., pp , Jul [3] M. Sugeno, T. Yasukawa, A fuzzy logi based approah to qualitative modeling, IEEE Trans. Fuzzy Syst., vol. 1, no. 1, pp. 7 31, Feb [4] L. Wang, R. Langari, Complex systems modeling via fuzzy logi, IEEE Trans. Syst., Man, Cybern., vol., no. 1, pp , Feb [5] A.F. Gomez Skarmeta, M. Delgado, M.A. Vila, About the use of fuzzy lustering tehniques for fuzzy model identifiation, Fuzzy Sets Syst., vol. 10, no., pp.179 1, Sep [] D.E. Gustafson, W.C. Kessel, Fuzzy lustering with a fuzzy ovariane matrix, Pro. IEEE lnt. Conf. on Fuzzy Syst., San Diego, 1979, pp [7] M. Delgado, A.F. Gomez Skarmeta, F. Martin, A fuzzy lustering based rapid prototyping for fuzzy rule based modeling, IEEE Trans. Fuzzy Syst., vol. 5, no., pp. 3 33, May [] Y. Yoshinari, W. Pedryz, K. Hirota, Constrution of fuzzy models through lustering tehniques, Fuzzy Sets Syst., vol. 54, no., pp , Mar [9] G. Panoutsos, M. Mahfouf, A neural-fuzzy modelling framework based on granular omputing: Conepts and appliations, Fuzzy Sets Syst., vol. 11, no. 1, pp. 0 30, Nov [30] H.R. Berenji, P.S. Khedkar, Clustering in produt spae for fuzzy inferene, IEEE Int. Conf. on Fuzzy Syst., San Franiso, CA, 1993, pp [31] E. Kim, M. Park, S. Kim, M. Park, A transformed input domain approah to fuzzy modeling, IEEE Trans. Fuzzy Syst., vol., no. 4, pp , Nov () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See

13 /TFUZZ , IEEE Transations on Fuzzy Systems TFS R1 13 TABLE VIII A COMPARISON BETWEEN FMID, MGG, LSE, AND THE PROPOSED EXPANDED MODELS OVER THE AUTO MPG DATA SET FOR DIFFERENT NUMBER OF CLUSTERS. IN ALL METHODS, THE VALUE OF THE FUZZIFICATION COEFFICIENT WAS SET TO. Proposed method Proposed method FMID [1] MGG [34] LSE Fuzzy strategy strategy model: Training Testing Training Testing Training Testing Training Testing Training Testing = = = = = [3] Y. Lin, G.A. Cunningham, A new approah to fuzzy neural modeling, IEEE Trans Fuzzy Syst., vol. 3, no., pp , May [33] C. Li, J. Zhou, B. Fu, P. Kou, J. Xiao, T S fuzzy model identifiation with a gravitational searh based hyperplane lustering algorithm, IEEE Trans. Fuzzy Syst., vol. 0, no., pp , Apr. 01. [34] J. Abonyi, R. Babuska, F. Szeifert, Modified Gath Geva fuzzy lustering for identifiation of Takagi Sugeno fuzzy models, IEEE Trans. Syst. Man, Cybern. B, Cybern., vol. 3, no. 5, pp. 1 1, Ot. 00. [35] M. F. Azeem, M. Hanmandlu, N. Ahmad, Struture identifiation of generalized adaptive neuro-fuzzy inferene systems, IEEE Trans Fuzzy Syst., vol. 11, no. 5, pp. 1, Ot [3] A. Celikyilmaz, I. B. Turksen, Enhaned fuzzy system models with improved fuzzy lustering algorithm, IEEE Trans. Fuzzy Syst., vol. 1, no. 3, pp , Jun. 00. [37] C. Li, J. Zhou, X. Xiang,. Li, X. An, T S fuzzy model identifiation based on a novel fuzzy C regression model lustering algorithm, Engineering Appliations of Artifiial Intelligene, vol., no. 4 5, pp. 4 53, Jun [3] G. E. Tsekouras, On the use of the weighted fuzzy means in fuzzy modeling, Advanes in Engineering Software, vol. 3, no. 5, pp , May 005. [39] E. Kim, M. Park, S. Ji, M. Park, A new approah to fuzzy modeling, IEEE Trans. Fuzzy Systems, vol. 5, no. 3, pp , Aug [40] E. H. Mamdani, Appliations of fuzzy algorithms for ontrol of a simple dynami plant, Pro. The Institution of Eletrial Engineers, vol. 11, no. 1, pp , De [41] G. Tsekouras, H. Sarimveis, E. Kavakli, G. Bafas, A hierarhial fuzzy lustering approah to fuzzy modeling, Fuzzy Sets Syst., vol. 150, no., pp. 45, Mar [4] B. Hartmann, O. Banfer, O. Nelles, A. Sodja, L. Tesli,I. Skrjan, Supervised hierarhial lustering in fuzzy model identifiation, IEEE Trans. Fuzzy Systems, vol. 19, no., pp , De [43]. Zhang, M. Mahfouf, A hierarhial Mamdani type fuzzy modelling approah with new training data seletion and multi objetive optimisation mehanisms: A speial appliation for the predition of mehanial properties of alloy steels, Applied Soft Computing, vol. 11, no., pp , Mar [44] Y.-Y. Lin, J.-Y. Chang, N.R. Pal, C.-T. Lin, A mutually reurrent interval type- neural fuzzy system (MRITNFS) with self -evolving struture and parameters, IEEE Trans. Fuzzy Syst., vol. 1, no. 3, pp , Jun [45] W. Zhao, K. Li, G.W. Irwin, A new gradient desent approah for loal learning of fuzzy neural models IEEE Trans. Fuzzy Syst., vol. 1, no.1, pp , Feb [4] X. Xie, H. Ma, Y. Zhao, D.-W. Ding ; Y. Wang, Control synthesis of disrete-time T S fuzzy systems based on a novel non-pdc ontrol sheme, IEEE Trans. Fuzzy Syst., vol. 1, no.1, pp , Feb [47] M. Chadli, H.R. Karimi, Robust Observer Design for Unknown Inputs Takagi Sugeno Models, IEEE Trans. Fuzzy Syst., vol. 1, no.1, pp , Feb Witold Pedryz (M SM 94 F 99) reeived the M.S., Ph.D. and D.Si. degrees from the Silesian University of Tehnology, Gliwie, Poland. He is urrently a Professor and Canada Researh Chair (CRC omputational intelligene) with the Department of Eletrial and Computer Engineering, University of Alberta, Edmonton, AB, Canada. He is also with the Department of Eletrial and Computer Engineering Faulty of Engineering, King Abdulaziz University, Jeddah, Kingdom of Saudi Arabia. In 009, he was eleted as a foreign member of the Polish Aademy of Sienes, Warsaw, Poland. He is the author of 14 researh monographs overing various aspets of omputational intelligene and software engineering. His main researh interests inlude omputational intelligene, fuzzy modeling and granular omputing, knowledge disovery and data mining, fuzzy ontrol, pattern reognition, knowledge-based neural networks, relational omputing, and software engineering. He has published numerous papers in this area. Prof. Pedryz was eleted as a Fellow of the Royal Soiety of Canada in 01. He has been a member of numerous program ommittees of IEEE onferenes in the area of fuzzy sets and neuroomputing. He is intensively involved in editorial ativities. He is an Editor-in-Chief of Information Sienes and Editor-in-Chief of the IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART A: SYSTEMS AND HUMANS. He urrently serves as an Assoiate Editor of the IEEE TRANSACTIONS ON FUZZY SYSTEMS and is a member of a number of editorial boards of other international journals. In 007, he reeived a prestigious Norbert Wiener Award from the IEEE Systems, Man, and Cybernetis Counil. He reeived the IEEE Canada Computer Engineering Medal in 00. In 009, he reeived a Cajastur Prize for soft omputing from the European Centre for Soft Computing for pioneering and multifaeted ontributions to granular omputing. Hesam Izakian (S 1) reeived the M.S. degree in Computer Engineering from the University of Isfahan, Iran. He is urrently working toward the Ph.D. degree with the Department of Eletrial and Computer Engineering, University of Alberta, Edmonton, AB, Canada. He is working under the supervision of Professor Witold Pedryz and his researh interests inlude Computational Intelligene, knowledge disovery and data mining, pattern reognition, and Software Engineering () 013 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See