A New Fuzzy Inference Approach Based on Mamdani Inference Using Discrete Type 2 Fuzzy Sets*

Transcription

1 2004 IEEE International Conference on Systems, Man and Cybernetics A New Fuzzy Inference Approach Based on Mamdani Inference Using Discrete Type 2 Fuzzy Sets* Ozge Uncu Kemal Kilic I.B. Turksen Dept. of Industrial Engineering FENS, Sabanci University Dept. of Mechanical and Middle East Technical University Tuzla, Istanbu1,Turkey Industrial Eng g Ankara, Turkey kkilic@sahanciuniv.edu University of Toronto uncu@ie.metu.edu.tr Toronto, ON, Canada turksen@mie.utoronto.ca Abstract -Fuzzy System Modeling (FSw is one of the most prominent system modeling tools in analyzing the data in the presence of uncerfainty Linguistic Fuzzy Rulebose (ZFR) structure, in which both the antecedent and consequent variables are represented by fizzy sets, is the most well known fizzy rulebase struciure in the literature. The proposed FSM method identifies LFR system model by executing Fuzzy C-Means (FCM clustering method. One of the sources of uncertainly in system modeling is the uncertainty in selecting learning parameters. In order to copiure this uncertainty in a more realistic way, the antecedent and consequent variables are represented by using Type 2 fuzzy sets that are consfructed by executing FCMmethod with different level of fizziness, m. values. The proposed system modeling approach is applied on a well-known benchmark data set where the goal is to predict the price of o stock. After comparing the results with the ones obtained with other system modeling tools, it can be claimed successful results are achieved. Keywords: Fuzzy System Modeling, Fuzzy Systems, Discrete Type 2 Fuzzy Sets, Fuzzy Inference 1 Introduction Fuzzy System Modeling (FSM) attracted noteworthy attention in data analysis and system modeling field in the last few decades. The major advantage of fuzzy system models is the transparency of them. The fuzzy system models have considerably higher understandability than the other ones (e.g., neural network). This is due to the fact that the fuzzy system models employ fuzzy sets that can be associated with linguistic labels. Whereas, the accuracy of the fuzzy models is comparable with others. There are a number of fuzzy rulebase structures, e.g. Mizumoto [lo], Takagi-Sugeno [12], that can be utilized in FSM. The general fuzzy rule structure can be written as * /04/$ IEEE (.a R : ALSO(IF antecedent,thenconrequenf,) (1) i-1, where c* is the number of rules in the rulebase. The fuzzy rulebase structure mainly differs in the representation of the consequents. If the consequent is represented with fuzzy sets then the rulebase can be categorized as LFR structure. Whereas, if the consequent is represented with a linear equation of input variables, then the rulebase structure is known as Takagi-Sugeno Fuzzy Rulebase (TS-FR) structure. The consequent of the rules are represented with a scalar value in Mizumoto structure. Thus, the Mizumoto Fuzzy Rulebase (M-FR) structure can he considered as a special case of both LFR and TS-FR structure. The LFR, TS-FR and M-FR structures are formalized in equations 2,3 and 4 respectively: E. ALSO(IF antecedent, THEN y E Y isrbj) L.. i-l ALSO(IF antecedent, THEN y, =a$ + bi) 1-1 D. ALSO (IF antecedent, THEN y, = b, ) 5-1 where, YcM is the domain of the output variable, Bi is the fuzzy set associated with output in i* rule, ai=(%,,,..., &,w) is the regression coefficient vector associated with the i* rule, bi is the scalar associated with the i& rule and NV is the number of input variables in the system model. The conventional fuzzy rulebase structures employ type 1 fuzzy sets botweither in antecedent andor consequent parts of the rules. However, recent studies shows that [5] - [7], [9],[13] the uncertainty can he captured in a better way by using higher order fuzzy sets, such as Type 2 fuzzy sets, which encapsulate more information granules. In [15] and [16], the extensions of the conventional TS-FR and M-FR, in which antecedents are represented by discrete Type 2 fuzzy sets and consequents are represented with a set of scalar values, (2) (3) (4)

2 was proposed. In the sequel, the conventional LFR structure will be modified such that both antecedent and consequent inference parameters will be represented by discrete Type 2 fuzzy sets. The conventional Mamdani inference [8] should be modified such that the proposed one should be able to infer model output by using the inference parameters represented with the discrete Type 2 fuzzy sets. The rest of this paper is structured as After elaborating the differences between the conventional LFR structure and its discrete Type 2 extension of it, the proposed FSM method will be explained in detail in section 2. Next, the proposed inference method will be explained in section 3. Then, the proposed method will be applied on a benchmark data set where the price of a stock is to be predicted. The results will be compared with the ones obtained by using the other fuzzy system modeling methods in section 4. Finally, the conclusions will be drawn and further research directions will be discussed in section 5. 2 Proposed FSM Approach In this section, first the discrete Type 2 extension of the LFR structure will be explained. Then, the proposed Fuzzy System Modeling (FSM) approach will be elaborated. As mentioned in section 1, the structure of the LFR can be formalized as in Eqn. 2. One can extend the LFR model with Type 2 fuzzy sets in three ways: by using i) Full Type 2 Fuzzy Sets (FTZFS), ii) Interval Type 2 Fuzzy Sets (IVTZFS), iii) Discrete Type 2 Fuzzy Sets (DTZFS). Please note that Mendel [9] named the third option as embedded type-2 fuzzy set. This study assumes that the reader is familiar with the definitions of Type 2 fuzzy sets. If not, the reader is encouraged to read [9] for formal definitions of FTZFS, IVT2FS and DT2FS. Informally, FT2FS is a fuzzy set characterized with a fuzzy membership function rather than a scalar value in unit interval. Thus, FT2FS A, which will be denoted as d, can be formalized as A = {(x>(%r,(u))) I x E x,u E J, c [O,lI} (5) where J, is a subset of unit interval, Xc!RNV is the domain of x, u6jx is the primary membership value associated with xd,l(u) is the secondary membership function that maps the primary membership value onto unit interval (i.e., $(U): [0,1] +[O,l]) and NV is the number of input variables in the system model. Similarly, IVT2FS can be informally explained as the mapping from the universe of discourse of a variable to an interval in [0,1]. Thus, IVT2FS is a special case of FT2FS where fx(u): [0,1] 31 and can be formalized as 2 = {(x,(u,u) I x E X,u E Jx c [0,11} (6) Finally, DT2FS is a special case IVT2FS where J, is set of real values with finite elements. Thus, DTZFS can be formalized as A = ((x,(u,s) I x E X,u E [0,11} (7) Vi = I,..., NM, where NM is the number of membership values that is associated with a particular value o fxd The question that needs to be asked is why do we need to use DT2FS rather than FT2FS or IVT2FS?. The answer is computational complexity. The fuzzy inference method proposed by Kamik et. al. [5] which utilizes FT2FS base fuzzy mlebase structure is computationally expensive (especially because of type reduction step). Thus, Liang et. al. [7] proposed an interval type 2 fuzzy system model structure where a model output can be inferred by using a computationally more efficient inference algorithm. However, the type reduction can still only be performed as the last step or as the first step. If type reduction is performed as the last step, there is still a computational burden that needs to carried while firing the fuzzy rules and finding the model output fuzzy step. If type reduction is performed as the fust step, there is no difference between conventional type 1 fuzzy system models and the type 2 fuzzy system model represented by IVT2FS. Thus, we propose to use DT2FS in the representation of fuzzy rules. The advantage of using DTZFS will be clarified in section 3 while explaining the proposed Fuzzy Inference (FI) method. As mentioned before, Mendel [9] suggested that IVT2FS based fuzzy system models can be viewed as a collection of embedded type 1 fuzzy system models. If the number of embedded type 1 fuzzy system models approaches infinity, the collection constitutes IVT2FS based fuzzy system model. However, Mendel proposed to identify the inference parameters (i.e., type 2 fuzzy sets) associated with each variable in each d e in the mlebase by using supervised learning methods. Thus, Mendel [9] assumes the system model structure a priori and then tries to find the best parameter values for this assumed model structure that fits to the data. In this study, we propose to use a natural way to construct discrete type 2 fuzzy system models by using the uncertainty in the selection of learning parameters of an unsupervised learning method as the source of uncertainty in values of inference parameters. In this method, Fuzzy C-Means (FCM) clustering method [2] is chosen as the method that will identify the basic structure of the system model. FCM method requires the user to provide the number of clusters in the system, c*, and the level of fuzziness, m, that defines the degree overlap between the fuzzy clusters. In 2273

3 ~ this study, the uncertainty in selecting the value of level of fuzziness is chosen as the source of uncertainty in the values of inference parameters. Thus, embedded type 1 fuzzy system models will be identified for different m values. The collection of these embedded type 1 fuzzy system models will represent the proposed discrete type 2 fuzzy system model. Therefore, the proposed tvpe 2 fuzzy rulebase structure, R, can be formalized as where, NM is the number of m values used in structure identification, m' is the rfi level of fuzziness value, R: is the i& rule identified by executing FCM clustering algorithm with m'. R; can he explicitly written as RI:IFxEXisrq'THEN ~ E isrb,: Y (9) wherei=l,..., c*andr=l,...,nm. As one can see, unlike the conventional fuzzy rulebase structure where the antecedent is represented as the aggregation of fuzzy sets associated with each input variable, the antecedent of the proposed structure is represented with W-dimensional fuzzy set as proposed in [11, ~31, [61, ~141. After giving the brief definitions of Type 2 fuzzy sets and explaining the proposed fuzzy system model structure, the next step is to discuss the proposed FSM method. The conventional FSM methods based on FCM clustering method [ll], [4] fit a fuzzy membership function to each fuzzy set associated with each system variable in each rule after finding the cluster centers of each cluster and the membership values for each training data vector in each clusters (rule). This can lead to unnecessary computational effort and erroneous results. Please refer to [14] for details. The proposed FSM method uses the cluster centers acquired by using the execution of FCM clustering algorithm to represent the fuzzy sets associated with the antecedent and consequent parts of the fuzzy rules. Then, the proposed FSM method can be explained as Let ND and NV denote number of data and number of input variables in the training data set, respectively and v:.'= (v;,?, vi:,..., v;,~',', v;,w+:3 denote the i" cluster center identified by executing FCM algorithm with level of fuzzmess, m' EM = {ml,...,"*'i, to partition the training data set into c clusters, where i=l,..., c, 1-1,...,NM, and FC,, _.., cmm. Then, the cluster centers of the iih induced NV-dimensional antecedent and 1- dimensional consequent cluster (fuzzy set) can he denoted as v;x'.c= (v;,,": v;,?,._., vi.~"3 and v;'.'.~= (~~,~+,'.e). By identiljmg v;" and for each (m',c) pair, the discrete type 2 fuzzy system model is constructed for all FC,;,...cm Next step is to select the number of clusters that is most suitable for the given training data with respect to accuracy measure. Let ~2,~ denote the model output deduced for the Ph input data vector in the training data set by using m' as the level of fuzziness value and c as the number of rules. Please note that the proposed inference mechanism will he explained later. Furthermore, let the square prediction error obtained by using yzd,k be denoted as SE( yyd,k ).Then, given that the number of.the clusters in the system is c, the optimal model output for Ph training data vector, yld,,.,, is found by using the following equation: A, = [yz,h I S E ( ). ~ ~, k ) = m i n [ ~ E ( ~ ~, k ) ~ ),=I NM (10) where c = c,.,..., c-, k = 1,..., ND. Let the sum of square prediction error acquired for a particular c be denoted as SSE(c). SSE can he calculated as where c = c,,,..., c-. Then, the optimal number of rules, c*, can he determined by using the following equation: At this stage, the best number of clusters, c*, is chosen and all antecedent and consequent parameters are identified for c* and for different level of fuzziness values, m EM. Since the ultimate goal of system modeling effort is to find a crisp model output for a given input data vector, the type of the system model or system model output should be reduced. Kamii et. al. [5] and Liang et. al. [7] proposed to reduce the type of system model output from type 2 fuzzy set to a crisp model output value as the last step of their fuzzy inference method. This means that the computations of fuzzy inference steps need to he carried on type 2 fiuzy sets till end of process. The main questions that needs to be asked is: "Is it possible to reduce the type of the system model from type 2 to type 1 for each different observation at the beginning of the inference process in order to avoid the computational burden?". A new inference method is proposed in [15] and [16], in which the type reduction is taken as the fxst step of the inference process. The idea can be informally explained as For different regions in input space, different discrete type 1 fuzzy system models behave better than others with respect to an accuracy measure 2274

4 such as Root Mean Square Error (RMSE). In the rulebase structure that is used in this study, for different regions in input space, different discrete type 1 fuzzy system models identified for different m values behave better than others, i.e., some regions of the input space require crisp system model, some regions of input space allows more overlap between rules. In order to select the most suitable embedded type 1 fuzzy system for a given observation, the approach proposed in [15],[16] also stores an m- lookup table in which each training input data vector is associated with an m value that gives the least error for the corresponding input data vector. Hence, the last step of the structure identification is to construct this m-lookup table. Let the optimal level of fuzziness value associated with the embedded type 1 fuzzy system model which yields the least prediction error for Ph input training data vector he denoted as mc mr can be determined as I,=I 3 Proposed Fuzzy Inference Method The proposed Fuzzy Infereuce (FI) method can be viewed as the type 2 extension of the one proposed by Uncu et. al. [I41 (which by the way is a modification of the FI method proposed by Delgado et. a1.[3]) The main difference is the type reduction step which is employed at the beginning of the method to reduce the type of the system model from type 2 to type 1 by selecting the most suitable embedded type 1 fuzzy system model for a given observation (test input data vector) with respect to an accuracy measure. As mentioned earlier, the type 2 fuzzy system model is constructed due to the uncertainty in selecting the most suitable level of fuzziness, m, value. Thus, selection of the most suitahle embedded type 1 fuzzy system model corresponds to the selection of the most suitable m value for a given observation. The m- lookup table that was built as the last of the structure identification process is utilized for this purpose: 4. Aggregate model output fuzzy set of each rule to fmd the overall model output fuzzy set. 5. Calculate the center of gravity of the model ontpnt fuzzy set to calculate the crisp model output value. Thus, the type 2 system model has to be reduced to execute the classical fuzzy inference methods. As mentioned before, the m-lookup table that has been stored in the system identification phase is utilized for this purpose. When a new test input data vector, x', is presented to the model, frst the closest training input data vector is chosen with respect to a distance measure. Then, the type 1 fuzzy system model corresponding to the m value associated with the chosen training input data vector is selected. This step can he performed by using the following equation: m, = m' ~~JSE(y~~,,)=min[SE(y~~,,)l} NM (13) where f E {l,,,,,nd}, m,, "EM and m, is the level of fuzziness value associated with they training input data Vk=l,..,ND. vector in m-lookup table. Thus, by selecting the type 1 fuzzy system model that corresponds to m', the discrete At this stage, consequent and antecedent inference type 2 fuzzy system model is reduced to the type 1 model parameters and m-lookup table are identified and hence with inference parameters (v?, CG;'','?, where m"= the structure identification is completed. m'. Given an input observation, x', the proposed infereuce algorithm can be summarized in five steps: Select the optimal m value for x'. This m value will be denoted as m', where "EM. Find the degree of fire for each rule for the given test input data vector x'. Find the model output fuzzy set for each rule by using the corresponding degree of fues Then, embedded type 1 fuzzy system model, which is associated with the chosen level of fuzziness value, is used in order to deduce a model output for the given observation. The degree of fire of the ith rule for x' can be defined as: \ The third step of the proposed FI method is to fmd model output fuzzy set for each rule by calculating the result of IMPLICATION operation with the degree of fire calculated in step 2 and the corresponding output fuzzy sets. It should be noted that in the FI method proposed by Mamdani [SI, the IMPLICATION is taken as minimum operation. Assume that the output fuzzy set in each rule is renresented bv wine - a scatter matrix that stores NS outnut sample values and the corresponding membership values in each rule. Thus, the model output fuzzy set of the ith rule, B:, which is mathematically represented with the membership function &(y), can be defined as the following set of ordered pairs: P,*(Y)={(Y~, min(p',(x1, P,(YJ)I,P=L...,NS (16) where,,li, (y,) can be defined as: 2275

5 The fourth step is to aggregate the model output fuzzy sets. This can be done by utilizing ALSO operator. Please note that as proposed by Mamdani [XI. ALSO operator is taken maximum in this study. Thus, the overall model output, which can be mathematically represented with membership function p'(y) can be determined as P'(Y) ={(yp, me &r), ~ Y ))I,P=~,SIS J (18),=I..< The last step is to finding the crisp model output, y*, by calculating the center of gravity of the model output fuzzy set as Ns *- Cp;l Y p "(fin( Pj(Xr)>PI(YP))) isl..c' Y - NS (19) p;l mm I=I.~E* (U (X?,Pj (Yp))) 4 Stock Price Prediction In order to test the proposed inference schema and FSM method, the proposed FSM approach is applied to the stock price data that is introduced by Sugeno and Yasukawa [Ill. The system variables can be defmed as xi : past change of moving average (I) over a middle period x2 : present change of moving average (I) over a middle period x3 : past separation ratio(1) w.1.t. moving average over a middle period q : present separation ratio(1) w.r.1. moving average over a middle period x5 : present change of moving average(2) over a short period x, : past change of price x, : present change of price xg : past separation ratio(2) w.r.t moving average over a short period xg : present change of moving average(3) over a long period xlo : present separation ralio(3) w.r.1 moving average over a short period y: stockprice Among these ten input variables, three of them, namely q, xg and xio, are chosen as significant [15]. There are 50 data vectors available to train the model and 50 more data vectors are used to validate the model. The proposed FSM method is executed with c=2,..,20 and ms{l.ol, 1.2, 1.5, 1.8, 2, 2.2, 2.5, 2.8, 3, 3.2, 3.5). c* is selected as 20 by the proposed method. Root Mean Square Error is utilized as the performance index. The proposed method is compared with the MATLAB" implementations of system modeling approaches such as ANFIS, multilayer perceptron Neural Network (NN) and Linear Regression (LR). Furthermore, we compared the proposed type 2 algorithm with one of the most recently developed type 1 fuzzy system modelling algorithm, This algorithm is a modified version of Sugeno-Yasukawa algorithm [l 11, that is to say an output clustering based algorithm. Briefly stating, the algorithm introduces three extensions to the classical S-Y algorithm. Firstly the concept of projecting the output clusters onto n- dimensional input space rather than single input variables as if they are independent from each other. Secondly, a k- NN based similarity measure to determine the degree of firing, and thirdly significance degrees of input variables rather than classifying the input variables as significant or insignificant. Further modifications also proposed to solve cluster validity problem and problems associated with output clustering algorithm. For more detail readers should refer [6]. The results for the test data set are provided in Table 1. Table 1. Comparison of the results in terms of the root mean square errors (RMSE) Pro osed ANFIS NN TRAIN Conclusions and Future Work In this study, the discrete type 2 extension of LFR system model structure is proposed as a continuation of the works conducted in [15] and [16]. The Mamdani -e FI method is also modified such that it can utilize the proposed fuzzy system model structure in order to determine the model output for a given observation. The proposed method is applied on an application in which the goal is to predict the price of a stock based on its fmcial indicators. The comparison of the results obtained by using the proposed method and the one obtained with other system modeling techniques such as ANFIS, NN, LR and a recently proposed Type.1 fuzzy system modelig algorithm [6], leads us lo the conclusion that, the proposed method yields more accurate results. There can be several extensions of this study. In this section, we will explain two possible extensions. First of all (and most importantly), the very same idea can be used in order to identify higher order fuzzy system models. In other words, the selection of more than one learning parameters, say N parameters, can be used as the source of uncertainty to identify type N+l fuzzy system model structure. The second extension can be suggested in the method of selecting the most suitable m value for a given observation. In this study, the m value that is associated 2276

6 with the training input data vector that is found to be closest to the observation is used. This requires an m- lookup table with ND number of rows. The m-lookup operation can be time consuming when the number of data vectors is extremely high in the training data set. One can analyze whether same m value is selected within same regions of input space by performing a clustering analysis and reduce the number of rows of m-lookup table. References [I] R. Bahuska and H.B. Verbruggen, Constructing fuzzy models by product space clustering, in H. Hellendoom and D. Drionkov (Eds.), Fuzzy Model ZdentiJcation: Selected Approaches, pp 53-90, Springer, Berlin, Germany, [2] J.C. Bezdek, Fuzzy Mathematics in Pattern Classification, Ph.D. Thesis, Applied Mathematics Center, Comell University, Ithaca, [3] M. Delgado, A.F. Gomez-Skermata and F. Martin, Rapid Prototyping of Fuzzy Models, in H. Hellendoom and D. Driankov (Eds.). Fuzzy Model Identification: Selected Approaches, pp Springer, Berlin, Germany, [4] M.R. Emami, Systematic Methodology of Fuzzy Logic Modeling and Control and Application to Robotics, Ph.D. Thesis, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, [5] N.N. Kamik, J.M. Mendel and Q. Liang, Type-2 Fuzzy Logic Systems, ZEEE Tran. On Fuzzy Systems, vol. 7, no. 6, pp , [lo] M. Mizumoto, Method of fuzzy inference suitable for fuzzy control, J. Soc. Instrument Control Engineering, vol. 58, pp ,1989. [l 11 M. Sugeno and T. Yasukawa, A Fuzzy Logic Based Approach to Qualitative Modeling, IEEE Transaction on Fuzz~>Sysfems, vol. 1, pp 7-31, [I21 T. Takagi and M. Sugeno, Fuzzy Identification of Systems and Its Applications to Modeling and Control, ZEEE Transactions on Systems, Man. and Cybemetics, vol. SMC-19, no. 1, pp , [13] LB. Tiirkven, Type 2 representation and reasoning for CWW, Fuzzy Sets and Systems, vol. 127, pp , [14] 0. Uncu and LB. Tiirkp, A Novel Fuzzy System Modeling Approach: Multidimensional Structure identification and Inference, Proc. of Tenth ZEEE International Conference on Fuzzy Systems, pp , Melbourne, Australia, December [19] O.Uncu, I. B. Turksen and K. Kilic, LOCALM- FSM: A New Fuzzy System Modeling Approach Using a Two-step Fuzzy Inference Mechanism Based on Local Fuzziness Level Selection, Znternationol Fuzzy Systems Association World Congress ZFSA 2003, pp , Istanbul, Turkey, June [I61 0.Uncu and I. B. Turksen, A New Two-step Fuzzy Inference Approach Based on Takagi-Sugeno Inference Using Discrete Type 2 Fuzzy Sets, Proc. of 2Td Znternational Conference of the North American Fuzzy Information Processing Sociery, NAFZPS 2003, pp 32-37, Chicago, USA, July [6] K. Kilic, A Proposed Fuzzy System Modeling Algorithm with an Application in Pharmacokinetic Modeling, Ph.D. Thesis, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, [7] Q. Liang and J. M. Mendel, Interval Type-2 Fuzzy Logic Systems: Theoly and Design, ZEEE Tran. on Fuzzy Systems, vol. 8, no. 9, pp ,2000. [SI E.H. Mamdani and S. Assilian, An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller, Znf..I Man-MachineStudies,vol. 7, pp. 1-13, [9] J. M. Mendel, Uncertain Rule-Based Fuzzy Logic Systems: introduction and new directions, Prentice, Upper Saddle River,