Interactive Fuzzy Modeling by Evolutionary Multiobjective Optimization with User Preference

Transcription

1 Interactive Fuzzy Modeling by Evolutionary Multiobjective Otiization with User Preference Yusue ojia, Hisao Ishibuchi Deartent of Couter Science and Intelligent Systes, Osaa Prefecture University - Gauen-cho, aa-u, Saai, Osaa , Jaan Eail: {nojia, hisaoi}@cs.osaafu-u.ac.j Abstract One of the new trends in genetic fuzzy systes (GFS) is the use of evolutionary ultiobjective otiization (EMO) algoriths. This is because EMO algoriths can easily handle two conflicting objectives (i.e., accuracy axiization and colexity iniization) when we design accurate and coact fuzzy rule-based systes fro nuerical data. Since the ain advantage of fuzzy rule-based systes coared with other non-linear ones is their linguistic interretability, the design of fuzzy rule-based systes can be viewed as linguistic data ining fro nuerical data. Fro the data ining oint of view, the required nowledge strongly deends on its user. That is, the interretability of fuzzy rule-based systes should be evaluated by taing into account the user s reference. Although there exist a nuber of interretability easures in the literature, users usually do not now which easure reresents their reference beforehand. In this aer, we roose interactive fuzzy odeling by evolutionary ultiobjective otiization with user s reference. User s reference is reresented by several satisfaction level functions which can be interactively odified by the user. The user s reference is used as one of ultile objectives in an EMO algorith. As a case study, we aly our aroach to real world tie-series data of land rice oveents in Jaan and deonstrate a user interface of our aroach. Keywords evolutionary ultiobjective otiization, fuzzy odelling, interactive evolutionary coutation, user reference. Introduction There are two ajor goals in the design of fuzzy rule-based systes: accuracy axiization and colexity iniization. Since the id-99s, a large nuber of aroaches have been roosed for iroving the accuracy of fuzzy rule-based systes while aintaining their interretability [-22]. Genetic algoriths have been frequently used under the nae of genetic fuzzy systes (GFS) [23-25]. We can easily handle two conflicting objectives: accuracy axiization and colexity iniization by the weighted su of the or ultiobjective forulations using evolutionary ultiobjective otiization (EMO) algoriths [26-28]. One of the hottest issues in GFS is how to easure the interretability of fuzzy rule-based systes [29-34]. A nuber of interretability easures have been already roosed and ileented in GFS. Interretability is, however, very subjective for users. Let us assue the following two rule sets. [Fuzzy rule-based syste A] If x is big then y is, If x 3 is sall then y is 5, If x 2 is very sall then y is 2, If x 3 is big then y is. [Fuzzy rule-based syste B] If x is big and x 2 is big and x 3 is sall then y=x +5x 2 +9, If x is very big and x 2 is very sall then y = 2x 2 +2, If x is very sall and x 2 is sall and x 3 is big then y=2. If we assue that a rule set with a sall nuber of rules is interretable, the fuzzy rule-based syste B is ore interretable. However, the fuzzy rule-based syste A sees to be ore interretable with resect to the rule length and the rule tye. Fro this observation, we can say that the interretability is totally deendent on user s reference (see [35] for ore detailed discussions). There is another roble on interretability. Users usually now which is interretable for the only after coaring aong soe alternative fuzzy rule-based systes. That is, an interactive otiization rocess ust be needed for the users. In our forer studies [36, 37], we have roosed an interactive genetic fuzzy rule selection ethod for attern classification robles. A reference function is used as one objective function in an EMO algorith. The reference function is coosed of several satisfaction level functions. The reason why we used several satisfaction level functions is that users don t now aroriate criteria and their riorities aong the beforehand. These satisfaction level functions are interactively odified during the evolution under the fraewor of interactive evolutionary coutation [38-4]. In this aer, we aly this idea to a fuzzy odeling roble. We deal with tie-series data of land rice oveents and deonstrate the rototye of a user interface. This aer is organized as follows: Section 2 exlains fuzzy odeling and its interactive otiization rocess. Section 3 exlains the tacled roble and deonstrates the effectiveness of our ethod with a rototye of our user interface. Section 4 concludes this aer. 2 Interactive Fuzzy Modeling 2. Fuzzy Modeling In this aer, for an n-inut and single-outut nonlinear function y = y(x), we use the following fuzzy if-then rules: Rule R : If x is A and... and x n is then y is B,, A n 2,...,, () where x i is the i-th inut variable of an n-diensional inut vector x = (x,, x n ), y is an outut variable, is a rule index, A i is an antecedent linguistic label (e.g., sall and large) for 839

2 x i, B is a consequent linguistic value, and is the total nuber of fuzzy if-then rules. The following fuzzy reasoning ethod has been frequently used in fuzzy rule-based systes since its first roosal in a neuro-fuzzy syste [42]: yˆ( x) ) b ), (2) where (x) is the coatibility grade of the fuzzy if-then rule R with the inut vector x, and b is a reresentative real nuber of the consequent linguistic value B. The coatibility grade (x) is usually calculated by the roduct oeration as x ) ) ), (3) ( n n where i () is the ebershi function of the antecedent linguistic value A i. The reresentative real nuber b can be viewed as a result of the defuzzification of the consequent linguistic value B. The fuzzy reasoning ethod in (2) can be viewed as a silified version of the Taagi-Sugeno (TS) odel where a linear function is used in the consequent art of each fuzzy if-then rule. The silified fuzzy reasoning ethod in (2) has several advantages. For exale, its reasoning echanis is very sile, and it is suitable for gradient-based learning algoriths. Since we use ultile granularities of fuzzy sets for an inut vector, soetie the effect of secific rules becoes lower due to general rules. For giving secific rules ore weight, we use an idea of inclusion-based fuzzy reasoning [43]. We extend the inclusion relation RA RB in [43]. When only the two rules R and R q with the relation A > A q, are coatible with the inut vector x, the secific rule R q is ainly used in fuzzy reasoning. That is, the weight of the general rule R is discounted. Our idea is to deterine the aount of the discount for R using the coatibility grade q (x) of the secific rule R q with the inut vector x. More secifically, the weight of R is defined as ( q (x)). When the secific rule R q is fully coatible with the inut vector x, the weight of the general rule R is zero. This eans that R has no effect on the calculation of the estiated outut value y ˆ( x). On the other hand, when the coatibility grade of R q with x is very sall, the aount of the discount for R is also very sall. In this case, R has alost the sae weight as R q. Since the general rule R ay include ultile rules, its weight is defined as w( R, x) ( )). (4) q Aq A q When there are no coatible fuzzy if-then rule saller than R, w(r, x) is secified as w(r, x) = because the weight of R should not be discounted in this case. It should be noted that the weight of each rule deends on the coatibility grades of other rules with the inut vector x. This eans that the weight is context-deendent. Different weights are assigned to the sae rule for different inut vectors. Moreover, the sae rule ay have different weights for the sae inut vector in different rule bases because the weight of each rule deends on other rules. Using the rule weight w(r, x) of each fuzzy if-then rule R, our inclusion-based fuzzy reasoning ethod is written as yˆ( x) w( R, x) ) b w( R, x) ). (5) 2.2 Multiobjective Fuzzy Rule Selection for Modeling We use a sile two-stage ethod for designing rule sets. In the first hase, a large nuber of candidate rules are generated fro the ossible cobinations of ebershi functions. The consequent real nuber is secified as the weighted average of outut values of coatible inut-outut airs as b A A ( ) y x ), (6) where A ) is the coatibility grade of the inut vector x with the antecedent art A of the linguistic rule R. In the second hase of our rule selection, a nuber of fuzzy rule sets are selected by a ultiobjective genetic algorith. Any subset S of the candidate rules can be reresented by a binary string of length as S s s, (7) 2 s where s = and s = reresent the inclusion of the -th candidate rule R in S and the exclusion of R fro S. Each fuzzy rule set S is evaluated by the three objectives: f (S): the total square error by S, f 2 (S): the nuber of selected fuzzy rules in S, f 3 (S): the overall user reference for S. The first and second objectives have been frequently used and corresond to accuracy axiization and colexity iniization, resectively. The first objective is calculated by, 2 f ( S) ( y yˆ )). (8) The third objective f 3 (S) is the newly roosed objective in this aer. We exlain it in the next subsection. The roble forulation of ultiobjective genetic fuzzy rule selection is written as Miniize f (S) and f 2 (S), and axiize f 3 (S). (9) 84

3 We use SGA-II of Deb et al. [27] to search for a nuber of non-doinated fuzzy rule-based systes with resect to these three objectives. In this aer, unifor crossover and biased bit-fli utation are used in SGA-II. The biased utation is that a larger robability is assigned to the utation fro to than that fro to. Figure shows the whole rocedure of the roosed ethod. We secify an interval (i.e., the nuber of generations) for internal evaluations. During this interval, the satisfaction level functions are not changed. After the interval, the user checs soe of non-doinated rule sets and odifies the satisfaction level functions. Then another internal evaluation rocess starts. By reeating this interactive rocess, the user can secify the own reference and find the rule set with the high user reference value. o Initialization Evaluation Calculation of nondoinated raning & crowding distance Generation udate Is internal terination condition satisfied Yes Dislay of nondoinated rule sets Modification of satisfaction level functions Is terination condition satisfied Yes Figure : The whole rocedure of the roosed ethod. 2.3 Preference Function In our forer study [36, 37], we have roosed a reference function coosed of several satisfaction level functions. The inuts for the satisfaction level functions are criteria on the accuracy and interretability of fuzzy rule sets. Each satisfaction level function is reresented by a traezoidal function in Fig. 2. g r (S) is the value of r-th criterion for the rule set S. u r (g r (S)) is the outut of r-th satisfaction level function. Users secify the reference and riority for each criterion by oving the oint B in Fig. 2. That is, the u x and u y of the oint B ean the reference and riority for the criterion, resectively. The u x can be also regarded as the axiu criterion level. A (, u r y ) u r (g r (S)) B (u r x, ur y ) g r (S) C (, ) Figure 2: A traezoidal function for reresenting satisfaction level functions of each criterion. o The satisfaction level function can be viewed as the requireent level. During evolution, users can odify the satisfaction level function according to the teorally obtained non-doinated rule sets. The third objective function f 3 (S) for an overall user s reference is calculated by C r S r r f ( S u g, () 3 ) where c is the nuber of criteria. 3 Case Study 3. Proble Descrition We aly our roosed ethod to a sile fuzzy odelling roble for tie-series data. The data we used in this aer is the land rice oveents of the three ajor etroolitan areas in Jaan available fro Ministry of Land, Infrastructure, Transort and Touris webage (htt:// index_e.htl). The data includes the land rice oveents fro 98 to 2. In this eriod, the bubble econoy was a big roble: the increased deand for office buildings in city centres due to internationalization and inforatization. The data is coosed of 63 airs of two inuts (i.e., area and year) and one outut (change of land rice). For silicity, we noralized the inut attributes into [, ]x[, ] sace. We used seven categorical values (all ossible cobinations) for area attribute. For year attribute, we used 48 fuzzy ebershi functions shown in Fig. 4 and don t care condition. Each ea of triangular ebershi functions corresonds to one of years. These artitions could be understandable (e.g., 9 s, Mid of 8 s, around 988). Fro this data, 343 fuzzy if-then rules were generated as candidate rules. Thus, the search sace is Year-on-year change of land rices (%) Toyo area Osaa area agoya area Year Figure 3: Year-on-year change of land rices of three ajor etroolitan areas in Jaan fro 98 to Criteria for Reresenting User s Preference There are a lot of interretability easures in the literature. In this case study, we used four sile criteria lie: - Maxiu square error by S, - Overla aong antecedent sets in S, - Total square error by S, - The nuber of fuzzy rules in S. 84

4 The axiu square error is calculated by g ( S) ax ( y y,..., ˆ )) 2. () When a user gives a high riority to this criterion, the larger changes could be fitted by a fuzzy rule set. We noralized each value of four criteria within the valid ranges based on the distribution values in the re-siulation without user s reference. The reason why we used the total square error and the nuber of fuzzy rules as the criteria for user s reference is to reduce the search sace based on the user s reference. The choice of interretability easures is future research issues. The correlation aong easures ust be exained. Figure 4: Fuzzy artitions with different granularities for Year attribute. 3.3 Prototye of User Interface Figure 5 shows the rototye of our user interface. There are two windows. The left one reresents the actual land rice oveent and the inferred land rice oveent calculated by the chosen fuzzy rule set. At the iddle of the left window, there are two grahs for reresenting non-doinated rule sets in ters of the total square error and the nuber of rules and the total square error and the user reference value. Red oen lots reresent non-doinated rule sets. Blue closed lot eans the chosen rule set. The above inferred land rice oveent corresonds to the chosen rule set. Users can choose one of the non-doinated rule sets by clicing any lot in the grahs. The right CUI window shows the rules in the chosen rule set. At the botto of the left window, there are four satisfaction level functions. Users can change the shaes of these functions by oving each oint B in Fig. 2. The button Evolve is a trigger to start internal evaluations. In this aer, we secified the nuber of generations for internal evaluations as. 3.4 Soe Results When a user secified the satisfaction level functions for the axiu error and the nuber of rules as in Fig. 6, the user obtained a rule set with a sall nuber of rules. The rule set sees to reresent the original characteristics of the data. Fro Table, we can see that there are soe general rules and secific rules in the fuzzy rule set. The value in arentheses in Table reresents the range of.5-level set of used ebershi function. For exale, 99 [] eans the sallest artition in which the ea is 99. Table : Obtained rule set in Fig. 6. Area Year Change % Osaa 99 [] 56. Osaa 992 [] Toyo 988 [] 68.6 Toyo & Osaa 993 [] -5.9 Osaa & agoya 99 [2] 28.4 Toyo & Osaa & agoya 988 [4] 7.2 Toyo & Osaa & agoya Figure 5: Prototye of user interface for interactive fuzzy odelling. 842

5 When a user gave high riorities to accuracy criteria (i.e., axiu error and total square error), a very accurate rule set was obtained as in Fig. 8. Figure 6: Exale. When a user gave a high riority to Overla aong antecedent sets in S as in Fig. 7, the user obtained a rule set with only one general rule and six secific rules (see Table 2). Figure 8: Exale 3. 4 Conclusions In this aer, we incororated user s reference into ultiobjective fuzzy odeling. We roosed a reference function coosed of four satisfaction level functions. We utilized this reference function as an additional objective in an EMO algorith. Through a case study, we deonstrated that a user can interactively secify satisfaction level functions during the evolution. We also showed that the user can obtain an accurate and interretable fuzzy rule-based syste based on his/her own reference. In our case study, we intuitively selected four criteria to reresent user s reference. Further studies are needed to choose aroriate criteria. We also have other interesting research issues to be discussed in future studies such as the visualization of ulti-diensional data and the iniization of huan user s fatigue caused by the interaction with our syste. The latter includes autoated reference odeling. Figure 7: Exale 2. Table 2: Obtained rule set in Fig. 7. Area Year Change % Osaa 989 [] 32.7 Osaa 99 [] 56. Osaa 992 [] Toyo 988 [] 68.6 Toyo 989 [].4 Toyo & Osaa 993 [] -5.9 Toyo & Osaa & agoya Acnowledgent This wor artially suorted by Grand-in-Aid for Young Scientists (B): KAKEHI (87228). References [] H. Ishibuchi, K. ozai,. Yaaoto, and H. Tanaa, Construction of fuzzy classification systes with rectangular fuzzy rules using genetic algoriths, Fuzzy Sets and Systes 65 (2/3) , 994. [2] H. Ishibuchi, K. ozai,. Yaaoto, and H. Tanaa, Selecting fuzzy if-then rules for classification robles using genetic algoriths, IEEE Trans. on Fuzzy Systes 3 (3) 26-27, 995. [3] H. Ishibuchi, T. Murata, and I. B. Tursen, Single-objective and twoobjective genetic algoriths for selecting linguistic rules for attern classification robles, Fuzzy Sets and Systes 89 (2) 35-5,

6 [4] M. Setnes, R. Babusa, and B. Verbruggen, Rule-based odeling: Precision and transarency, IEEE Trans. on Systes, Man, and Cybernetics - Part C 28 () 65-69, 998. [5] J. Yen and L. Wang, G. W. Gillesie, Iroving the interretability of TSK fuzzy odels by cobining global learning and local learning, IEEE Trans. on Fuzzy Systes 6 (4) , 998. [6] D. auc and R. Kruse, Obtaining interretable fuzzy classification rules fro edical data, Artificial Intelligence in Medicine 6 (2) 49-69, 999. [7] Y. Jin, Fuzzy odeling of high-diensional systes: Colexity reduction and interretability iroveent, IEEE Trans. on Fuzzy Systes 8 (2) 22-22, 2. [8] M. Setnes and H. Roubos, GA-based odeling and classification: Colexity and erforance, IEEE Trans. on Fuzzy Systes 8 (5) , 2. [9] H. Ishibuchi, T. aashia, and T. Murata, Three-objective genetics-based achine learning for linguistic rule extraction, Inforation Sciences 36 (-4) 9-33, 2. [] L. Castillo, A. Gonzalez, and R. Perez, Including a silicity criterion in the selection of the best rule in a genetic fuzzy learning algorith, Fuzzy Sets and Systes 2 (2) 39-32, 2. [] H. Roubos and M. Setnes, Coact and transarent fuzzy odels and classifiers through iterative colexity reduction, IEEE Trans. on Fuzzy Systes 9 (4) , 2. [2] J. Abonyi, J. A. Roubos, and F. Szeifert, Data-driven generation of coact, accurate, and linguistically sound fuzzy classifiers based on a decision-tree initialization, International Journal of Aroxiate Reasoning 32 () -2, 23. [3] H. Ishibuchi and T. Yaaoto, Fuzzy rule selection by ulti-objective genetic local search algoriths and rule evaluation easures in data ining, Fuzzy Sets and Systes 4 () 59-88, 24. [4] J. Casillas, O. Cordon, M. J. del Jesus, and F. Herrera, Genetic tuning of fuzzy rule dee structures reserving interretability and its interaction with fuzzy rule set reduction, IEEE Trans. on Fuzzy Systes 3 () 3-29, 25. [5] R. Alcala, J. Alcala-Fdez, J. Casillas, O. Cordon, and F. Herrera, Hybrid learning odels to get the interretability-accuracy trade-off in fuzzy odeling, Soft Couting (9) , 26. [6] P. Pulinen and H. Koivisto, Identification of interretable and accurate fuzzy classifiers and function estiators with hybrid ethods, Alied Soft Couting 7 (2) , 27. [7] H. Ishibuchi and Y. ojia, Analysis of interretability-accuracy tradeoff of fuzzy systes by ultiobjective fuzzy genetics-based achine learning, International Journal of Aroxiate Reasoning 44 () 4-3, 27. [8] M. Cococcioni, P. Ducange, B. Lazzerini, and F. Marcelloni, A Paretobased ulti-objective evolutionary aroach to the identification of Madani fuzzy systes, Soft Couting () 3-3, 27. [9] A. F. Goez-Sareta, F. Jienez, and G. Sanchez, Iroving interretability in aroxiative fuzzy odels via ultiobjective evolutionary algoriths, International Journal of Intelligent Systes 22 (9) , 27. [2] J. Gonzalez, I. Rojas, H. Poares, L. J. Herrera, A. Guillen, J. M. Paloares, and F. Rojas, Iroving the accuracy while reserving the interretability of fuzzy function aroxiators by eans of ulti-objective evolutionary algoriths, International Journal of Aroxiate Reasoning 44 () 32-44, 27 [2] A. Botta, B. Lazzerini, F. Marcelloni, and D. C. Stefanescu, Context adatation of fuzzy systes through a ulti-objective evolutionary aroach based on a novel interretability index, Soft Couting 3 (5) , 29. [22] L. Sanchez, J. Otero, and I. Couso, Obtaining linguistic fuzzy rule-based regression odels fro irecise data with ultiobjective genetic algoriths, Soft Couting 3 (5) , 29. [23] O. Cordon, F. Herrera, F. Hoffann, and L. Magdalena, Genetic Fuzzy Systes, World Scientific, 2. [24] O. Cordon, F. Goide, F. Herrera, F. Hoffann, and L. Magdalena, Ten years of genetic fuzzy systes: Current fraewor and new trends, Fuzzy Sets and Systes 4 () 5-3, 24. [25] F. Herrera, Genetic fuzzy systes: Taxonoy, Current research trends and rosects, Evolutionary Intelligence, 27-46, 28. [26] K. Deb, Multi-objective otiization using evolutionary algoriths. John Wiley & Sons, Chichester, 2. [27] K. Deb, A. Prata, S. Agarwal, and T. Meyarivan, A fast and elitist ultiobjective genetic algorith: SGA-II, IEEE Trans. on Evolutionary Coutation 6 (2) 82-97, 22. [28] C. A. C. Coello, D. A. van Veldhuizen, and B. B. Laont, Evolutionary algoriths for solving ulti-objective robles. Kluwer Acadeic Publishers, Boston, 22. [29] S. Guillaue, Designing fuzzy inference systes fro data: An interretability-oriented review, IEEE Trans. on Fuzzy Systes 9 (3) , 2. [3] R. Miut, J. Jael, and L. Groll, Interretability issues in data-based learning of fuzzy systes, Fuzzy Sets and Systes 5 (2) 79-97, 25. [3] C. Mencar, G. Castellano, and A. M. Fanelli, On the role of interretability in fuzzy data ining, International Journal of Uncertainty Fuzziness and Knowledge-Based Systes 5 (5) , 27. [32] C. Mencar and A. M. Fanelli, Interretability constraints for fuzzy inforation granulation, Inforation Sciences 78 (24) , 28. [33] S. M. Zhou and J. Q. Gan, Low-level interretability and high-level interretability: a unified view of data-driven interretable fuzzy syste odelling, Fuzzy Sets and Systes 59 (23) 39-33, 28. [34] J. M. Alonso, L. Magdalena, and S. Guillaue, HILK: A new ethodology for designing highly interretable linguistic nowledge bases using the fuzzy logic foralis, International Journal of Intelligent Systes 23 (7) , 28. [35] H. Ishibuchi and Y. ojia, Discussions on interretability of fuzzy systes using sile exales, Proc. of 29 IFSA World Congress and 29 EUSFLAT conference (to aear). [36] Y. ojia and H. Ishibuchi, Incororation of user reference into ultiobjective genetic fuzzy rule selection for attern classification robles, Proc. of 4th International Syosiu on Artificial Life and Robotics 29, 86-89, 29. [37] Y. ojia and H. Ishibuchi, Interactive genetic fuzzy rule selection through evolutionary ultiobjective otiization with user reference, Proc. of 29 IEEE Syosiu on Coutational Intelligence in Multicriteria Decision-Maing, 4-48, 29. [38] H. Taagi, Interactive evolutionary coutation: fusion of the caabilities of EC otiization an huan evaluation, Proceedings of the IEEE, 8 (9) , 2. [39] S. B. Cho, Towards creative evolutionary systes with interactive genetic algorith. Alied Intelligence 6: 29-38, 22. [4] A. M. Brintru, J. Rasden, H. Taagi and A. Tiwari, Ergonoic chair design by fusing qualitative and quantitative criteria using interactive genetic algoriths. IEEE Trans. on Evolutionary Coutation 2 (3) , 28. [4]. Kubota, Y. ojia, F. Kojia, and T. Fuuda, Multile fuzzy statevalue functions for huan evaluation through interactive trajectory lanning of a artner robot, Soft Couting (), 89-9, 26. [42] H. Ichihashi, Iterative fuzzy odeling and a hierarchical networ, Proc. of 4th IFSA world congress, 49-52, 99. [43] H. Ishibuchi, T. Yaaoto, and T. aashia, An aroach to fuzzy default reasoning for function aroxiation, Soft Couting (9), ,