2002 Journal of Software.. (stacking).

Transcription

1 /2002/13(02) Journal of Software Vol13, No2,,, (,200433) : (GA) (stacking), 2,,, : ; ; ; ; : TP18 :A, [1],,, :,, :,,,, [2],,,, :,,,,, (plurality voting, PV), SCANN(stacking,correspondence analysis,nearest neighbor) [3],, :, (stacking) [4],, SCANN, (level-1 induction),, 1 : ; : : ( ); 973 (G ) : (1971 ),,,, Web, ; (1965 ),,,,,,,Web ; (1963 ),,,,,, ; (1966 ),,

2 246 Journal of Software 2002,13(2) 1 11 (1),L={(x m,y m ),m=1,,m}, x i,y i,m (2) A, N,A 1,A 2,,A N A R S,A=(R,S) (3), 12 Fig 1 [4], [4] Final predictor Genetic algorithm induction Level-1 :L 1 ={r 11, r 12,, r 1k, r 21, r 22,,r 2k,, r n1, r n2,, r nk } Rule generator Level-0 :A={(R 1,S 1 ), (R 2,S 2 ),, (R n, S n )} Level-0 :L 0 ={(X 1,Y 1 ), (X 2,Y 2 ),, (X M,Y M )}, 1 0 (level-0 induction),,,,,, Level-1 Level-1,,,,, Flowchart of combining method based on genetic algorithm 1 13 (1) Goldberg GA,,,, : if(v 1L L A 1 R V 1R )and(v 2L L A 2 R V 2R )and and(v nl L A n R V nr ) then C j, A i,v il,v ir, L, R,C j (2) :,, 1;, 1;, (3),,,,,,,, ; (4) : [2] :,

3 : 247 :, 2 21, :(1) Iris - Irvine ( Agrawal [5] Iris C50( 1 Table 1 C50-Based rule set of Iris 1 C50 Iris Rule 1:( 35 ) Petal-Length <= 19 class Iris-Setosa Rule 2:( 32 ) Petal-Length > 19 Petal-Length <= 5 Petal-Width <= 16 class Iris-Versicolor Rule 3:( 29 ) Petal-Width > 16 class Iris-Virginica Rule 4:( 28 ) Petal-Length > 5 class Iris-Virginica Default class:iris-setosa, C50, : 2, Petal-Length 491, 4, 470, 2 C50 Table 2 Accuracy of C50 rules and GA-optimizing rules on Iris 2 Iris C50 GA C50 rule set GA rule after optimization Training set Testing set Training set Testing set Setosa 25/25 25/25 25/25 25/25 Versicolor 24/25 23/25 24/25 23/25 Virginica 25/25 24/25 25/25 25/ % 72/75 96% 9867%,, 9733% 2,,Iris-Versicolor Iris-Virginia, ( ) [6],,, SQL,, [5] C45(ID3 ), 18,8 Group A, Group B [7], age salary [7] 6, :(25100<salary<74800) (age>608), ( : 6, salary ) 20, :

4 248 Journal of Software 2002,13(2) (74800<salary<=124000) (39<age<=587), 10, (50100<salary<100200) (age<398) 2, C45, ( ), 23 ( ) C50( ) Iris [8],Iris (NN) : rule 1:Petal-Length<=19 Iris-Setosa rule 2:if Petal-Length<=49 Petal-Width<=16 Iris-Versicolor Default class:iris-virginica C50 1,, C50, 3, 3,, 4,NN, NN, iris-virginica, C50,, Table 3 C50-Based rule set after optimization 3 C50 Rule 1:(cover 35) Petal-Length <= 19 class Iris-setosa Rule 2:(cover 32) Petal-Length>19 Petal-Length<=495 Petal-Width<=16 class Iris- Versicolor Rule 3:(cover 29) Petal-Width>16 class Iris-Virginica Rule 4:(cover 28) Petal-Length>495 class Iris-Virginica Default class:iris-setosa Table 4 Accuracy of multiple algorithms after optimization 4 3 Neural network rule set C50 rule set rule set after optimization Training set Testing set Training set Testing set Training set Testing set Setosa 25/25 25/25 25/25 25/25 25/25 25/25 Versicolor 24/25 23/25 24/25 23/25 24/25 23/25 Virginica 25/25 25/25 25/25 24/25 25/25 25/ % 9733% 9867% 72/75 96% 9867% NN, C50, GA,, 9733%,,,,,,,,,,,,,,,,,,,,,

5 : 249,,,, References: [1] Fayyad, UM, Piatetsky-Shapiro, G, Smyth, P, et al Advances in Knowledge Discovery and Data Mining Cambridge, MA: AAAI/MIT Press, 1996 [2] Goldberg, DE Genetic Algorithms in Search, Optimization, and Machine Learning New York: Addison-Wesley, 1989 [3] Merz, CJ Using correspondence analysis to combine classifiers Machine Learning, 1999,36(1~2):33 58 [4] Wolpert, DH Stacked generalization Neural Networks, 1992,5(2): [5] Agrawal, R, Imielinski, T, Swami, A Database mining: a performance perspective IEEE Transactions on Knowledge and Data Engineering, 1993,5(6): [6] Setiono, R Techniques for extracting rules from artificial neural networks In: Plenary Lecture Presented at the 5th International Conference on Soft Computing and Information Systems Iizuka, Japan, publications html [7] Lu, H, Setiono, R, Liu, H NeuroRule: a connectionist approach to data mining In: Umeshwar, D, Peter, M D, Shojiro, N, eds Proceedings of the 21st VLDB Conference Z rich, Switzerland: Morgan Kaufmann, [8] Setiono, R, Liu, H Understanding neural networks via rule extraction In: Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI 95) Montreal: Morgan Kaufmann, http://wwwinformatikuni-trierde/~ley/ db/conf/ijcai/ijcai95html#setion0l95 A Method to Optimize Classifiers by Using Genetic Algorithms JI Wen-yun, ZHOU Ao-ying, ZHANG Liang, JIN Wen (Department of Computer Science and Engineering, Fudan University, Shanghai , China) {wyji,ayzhou,zhangl}@fudaneducn Abstract: This paper focuses on methods of optimizing a single classifier and combining multiple classifiers by genetic algorithms (GA) The method uses the strategies of stacking There are two steps in classical strategies of stacking, and GA is used as the second step in the method Experimental results show that it performs well on the task of optimization Comparing with the single algorithm, it enhances the precision In task of combining optimization, it can obtain more understandable result than constituent learners Key words: classification; genetic algorithm; optimization; machine learning; data mining; classification rules Received February 15, 2000; accepted July 12, 2000 Supported by the National Natural Science Foundation of China under Grant No ; the National Grand Fundamental Research 973 Program of China under Grant NoG