Image Classification Using Feature Subset Selection

Transcription

1 Proceedngs of the 5th WSEAS Int. Conf. on COMPUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS and CYBERNETICS, Vence, Italy, November 20-22, Image Classfcaton Usng Feature Subset Selecton SANG-SUNG PARK 1, KWANG-KYU SEO 2, HO-SEOK MOON 1, YOUNG-GEUN SHIN 1, DONG-SIK JANG 1 1 Industral Systems and Informaton Engneerng, Korea Unversty 1, 5-ka, Anam-Dong, Sungbuk-Ku, Seoul , KOREA formaton and Systems Engneerng, Sangmyung Unversty San 98-20, Anso Abstract: Classfcaton technology s essental for fast retreval n large database. Ths paper proposes a combnng GA and SVM model to content-based mage retreval. The proposed method s also used to classfcaton smlar mages from database. Jont HSV hstogram and average entropy computed from gray-level co-occurrence matrces n the localzed mage regon s employed as nput vectors. Genetc algorthm s employed to select feature subsets elmnated rrelevant factors as used nputs and to determne the optmal parameters of Support Vector Machne. Expermental results show that the proposed model outperforms exstng method. Key-Words: SVM, Genetc Algorthm, CBIR, Feature Selecton, Image 1 Introducton The development of nformaton technologes makes the demand of multmeda n-formaton servces sgnfcant. Recent research on retreval methods has become very mportant for mage and vdeo searches. In ths paper, we deal wth content-based mage retreval, whch s a technque to retreve mages based on ther vsual propertes such as color [1], texture [2], and shape [3, 4]. Systems [5, 6, 7] are well known for supportng ths content-based mage retreval. Fast retreval n databases has been one of the actve research areas. In that process, wthout any clusterng schemes and adequate ndexng structures, retrevals of smlar mages are tme-consumng because the database system must compare the query mage to each mage n the database. Ths cost can be partcularly prohbtve f the database mages are very large and ther features tend to have hgh-dmensonalty. Ths hgh-dmensonal ndexng structure ncreases the retreval tme and memory space exponentally, as the member of feature dmenson ncreases. Thus, frequently, t does not have any advantages aganst the smple sequental search. So, fast search algorthms, whch can deal wth hgh-dmensonal feature data, are often an essental component of the mage database. There have been a number of ndexng data structures suggested to handle hgh-dmensonal data [8, 9, 10]. In order to classfy mages effcently, we need to learn the prevous mage patterns. Ths can mprove the accuracy of mage classfcaton and detecton. In addton, we need to classfy the mages from a large and complex database. In ths respect, we propose a new mage classfcaton technque based on SVM (Support Vector Ma-chne) that s useful for speedly fndng the mages from a large mage database sys-tem. In ths scheme, smlar mages are classfed based on the mage feature and assocated classfcaton algorthm. When the query s presented, smlar mages to the query are retreved only from the most smlar cluster to the query, thus full-database searches are not necessary. We use a hybrd model wth combnng GA(Genetc Algorthm) and SVM as clusterng technque for narrowng the search space. GAs are computatonal models of evoluton. They work on the bass of a set of canddate solutons. The SVM s a tranng algorthm for learnng classfcaton and regresson rules from data. In ths study, GA s employed to select feature subsets elmnated rrelevant factors as used nputs and to determne the optmal parameters of SVM. 2 Image Features In order to perform the content-based mage retreval, features whch are representatve of mage content,

2 Proceedngs of the 5th WSEAS Int. Conf. on COMPUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS and CYBERNETICS, Vence, Italy, November 20-22, should be extracted. In ths paper, color and texture nformaton are used to represent mage features. For color, ont HSV hstogram extracted from local regon s employed. For texture, entropes computed from local regon are employed. These features extracted from each mage n the database are used as nput vector to the classfer. Color: For representng color, we used HSV (Hue, Saturaton, Value) color model because ths model s closely related to human vsual percepton. For color quantzaton, we unformly quantzed HSV space nto 18 bns for hue (each bn consstng of a range of 20 degree), 3 bns for saturaton and 3 bns for value for lower resoluton. In order to represent the local color hstogram, we dvded mage nto equal-szed rectangular regons and extract HSV ont hstogram that has quantzed 162 bns for each regon. And to obtan compact representaton, we extract from each ont hstogram the bn that has the maxmum peak. The HSV representaton of an mage from RGB s obtaned usng the followng relatonshps: θ, G B, H = 2 π θ G B, 1 [( R G ) + ( R B )] where θ 2 = [( R G) + ( R B)( G B)] S 3 R G B 1 cos 1, 2 2 = 1 [ mn(, ] + + (1) R G, B ), (2) 1 V = ( R+ G+ B). (3) 3 Take hue, saturaton, and value assocated to the bn as representng features n that rectangular regon and normalze to be wthn the same range of [0,1]. Thus, each mage has the dmensonal color vector. Texture: Most natural mages nclude textures. Scenes contanng pctures of wood, grass, etc. can be easly classfed based on the texture rather than color or shape. Therefore, t may be useful to extract texture features for mage clusterng. Lke as color feature, we nclude a texture feature extracted from localzed mage regon. As a texture feature, we used the entropy extracted from the co-occurrence matrx [5]. Detaled feature extracton s performed as follows: 1. Converson of color mage to gray mage 2. Dvdng mage nto 3 3 rectangular regons as n color case. 3. Obtanng co-occurrence matrx for four (horzontal 0 0, vertcal 90 0 and two dagonal 45 0 and ) orentaton n regon and normalze entres of four matrces to [0, 1] by dvdng each entry by total number of pxels. 4. Extractng average entropy value from four matrces. e = k p(, ) log(p(, )), 4 k = 1, 2,3, 4 5. Constructng texture feature vector by concatenatng entropes over all rectangular regons. Thus, each mage has the 3 3(=9) dmensonal texture vector. 3 GA(Genetc Algorthm) GAs are computatonal models of evoluton. They work on the bass of a set of canddate solutons. Each canddate soluton s called a 'chromosome', and the whole set of solutons s called a 'populaton'. The algorthm allows movement from one populaton of chromosomes to a new populaton n an teratve fashon. Each teraton s called a 'generaton'. There are varous forms of GAs, a smple verson, whch s called statc populaton model was used n all the experments [11, 12]. In the statc populaton model, the populaton s ranked accordng to the ftness value of each chromosome. At each generaton, two (and only two) chromosomes are selected as parents for reproducton. GAs operate teratvely on a populaton of structures, each one of whch represents a canddate soluton to the problem at hand, properly encoded as a strng of symbols (e.g.,bnary). A randomly generated set of such strngs forms the ntal populaton from whch the GA starts ts search. Three basc genetc operators gude ths search: selecton, crossover, and mutaton. The genetc search process s teratve: evaluatng, selectng, and recombnng strngs n the populaton durng each teraton (generaton) untl reachng some termnaton condton. The basc algorthm, where P(t) (4)

3 Proceedngs of the 5th WSEAS Int. Conf. on COMPUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS and CYBERNETICS, Vence, Italy, November 20-22, s the populaton of strngs at generaton t, s gven below: t = 0 ntalze P(t) evaluate P(t) whle (termnaton condton s not satsfed) do begn select P(t + 1) from P(t) recombne P(t + 1) evaluate P(t + 1) t = t + 1 end Evaluaton of each strng s based on a 1tness functon that s problem-dependent. It determnes whch of the canddate solutons are better. Ths corresponds to the envronmental determnaton of survvablty n natural selecton. Selecton of a strng, whch represents a pont n the search space, depends on the strng s 1tness relatve to those of other strngs n the populaton. It probablstcally removes, from the populaton, those ponts that have relatvely low ftness. Mutaton, as n natural systems, s a very low probablty operator and ust flps a specfc bt. Mutaton plays the role of restorng lost genetc materal. Crossover n contrast s appled wth hgh probablty. It s a randomzed yet structured operator that allows nformaton exchange between ponts. Its goal s to preserve the fttest ndvduals wthout ntroducng any new value. capacty control to prevent over-fttng and thus s a partal soluton to the bas-varance trade-off dlemma. Two key elements n the mplementaton of SVM are the technques of mathematcal programmng and kernel functons. The parameters are found by solvng a quadratc programmng problem wth lnear equalty and nequalty constrants; rather than by solvng a non-convex, unconstraned optmzaton problem. The flexblty of kernel functons allows the SVM to search a wde varety of hypothess spaces. For constructng the decson rules, four common types of SVM are gven as follows: T - Lnear: K ( x, x ) = x x (5) T d - Polynomal: K ( x, x ) = ( x x + r) (6) - Radal bass functon (RBF): K ( x, x 2 2 / 2δ ) = exp( x x ) (7) T - Sgmod: K( x, x ) = tanh( x x r) (8) + 5 Proposed Algorthm GA Step Populaton SVM Classfcaton Step 4 SVM(Support Vector Machne) The support vector machne (SVM) [13, 14, 15] s a tranng algorthm for learnng classfcaton and regresson rules from data, for example the SVM can be used to learn polynomal, radal bass functon (RBF) and mult-layer perceptron (MLP) classfers. SVMs were frst suggested by Vapnk n the 1960s for classfcaton and have recently become an area of ntense research owng to developments n the technques and theory coupled wth extensons to regresson and densty estmaton. SVMs arose from statstcal learnng theory; the am beng to solve only the problem of nterest wthout solvng a more dffcult problem as an ntermedate step. SVMs are based on the structural rsk mnmzaton prncple, closely related to regularzaton theory. Ths prncple ncorporates No Generaton Ftness measurement Selecton Crossover Mutaton Ftness measurement Stop rule Yes End Feature-vector rearrange Classfcaton Accuracy measurement Best Classfers Fg 1. The flow chart of proposed algorthm

4 Proceedngs of the 5th WSEAS Int. Conf. on COMPUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS and CYBERNETICS, Vence, Italy, November 20-22, Ths paper proposes a hybrd model wth combnng GA and SVM. In ths study, GA s employed to select feature subsets elmnated rrelevant factors as used nputs and to determne the optmal parameters of SVM. The flow chart of the proposed algorthm s depcted n Fg.1. The proposed algorthm n Fg 1 s to optmze SVM s varables and nput data for Image retreval. The procedure of proposed algorthm begns by selectng random chromosome n the populaton whch s represented by strng nput data and SVM varables. Each strngs wll sent to SVM classfer and evaluate the ftness. The SVM model s used to obtan ht raton of each chromosome. The ftness n ths study s gven below: 1 F = λ1q1+ λ2 (9) Q 2 Q 1 s accuracy of each class whch s classfed by usng subset. Q2 s number of selected egenvector. We defne λ 1 s 100 and λ 2 s 10 n the experment. 6 Experments To show the effectve classfcaton of the proposed method, we checked the classfcaton accuracy. All experments were performed on a Pentum IV wth 512 Mbytes of man memory and 100Gbytes of storage. All programs have been mplemented n Vsual C++. We expermented on 1,200 mages where most of them have dmensons of pxels. The 1,200 mages can be dvded nto 6 categores each wth 200 mages such as horse, rose, polar bear, sunset, valley and dolphn. We performed two experments: 1) Classfcaton results accordng to kernel of dfferent types. Image Type of kernel Tranng (%) Testng (%) Horse Rose Polar - Bear Sunset Valley Dolphn Lnear Polynomal RBF Sgmod Lnear Polynomal RBF Sgmod Lnear Polynomal RBF Sgmod Lnear Polynomal RBF Sgmod Lnear Polynomal RBF Sgmod Lnear Polynomal RBF ) Classfcaton results usng SVM classfer and proposed classfer. Average Sgmod Lnear As shown n Table 1, both tranng and test success rates that were acheved under each dfferent method. As can be seen, proposed classfcaton wth RBF kernel has consstently gven the best performance of other. The average classfcaton of 6 classes wth RBF kernel acheves 96.93% success on the tranng set and % wth the test set. Polynomal RBF Sgmod Table 1. The performance of proposed classfcaton accordng to kernel type

5 Proceedngs of the 5th WSEAS Int. Conf. on COMPUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS and CYBERNETICS, Vence, Italy, November 20-22, Table 2 shows classfcaton results usng SVM classfer and proposed classfer. SVM classfcaton shows average accuracy 91.65%, whereas proposed classfcaton shows average accuracy 93.87%. Image SVM(%) GA+SVM(%) Horse Rose Polar Bear Sunset Valley Dolphn Average Table 2. Classfcaton results usng SVM classfcaton and proposed classfcaton 7 Concluson In ths paper proposes a combnng GA and SVM model for content-based mage retreval. As nput elements to system, domnant trple (hue, saturaton, and value) whch are extracted from quantzed HSV ont hstogram are used for representng color nformaton and average entropy computed gray-level co-occurrence matrces are used for texture nformaton n the mage. The proposed method served to exemplfy that kernel-based learnng algorthms are ndeed hghly compettve on varety problems wth dfferent characterstcs and can be employed as an effcent method for CBIR. The study needs further research as follows. The selecton of the kernel functon and the determnaton of optmal values of the parameters have a crtcal mpact on the performance n SVM. Therefore t s necessary to nvestgate to develop a structured method of selectng an optmal value of parameters and kernel functon n SVM for the best predcton performance. In addton, we develop the generalzaton of SVM on the bass of the approprate level of the tranng set sze and gve a gudelne to measure the generalzaton performance. Acknowledgement: Ths work was supported by the Bran Korea 21 Proect n References: [1] Smth, J.R., Chang, S.F., Tools and technques for color mage retreval, In Proc. SPIE: Storage and Retreval for Image and Vdeo Databases IV, Vol. 2670, 1996, pp [2] Manunath, B.S., Ma, W.Y., Texture features for browsng and retreval of mage data, Tech. Rep. CIPR TR, 95-06, [3] Jan, A.K., Valaya, A., Shape-based retreval: A case study wth trademark mage databases. Pattern Recognton, Vol. 31, No. 9, 1998, pp [4] Swan, M., Ballard, D., Color ndexng, Internatonal Journal of Computer Vson, Vol. 7, No. 1., 1991, pp [5]Flckner, M., Sawhney, H., Nblack, W., Ashley, J., Huang, Q., Dom, B., Gorkan, M., Hafer, J., Lee, D., Petkovc, D., Steele, D., Yanker, P., Query by mage content: The QBIC system, IEEE Computer, Vol. 28, No. 9., 1995, pp [6] Smth, J.R., Chang, S.E., VsualSEEK: A fully automated content-based mage query system, In Proc. ACM Multmeda, 1996, pp [7] Carson, C., Belonge, S., Greenspan, H., Malck, J, Blobworld: Image segmentaton usng expectaton-maxmzaton and ts applcaton to mage queryng, IEEE Trans on Pattern Analyss and Machne Intellgence, Vol. 24, No. 8., 2002, pp [8] Whte, D.A., Jan, R., Smlarty ndexng wth the SS-tree, In Proc. 12th IEEE Inter-natonal Conference on Data Engneerng, 1996, pp [9] Ln, K.I., Jagadsh, H.V., Faloutsos, C., The TV-tree: An ndex structure for hgh-dmensonal data, VLDB Journal, Vol. 3, No. 4., 1994, pp [10]Berchtold, S., Kem, D.A., Kregel, H.P., The X-tree: An ndex structure for hgh-dmensonal data, In Proc. 22th Int. Conf. on Very Large Data Bases, 1996, pp [11]D. Whney, A genetc algorthm tutoral. Techncal Report, Department of computer scence, Colorado state unversty, 1993, CS [12]R. L. Haupt, An ntroducton to genetc algorthms for electromagnetcs, IEEE Magazne, Antennas Propagaton, Vol. 37., 1995, pp.7-15.

6 Proceedngs of the 5th WSEAS Int. Conf. on COMPUTATIONAL INTELLIGENCE, MAN-MACHINE SYSTEMS and CYBERNETICS, Vence, Italy, November 20-22, [13]V. Vapnk., Statstcal Learnng Theory, Sprnger, New York, [14]H. Drucker, D. Wu, and V.N. Vapnk., Support vector machnes for spam catergor-zaton, IEEE Transactons on Neural Networks, Vol. 10, No. 5., 1999, pp [15]A. Fan and M. Palanswam., Selectng bankruptcy predctors usng a support vector machne approach, Proceedngs of the Internatonal Jont Conference on Neural Networks, 2000.