An Efficient Image Pattern Recognition System Using an Evolutionary Search Strategy

Transcription

1 Proceedngs of the 009 IEEE Internatonal Conference on Systems, Man, and Cybernetcs San Antono, X, USA - October 009 An Effcent Image Pattern Recognton System Usng an Evolutonary Search Strategy Pe-Fang Guo, Prabr Bhattacharya and Nawwaf Kharma Abstract A mechansm nvolvng evolutonary genetc programmng (GP and the expectaton maxmzaton algorthm (EM s proposed to generate feature functons automatcally, based on the prmtve features, for an mage pattern recognton system on the dagnoss of the dsease OPMD. Pror to the feature functon generaton, we ntroduce a novel technque of the prmtve texture feature extracton, whch deals wth nonunform mages, from the hstogram regon of nterest by thresholds (HROI. Compared wth the performance acheved by support vector machne (SVM usng the whole prmtve texture features, the GP-EM methodology, as a whole, acheves a better performance of 90.0% recognton rate on dagnoss, whle proectng the hyperspace of the prmtve features onto the space of a sngle generated feature. Keywords evolutonary computaton, pattern recognton, feature generaton, mage processng, genetc programmng, the expectaton maxmzaton algorthm, Gaussan mxture estmaton, texture analyss, moments I. INRODUCION Feature generaton s generally consdered a process of mappng the prmtve features nto more effectve features durng the run of the algorthms. Many researchers expermented wth varous problems for the technques of feature generaton, such as artfcal neural networ (ANNs for machne condton montorng devces [], fuzzy systems for health montorng systems [], and evolutonary algorthms (EA for obect recognton [3] and the classfcaton of machnery faults [4]. Inspred by bologcal evoluton, EA s a generc populaton-based optmzaton algorthm [5]. Evolutonary genetc programmng (GP, ntroduced by Koza [6], has been wdely employed n varous applcatons. A methodology for onlne feature select (FS and classfer desgn was proposed usng GP n [7]. A progresson of approaches nvolvng GP was descrbed n the problem of the landmar detecton [8]. Generally, the expectaton maxmzaton algorthm (EM can be appled n many settngs where we wsh to estmate some set of parameters that descrbe an underlyng probablty dstrbuton [9]. he EM algorthm has been employed n mssng value predcton [0] and the sgnal detecton problem n wreless communcaton systems []. exture analyss s based on statstcal propertes of the ntensty hstogram. Hstograms have been the most popular strategy for ntensty level computaton owng to ther smplcty and effcency []. However, for mages of dfferent szes and shapes wth localzed bacground structures, the use of the smple whole regons of the channel hstogram for the mage pattern classfcaton would nfluence the classfcaton accuracy. One way to solve ths problem would be to restrct the bn length to a regon that captures the nowledge of the mages wthn that partcular part regardless of the szes, shapes and bacgrounds. hs study s the frst to present a generalzed prmtve texture feature extracton technque based on a hstogram regon of nterest by thresholds (HROI, a bn-length-based technque, for characterzng dagnostcally the dsease OPMD. Further to extend the dea of an automatc mage pattern recognton system, a new technque of the feature generaton, composed of the prmtve features extracted from HROI, s proposed by means of ntegratng GP and the EM algorthm (GP-EM. hs paper s organzed as follows. Secton II ntroduces the prmtve feature preparaton. In Secton III, we summarze the proposed GP-EM system. he EM algorthm, from whch the -means problem s derved, s presented n Secton IV. he GP applcaton s descrbed n Secton V. Expermental results are gven n Secton VI, and the fnal secton s the concluson. II. PRIMIIVE FEAURE PREPARAION he use of the feature generaton requres prmtve features as a prerequste for the subsequent applcatons. In ths study, the frst phase s to prepare our expermental prmtve feature set whch conssts of three steps ( mage grayscale converson; ( HROI defnton; and (3 statstcal texture feature extracton. he procedure s llustrated n Fg., usng the database called CellsDB. Fgure. he procedure of the prmtve feature extracton. Recorded from mcroscopc magery, CellsDB s a database of pre-segmented cell mages dvded nto two categores, /09/$ IEEE 605

2 healthy and sc, assocated wth Oculopharyngeal Muscular Dystrophy (OPMD dsease. Each of the mages n CellsDB, a non-unform mage group, contans a sngle cell on a whte bacground at an optmal level of RGB values. he presence of nclusons n patents cell mages leads to the suggeston that proten aggregaton s a crtcal molecular component of such OPMD dsease. Our conducted research requres the dentfcaton of the nclusons wthn the cell mages. he samples, a healthy cell and a sc cell, from CellsDB are shown n Fg.. Intensty Levels Bns Fgure 3. Grayscale hstograms of all mage samples. (a A typcal healthy cell mage (b A typcal sc cell mage Fgure. Sample mages from CellsDB. A. Cell Image Grayscale Converson Converson of a color mage to grayscale s very common n dgtal mage processng []. o be able to capture the ncluson of color nformaton, we begn our color analyss by convertng the cell mages nto grayscale. he reasons to choose mage grayscale converson are: ( It reduces three D R, G and B hstograms to an ndependent grayscale mage wthout losng any nformaton; ( It fltrates the cell mage bacground n the margn of the hstogram regardless of dfferences n sze and shape. he resultant number of the grayscale cell mages s the desred lnear lumnance value. Each of the grayscale mages s commonly stored wth 8 bts per sampled pxel, whch allows 56 dfferent ntenstes (.e., shades of gray to be recorded. B. HROI Defnton When readng the cell mages, the dfference between the healthy and sc ones s whether the nclusons are nsde the cells or not. Normally, the healthy cell mages are dar green, regardless of ther szes and shapes. he nclusons wthn the sc cells wth a bumpy appearance, whch ndcates the dsease OPMD, are often a lghter green color correspondng to the mddle regon of the grayscale hstogram. If tang nto consderaton the whole regons of the grayscale hstogram, the pxel ntensty can be ms-computed because of dfferences n sze and shape of the cell mages. From ths mscomputaton, we could mss the ncluson ntensty nformaton n terms of ther dscrmnant capabltes,.e., ther sc versus healthy. In ths case, where the lghter nclusons cover the mddle regon of the grayscale hstogram, we use two thresholds to lmt an nterval of gray values, and then defne ths nterval as the hstogram regon of nterest by thresholds (HROI. hereby, HROI s able to capture only the ncluson ntensty nformaton wthn the cell mages and not the cell structural dfferences. As llustrated n Fg. 3, the upper threshold, upper, s set at the bn of 00 by analyzng the output of all the mage grayscale hstograms because there s a zero ntensty regon centered at the value of ths bn AV Bns Fgure 4. he lower threshold, lower, at the bn of 50, defned by the average of varance (AV. Snce pxel ntenstes for both healthy and sc mages overlap n the lower ntensty boundary on the bns ranged (5, 70, we use the varance to determne the lower threshold, lower. In statstcs, the varance s one measure of statstcal dsperson, averagng the squared dstance of ts possble values from the expected value (the mean. he smaller values of the varance represent that the features have relatvely lower varablty to the mean [3]. In Fg. 4, the average of the varances (AV measures the scale or degree of beng spread out of the prmtve texture feature set n the bn range of (5, 70. A feature set wth good stablty and relablty should exhbt a small AV among the samples. he optmal value of the bn should be at the lowest AV, so we set the lower threshold, lower, at the bn of 50. Fg. 5 llustrates the desgn of HROI lmted to a bn range of (50, 00, wth the grayscale hstograms, whch are converted from two typcal cell mages, healthy and sc, shown n Fg.. 606

3 Intensty Levels Bns Fgure 5. HROI representaton, wth the grayscale hstograms of two samples n Fg.. C. Statstc exture Features Extracted from HROI o characterze the complexty of the CellsDB mages, the prmtve feature extracton prmtve feature relates to ther textural characterstcs such as randomness, unformty, etc. Addtonally, the advantage of moments s straghtforward, and moments also carry a physcal nterpretaton of boundary shape. he texture features extracted from HROI are as follows: Entropy (F: measures the randomness or homogenety of the hstogram dstrbuton wth respect to length or orentaton. F LHROI z log z LHROI F medan{ z } where z ndcates ntensty, z s the hstogram of the ntensty levels, and L HROI s the number of ntensty levels wthn HROI. At the end, a total of statstc texture features are normalzed to center t at zero mean and scale t to unt standard devaton. III. HE GP-EM RECOGNIION SYSEM As a result of the prmtve feature data X (the - dmensonal feature data extracted from HROI, the approach to the feature generaton for the gven tas of the dagnoss of the dsease OPMD could be employed. Fg. 6 llustrates the evolutonary process of the GP-EM system. X the GP-EM system Parameter ntalzaton Varaton of the GP tree representatons Gaussan mxture he EM algorthm he -means problem GP applcaton Convergence N G Angular Second Momentum (F: measures the unformty. F L HROI p ( Mean (F3: measures the average of ntensty. L HROI F3 z z z Moments (F4~ F8: measure the fve (second to sxth order moments. L HROI n Fl z z n,3,4,5,6 and l 4,5,6,7,8 Pea Densty (F9: measures the strength of the local domnant pea of the ntensty. F LHROI 9 max{ z } Range of the Densty (F0: measures the range of the ntensty. LHROI F0 range{ z } Medan Densty (F: measures the medan value of the ntensty. the valdaton Y he result of the generated feature, F CellsDB Mnmum dstance classfer (MDC Classfcaton results Fgure 6. he GP-EM recognton system for the CellsDB mages. Suppose we measure the prmtve feature data X wth m samples and the nown classes. he generated feature functons, based on the GP tree representatons, are gong to provde the data mappng from the prmtve feature data X to the generated values, g G. he parametrc EM algorthm assumes that ts nput mapped data G follows a partcular probablty Gaussan dstrbuton for each class for estmaton of the parameters vector = { = (,,, =,, }. Snce each of the Gaussan dstrbutons has the same varance wth unform probablty chosen by each ndependently drawn Gaussan sgnal g, we get the unvarate case that the mean vector,, becomes the unnown parameter vector only. he dervatve of the -means problem va the EM algorthm s descrbed n Secton IV. hen, under the hypothess of the -means problem va the EM, the performance of the GP-EM system s gong through a 607

4 typcal GP process (descrbed n Secton V, ncludng the selecton of genetc operators and the ftness measure. At the end of the evolutonary GP-EM tranng process, the system returns the best tree as the nput to the mnmum dstance classfer (MDC n the valdaton stage. IV. HE K-MEANS PROBLEM VIA HE EM ALGORIHM Gven the data G, a set of m generated values mapped from the data X, the PDF of G, whch follows component fnte mxture dstrbutons f evaluated at g, can be wrtten as []: g / f ( g /,..., ( where s the mxng probablty. he component densty functons f (g / wth a d- dmensonal mean vector and a d X d covarance matrx s: f ( g ex ( g / ( ( g ( where the parameters = {(,,, =,, } are determned to measure how well the correspondng mxture model fts the gven data G. One of the most common methods s to assume equal pror probabltes for all categores by settng P( = / n whch s the number of classes [4]; all the groups have the same covarance matrx and terms nvolvng ths constant can be cancelled, whch means the covarance matrx contans only one component [5]: the varance. herefore, the parameters = {(, =,, } s the complete set that descrbes the means of each of the dstrbutons. he log-lelhood of the probablty of the complete data Y for a mxture of the Gaussan dstrbutons s [6]: ln P( Y / h' m (ln d z ( g ' where the revsed hypothess h' = ( ',, ' and z s a bnary dmensonal vector that has the value f g was created by the th dstrbuton and 0 otherwse. After tang the expected value of ln P(Y / h' and applyng the maxmum lelhood (ML estmator, the EM algorthm repeats the followng steps n the estmaton of the -means problem [6] [9]: E[ z ] n g g / g g / n ex ( g ex ( g n n hen, usng E[z ], the probablty that nstance g s generated by the th Gaussan dstrbuton (E-step, to obtan a new maxmum lelhood hypothess (M-step: m E[ z ] g m E[ z ] ( (3 (4 (5 Equatons (4 and (5 defne the two steps n the mplementaton of the GP-EM recognton system for the devsng of the mxture model of the Gaussans, n practce. V. GP APPLICAION One of the ey features of the GP applcaton s that t uses tree structure representatons to solve problems. here are two maor tass processed n the GP loop: ftness measure and genetc operators to allow for the dentfcaton of the best ndvduals n a probablstc way; see a detaled descrpton of GP n [6]. A. he Functon Set and the ermnal Set able I lsts the functon set, a mathematcs elementary functon set, that attempts to ft the gven problem n ths study. Dvson and square root operatons are protected aganst dvsons by zero and negatve values, respectvely. ABLE I. DESIGNS OF HE FUNCION SE Sgns Functons ermnals +, - Addton, Subtracton *, / Multplcaton, Dvson S, C, Sne, Cosne, an functons P, R, E Square, Square root, Exponental A, N Absolute, Negatve value For the purpose of ths experment, the termnal set s assocated wth each of texture features descrbed n Secton II and wth an addtonal constant, c, nvolved. B. GP ree Representatons Fg. 7 shows a tree representng the feature functon of (F F3 x cos (F. On the tree, functons are nternal nodes, whch represent elementary functons such as addton and multplcaton, etc; and termnals are leaf nodes whch receve the prmtve features. Fgure 7. A GP tree representaton: (F - F3 x cos (F. C. Operator Selecton GP allocates every ndvdual some chance of beng selected to partcpate n the operatons of reproducton, crossover and mutaton [7]. he reproducton operator selects the best ndvduals on the bass of ther ftness by copyng t nto the new populaton. Crossover operator s performed by replacng a randomly chosen sub-tree of one parent program by a sub-tree from the other parent program. Fg. 8 llustrates the crossover operaton. he basc dea of the GP mutaton for termnal and functon operators s that any nonnvarant pont anywhere n the overall program s randomly chosen, wthout restrcton. 608

5 soluton n the current. he ftness results, normalzed n the ranged of (0,, for a sngle experment are shown n Fg. 9. Ftness.00 Fgure 8. GP sub-tree crossover. D. Ftness Functon Suppose there s a set of m nstances n the data G dvded nto the subsets G,,G. he wthn-class scatter matrx s [6]: S W gg then the between-class scatter matrx s: ( g ( g (6 SB n( ( where s replaced by the mean of the th Gaussan dstrbuton, s the total mean vector, G s the th subset of G and n s the number of samples n the th class. In ths study, we choose an optmal partton measurement as the ftness functon by usng the logarthm of J value whch equals to the rato of the between-class matrx and the wthnclass scatter matrx: J ln S W SB VI. EXPERIMENAL RESULS he experments use 500 cell mages, 50 healthy and 50 sc, randomly selected from CellsDB. We choose medum populaton sze and relatvely large number of generatons to help the algorthm adequately sample the search surface to fnd better solutons of the generated features. able II lsts the GP parameters used for the CellsDB experments. ABLE II. GP PARAMEERS Parameter Names Values Populaton Sze 3 No. of Generatons 400 Max. of ree Levels 5 No. of Operatons n the Functon Set Reproducton Probablty, P r 0.5 Crossover Probablty, P c 0.50 Mutaton Probablty, P m For classfcaton, we employ the classfer MDC by runnng the 0-fold cross-valdaton method, the statstcal practce to randomly dvde the data nto mutually exclusve parttons (folds for tranng and testng. A. Convergence Results In the GP-EM system, we stop the GP for 400 consecutve generatons when repeated applcatons of genetc operators yeld lttle effort on the ftness assocated wth the most elte (7 ( Max_ftness Mean_ftness Mn_ftnes s Ge ne rato ns Fgure 9. Convergence results. On an average, the tme for each of the 0 ndependent runs too about four mnutes on a Pentum 4 PC, wth CPU GHz, GB of RAM and an 80 mega-byte hard ds drve. Mcrosoft Vsual C was used for mplementng the GP-EM algorthm. B. Feature Generated Results After convergence, the resultng generated feature, the best, of a LISP-le expresson for the sngle experment s: F CellsDB = (+(*(+(+F0F0 F9F9*(S(*F6F(/F4F7 *(*(+F4F +FF C(/(*FF6 E(/F7F5 where F l (l =,, refers to prmtve texture features descrbed n Secton II, and F s a random constant whch equals ; the sgns of +, -, *, /, S, C, and E represent sne, cosne, tan and exponental functons, respectvely, whch are lsted n able I. A ratonal expresson of the resultng generated feature s: F CellsDB F0 sn( F6 F tan( F4 F7 (F (F F cos( F6 e (F7 F5 C. Classfcaton Results and Comparsons Classfcaton Results for 0 ndependent runs For the cell mage data, the statstcal measure for 0 ndependent runs on the combned tranng sets and test sets s llustrated n able III. We acheve the average tranng score of 9.40% accuracy. Wth only one generated feature, F CellDB, the average performance over all 0 runs shows that 90.0% of the test data can be classfed correctly. ABLE III. HE RESULS OF CLASSIFICAION ACCURACY (CA FOR 0 INDEPENDEN RUNS ON CELLSDB Statstcs ranng sets (CA est sets (CA Maxmum (% Mnmum (% Average (% Std (%

6 Comparsons of the GP-EM system and the SVM o evaluate the proposed GP-EM recognton system and demonstrate ts advantages, the experments on a lnear support vector machne (SVM classfer are carred out wth comparsons. he basc dea of the SVM s to construct a hyperplane as the decson plane, whch separates the postve (+ and negatve (- classes wth the largest margn, the sum of the dstances from the hyperplane to the closest data ponts of each of the two classes [7]. able IV llustrates the comparson results of classfcaton recognton rates between the GP-EM system wth one generated feature, F CellDB, and the classfer SVM wth prmtve texture features over 0 ndependent runs for the CellsDB mages. Sgnfcantly, for the test CA, the GP-EM system acheves the average performance of 90.0%, better than the performance acheved by the SVM approach. ABLE IV. COMPARISON RESULS OF CLASSIFICAION ACCURACY (CA BEWEEN HE GP-EM SYSEM AND HE SVM APPROACH FOR 0 INDEPENDEN RUNS ON CELLSDB Statstcs he GP-EM system usng one generated feature (CA he classfer SVM usng prmtve texture features (CA Maxmum (% Mnmum (% Average (% Std (% VII. CONCLUSION In ths study, we propose a new fully automatc GP-EM recognton system for classfcaton appled to the problem of the dagnoss of the dsease OPMD. Va the EM algorthm, the GP-EM generated data s modeled as a Gaussan mxture, n whch the learnng tas results n a smpler hypothess of the - means problem. On the other hand, the evolutonary GP process s drven by a ftness measure accountng for nter-class separablty and ntra-class homogenety. Overall, the proposed GP-EM system usng only one generated feature leads to hgher classfcaton accuraces wth a lower standard devaton, compared wth the performance of the SVM approach usng prmtve features. In addton to the man contrbuton, there are other unque and novel attrbutes and tools provded by ths study for bomedcal mage processng: ( he proposed GP-EM recognton system not only shows the capablty of enhancng the classfcaton performance but also succeeds n reducng the dmensonalty from the hyperspace of the prmtve features to a -dmensonal F CellsDB -space. ( he structures of the bomedcal mages affect the qualty of the prmtve feature extracton, a prerequste for the learnng tas of the feature generaton. he proposed HROI, an nnovatve hstogram-based technque, shows the applcablty to non-unform mages. (3 Wth the fltraton of the mage bacground n the margn and the applcaton of the thresholds, the resultant of mage grayscale hstogram s able to encompass all the ncluson nformaton contaned n the hstograms of three color channels, ncludng those whch cannot be captured by sngle ntenstes of D R, G and B hstograms. he current system has a lmtaton of the shortage of the termnators. hs can be solved by mplementng more prmtve features. herefore, the feature search space mprovement and the algorthm development wll be done n our future wor. ACKNOWLEDGMEN hs wor s supported by grants from the NSERC and Canada Research Char Foundaton. REFERENCES [] M. L. D. Wong, L.B. Jac, and A. K. Nand, Modfed self-organsng map for automated novelty detecton appled to vbraton sgnal montorng, Mechancal Syst. and Sgnal Process., vol.0, pp , 006. [] G. G. Yen, and P. Meesad, An effectve neuro-fuzzy paradgm for machnery condton health montorng, IEEE rans. Syst., Man, Cybern. B, Cybern., vol. 3, pp , 00. [3] K. Krawec, and B. Bhanu, Vsual learnng by coevolutonary feature synthess, IEEE rans. Syst., Man, Cybern. B, Cybern., vol. 35, pp , 005. [4] P. Chen,. oyota, and Z. He, Automated functon generaton of symptom parameters and applcaton to fault dagnoss of machnery under varable operatng condtons, IEEE rans. Syst., Man, Cybern. A, Syst., Humans, vol.3, pp , 00. [5] A. E. Eben, and J. E. Smth, Introducton to Evolutonary Computng, NY: Sprnger, 003. [6] J. R. Koza, Genetc Programmng II: Automatc Dscovery of Reusable Programs, Cambrdge, MA: MI Pr., 994. [7] D. P. Mun, N. R. Pal, and J. Das, Genetc programmng for smultaneous feature selecton and classfer desgn, IEEE rans. Syst., Man, Cybern. B, Cybern., vol.36, 06-7, 006. [8] V. Cesels, G. Wesnghe, A. Innes, and S. John, Analyss of the superorty of parameter optmzaton over genetc programmng for a dffcult obect detecton problem, IEEE Congress on Evol. Comput., pp.64-7, 006. [9]. M. Mtchell, Machne Learnng, NY: McGraw-Hll, 997. [0] Ln,. I., Lee, J. C., and Ho, H. J. On fast supervsed learnng for normal mxture models wth mssng nformaton. Pattern Recog., 39 (006, [] F. Chan, and J. Cho, Neghborhood explorng detector: An EM-based sgnal detector for multple antenna systems, IEEE rans. Sgnal Process, vol.55, pp , 007. [] R. C. Rafael, Dgtal Image Processng, MA: A. Wesley, 00. [3] G. Keller, and B. Warrac, Statstcs for Management and Economcs, NY: Duxbury Press, 000. [4] M.L. Raymer,.E. Doom, L.A. Kuhn, and W.F. Punch, Knowledge dscovery n medcal and bologcal datasets usng a hybrd bayes classfer/evolutonary algorthm raymer, IEEE rans Syst., Man, Cybern. B, Cybern., vol. 33, pp , 003. [5] M. James, Classfcaton Algorthms, London: Collns, 985. [6] C. M. Bshop, Pattern Recognton and Machne Learnng, NY: Sprnger, 006. [7] S.W. Lee, and A. Verr, Pattern Recognton wth Support Vector Machnes (LNCS 388, NY: Sprnger-Verlag,