INDEPENDENT COMPONENT ANALYSIS FOR NAÏVE BAYES CLASSIFICATION

Size: px
Start display at page:

Download "INDEPENDENT COMPONENT ANALYSIS FOR NAÏVE BAYES CLASSIFICATION"

Transcription

1 INDEPENDENT COMPONENT ANALYSIS FOR NAÏVE BAYES CLASSIFICATION FAN LIWEI (M.Sc., Dalan Unversty of Technology) A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF INDUSTRIAL & SYSTEMS ENGINEERING NATIONAL UNIVERSITY OF SINGAPORE 2010

2

3 Acknowledgement ACKNOWLEDGEMENT I would lke to express my utmost grattude to my supervsor Assocate Professor Poh Km Leng, for hs constructve comments and constant support throughout the whole course of my study. I greatly acknowledge Assocate Professor Leong Tze Yun for her nvaluable comments and suggestons on varous aspects of my thess research and wrtng. I would also lke to thank Assocate Professor Ng Szu Hu and Dr. Ng Ken Mng who served on my oral examnaton commttee and provded me many helpful comments on an earler verson of ths thess. I would lke to thank the Natonal Unversty of Sngapore for offerng a Research Scholarshp and the Department of Industral and Systems Engneerng for the use of ts facltes, wthout any of whch t would be mpossble for me to carry out my thess research. I am also very grateful to the members of SMAL Laboratory and the members of Bo-medcal Decson Engneerng group for ther frendshp and help n the past several years. Specal thanks go to my parents and my sster for ther constant encouragement and support durng n the past several. Fnally, I must say thanks to my husband, Zhou Peng, for hs encouragement and pushng throughout the entre perod of my study.

4 Table of Contents TABLE OF CONTENTS ACKNOWLEDGEMENT... SUMMARY... v LIST OF TABLES... v LIST OF FIGURES... v LIST OF NOTATIONS... x CHAPTER 1 INTRODUCTION BACKGROUND AND MOTIVATION OVERVIEW OF ICA-BASED FEATURE EXTRACTION METHODS RESEARCH SCOPE AND OBJECTIVES CONTRIBUTIONS OF THIS THESIS ORGANIZATION OF THE THESIS... 9 CHAPTER 2 LITERATURE REVIEW INTRODUCTION BASIC ICA MODEL DIRECT ICA FEATURE EXTRACTION METHOD Supervsed classfcaton Unsupervsed classfcaton Comparsons between varous feature extracton methods and classfers CLASS-CONDITIONAL ICA FEATURE EXTRACTION METHOD METHODS FOR RELAXING THE STRONG INDEPENDENCE ASSUMPTION CONCLUDING COMMENTS CHAPTER 3 COMPARING PCA, ICA AND CC-ICA FOR NAÏVE BAYES INTRODUCTION NAÏVE BAYES CLASSIFIER Basc model Dealng wth numercal features for naïve Bayes PCA, ICA AND CC-ICA FEATURE EXTRACTION METHODS Uncorrelatedness, ndependence and class-condtonal ndependence... 41

5 Table of Contents Prncpal component analyss Independent component analyss Class-condtonal ndependent component analyss EMPIRICAL COMPARISON RESULTS CONCLUSION CHAPTER 4 A SEQUENTIAL FEATURE EXTRACTION APPROACH FOR NAÏVE BAYES CLASSIFICATION OF MICROARRAY DATA INTRODUCTION MICROARRAY DATA ANALYSIS SEQUENTIAL FEATURE EXTRACTION APPROACH Stepwse regresson-based feature selecton CC-ICA based feature transformaton NAÏVE BAYES CLASSIFICATION OF MICROARRAY DATA EXPERIMENTAL RESULTS CONCLUSION CHAPTER 5 PARTITION-CONDITIONAL ICA FOR BAYES CLASSIFICATION OF MICROARRAY DATA INTRODUCTION FEATURE SELECTION BASED ON MUTUAL INFORMATION PC-ICA FOR NAÏVE BAYES CLASSIFIER General overvew of ICA General overvew of CC-ICA Partton-condtonal ICA METHODS FOR GROUPING CLASSES INTO PARTITIONS EXPERIMENTAL RESULTS CONCLUSION CHAPTER 6 ICA FOR MULTI-LABEL NAÏVE BAYES CLASSIFICATION INTRODUCTION MULTI-LABEL CLASSIFICATION PROBLEM MULTI-LABEL CLASSIFICATION METHODS Label-based transformaton Sample-based transformaton ICA-BASED MULTI-LABEL NAÏVE BAYES... 99

6 Table of Contents Basc mult-label naïve Bayes ICA-MLNB classfcaton scheme EMPIRICAL STUDY CONCLUSION CHAPTER 7 CONCLUSIONS AND FUTURE RESEARCH SUMMARY OF RESULTS POSSIBLE FUTURE RESEARCH BIBLIOGRAPHY v

7 Summary SUMMARY Independent component analyss (ICA) has receved ncreasng attenton as a feature extracton technque for pattern classfcaton. Some recent studes have shown that ICA and ts varant called class-condtonal ICA (CC-ICA) seem to be sutable for Bayesan classfers, especally for naïve Bayes classfer. Nevertheless, there are stll some lmtatons that may restrct the use of ICA/CC-ICA as a feature extracton method for naïve Bayes classfer n practce. Ths thess focuses on several methodologcal and applcaton ssues n applyng ICA to naïve Bayes classfcaton for solvng both sngle-label and mult-label problems. In ths study, we frst carry out a comparatve study of prncpal component analyss (PCA), ICA and CC-ICA for naïve Bayes classfer. It s found that CC-ICA s often advantageous over PCA and ICA n mprovng the performance of naïve Bayes classfer. However, CC-ICA often requres more tranng data to ensure that there are enough tranng data for each class. In the case where the sample sze s smaller than the number of features, e.g. n mcroarray data analyss, the drect applcaton of CC-ICA may become nfeasble. To address ths lmtaton, we propose a sequental feature extracton approach for naïve Bayes classfcaton of mcroarray data. Ths offers researchers or data analysts a novel method for classfyng datasets wth small sample sze but extremely large attrbute sze. Despte the usefulness of the sequental feature extracton approach, the number of samples for some classes may be lmted to just a few n mcroarray data analyss. The result s that CC-ICA cannot be used for these classes even f feature v

8 Summary selecton has been done on the data. Therefore, we extend CC-ICA and present the partton-condtonal ndependent component analyss (PC-ICA) for naïve Bayes classfcaton of mcroarray data. As a feature extracton method, PC-ICA essentally represents a compromse between ICA and CC-ICA. It s partcularly sutable for datasets whch come wth only few examples per class. The research work mentoned above only deals wth sngle-label naïve Bayes classfcaton. Snce mult-label classfcaton has receved much attenton n dfferent applcaton domans, we fnally nvestgate the usefulness of ICA for mult-label naïve Bayes (MLNB) classfcaton and present the ICA-MLNB scheme for solvng multlabel classfcaton problems. Ths research does not only demonstrate the usefulness of ICA n mprovng MLNB but also enrches the applcaton scope of the ICA feature extracton method. v

9 Lst of Tables LIST OF TABLES 3.1 UCI datasets wth ther specfc characterstcs 3.2 Experment results of the UCI datasets 4.1 Summary of fve mcroarray datasets 4.2 Classfcaton accuracy rates (%) of three classfcaton rules on fve datasets 5.1 Summary of two mcroarray datasets 6.1 A smple mult-label classfcaton problem 6.2 Sx bnary classfcaton problems obtaned from label-based transformaton 6.3 Sngle-label problem through elmnatng samples wth more than one label 6.4 Sngle-label problem through selectng one label for mult-label samples 6.5 Sngle-label problem through creatng new classes for mult-label samples v

10 Lst of Fgures LIST OF FIGURES 1.1 Structure of the thess 2.1 Flow chart of the drect ICA feature extracton method for classfcaton 2.2 Flow chart of the CC-ICA feature extracton method for classfcaton 3.1 Structure of naïve Bayes classfer 3.2 Graphcal llustraton of PCA and ICA for naïve Bayes classfer 3.3 Relatonshp between average accuracy rate and the number of features 4.1 Boxplots of the holdout classfcaton accuracy rates for Leukema-ALLAML 4.2 Boxplots of the holdout classfcaton accuracy rates for Leukema-MLL 4.3 Boxplots of the holdout classfcaton accuracy rates for Colon Tumor 4.4 Boxplots of the holdout classfcaton accuracy rates for Lung Cancer II 5.1 Graphcal llustraton of the dfference among PC-ICA, CC-ICA and ICA 5.2 Boxplots of classfcaton accuracy rates for ICA and PC-ICA based on Leukema-MLL dataset when the number of genes selected (N) s changeable 5.3 Boxplots of classfcaton accuracy rates for ICA and PC-ICA based on Lung Cancer I dataset when the number of genes selected (N) s changeable 6.1 The average Hammng loss for MLNB and ICA-MLNB classfcaton of Yeast data when the number of features vares from 11 to Comparatve boxplots of Hammng loss for MLNB and ICA-MLNB classfcaton of Yeast data wth varous feature szes 6.3 The average Hammng loss for MLNB and ICA-MLNB classfcaton of natural scene data when the number of features vares from 11 to 20 v

11 Lst of Fgures 6.4 Comparatve boxplots of Hammng loss for MLNB and ICA-MLNB classfcaton of natural scene data wth varous feature szes x

12 Lst of Notatons LIST OF NOTATIONS ANN BN BSS CC-ICA ECG EEG fmri ICA ICAMM KICA KNN KPCA LDA ML-KNN MLNB MRMR NB PCA PC-ICA TCA TICA SVM Artfcal neural networks Bayesan network Blnd source separaton Class-condtonal ndependent component analyss Electrocardogram Electroencephalography Functonal magnetc resonance magng Independent component analyss ICA mxture model Kernel ndependent component analyss K-nearest neghborhood Kernel prncpal component analyss Lnear dscrmnant analyss Mult-label K-nearest neghborhood Mult-label naïve Bayes Mnmum redundancy maxmum relevance Naïve Bayes Prncpal component analyss Partton-condtonal ndependent component analyss Tree-dependent component analyss Topographc ndependent component analyss Support vector machnes x

13 Chapter 1 Introducton CHAPTER 1 INTRODUCTION Independent component analyss (ICA) s a useful feature extracton technque n pattern classfcaton. Ths thess contrbutes to the development of varous ICAbased feature extracton methods or schemes for the naïve Bayes model to classfy dfferent types of datasets. In ths ntroductory chapter, we frst provde the background and the motvaton for ths study, whch s followed by a bref overvew of ICA-based feature extracton methods. After that we outlne the scope and objectve of ths study. Fnally, we summarze the content and the structure. 1.1 Background and motvaton Pattern classfcaton, whch ams to classfy data based on a pror knowledge or statstcal nformaton extracted from the patterns, s a fundamental problem n artfcal ntellgence. Nowadays, pattern classfcaton s a very actve area of research that draws the attenton of researchers from dfferent dscplnes ncludng engneerng, computer scence, statstcs and even socal scences. Snce better classfcaton results can provde useful nformaton for decson makng, numerous studes have been devoted to mprove the performance of pattern classfcaton from dfferent aspects. Intutvely, better classfcaton results may be obtaned from a set of representatve features constructed from the knowledge of doman experts. When such expert knowledge s not avalable, general feature extracton technques seem to be very useful. They helps to remove redundant or rrelevant nformaton, dscover the 1

14 Chapter 1 Introducton underlyng structure, facltate the subsequent analyss, and mprove classfcaton performance. In the past several decades, machne learnng researchers have developed a number of feature extracton methods, such as, prncpal component analyss (PCA), multfactor dmensonalty reducton, partal least squares regresson, and ndependent component analyss (ICA). Of the varous feature extracton methods, ndependent component analyss (ICA) s recently found to be very useful and effectve n helpng to extract representatve features n pattern classfcaton. ICA s a relatvely new statstcal and computatonal technque for revealng the hdden factors that underle a set of random varables. Although ICA was ntally developed to solve the blnd source separaton (BSS) problem, prevous studes have shown that ICA can serve as an effectve feature extracton method for mprovng the classfcaton performance n both supervsed classfcaton (Zhang et al., 1999; Kwak et al., 2001; Cao and Chong, 2002; Herrero et al., 2005; Chuang and Shh, 2006; Wdodo et al., 2007; Yu and Chou, 2008) and unsupervsed classfcaton (Lee and Batzoglou, 2003; Kapoor et al., 2005; Kwak, 2008). It has also been found that ICA may help to mprove the performance of varous classfers, such as support vector machnes, artfcal neural networks, decsons trees, hdden Markov models, and the naïve Bayes classfer (Sanchez-Poblador et al., 2004; L et al., 2005; Melssant et al., 2005; Yang et al., 2005). NB, also called smple Bayesan classfer, s a smple Bayesan network that assumes all features are condtonally ndependent gven the class varable. Snce no structure learnng s requred, t s very easy to construct and mplement NB n practce. Despte ts smplcty, the naïve Bayes has been found to be compettve wth 2

15 Chapter 1 Introducton other more advanced and sophstcated classfers (Fredman et al., 1997). It s therefore not surprsng that naïve Bayes classfer has ganed great popularty n solvng varous classfcaton problems. Nevertheless, the class-condtonal ndependence assumpton between features taken by naïve Bayes classfer s often volated n some real-world applcatons. Snce ICA ams to transform the orgnal features nto new features that are statstcally ndependent of each other as possble, the ICA transformaton s lkely to ft well the NB model and ts ndependent assumpton (Bressan and Vtra, 2002). Several earler studes have been devoted to nvestgate the applcablty of ICA as a feature extracton tool for the naïve Bayes classfer. It was found that ICA and ts varants, such as class-condtonal ICA (CC-ICA), are often capable of mprovng the classfcaton performance of the NB model. Nevertheless, some lmtatons of CC-ICA may restrct the use of CC-ICA as a feature extracton tool to mprove the performance of NB classfer n mcroarray data analyss. In ths thess, we propose several ICA-based feature extracton methods for addressng the lmtatons n applyng ICA to naïve Bayes classfcaton of mcroarray data. In addton, snce most prevous studes manly focused on sngle-label classfcaton problems, the queston of how to adapt the ICA feature extracton method for multlabel classfcaton problems remans to be nvestgated. Therefore, we also nvestgate the use of ICA as a feature extracton method for mult-label naïve Bayes classfcaton. 3

16 Chapter 1 Introducton 1.2 Overvew of ICA-based feature extracton methods Wth the development of modern scence and technology, large amounts of nformaton can be obtaned and recorded for a varety of problems. However, the exstence of too much nformaton may often reduce the effectveness of data analyss. In pattern classfcaton, t mples that the performance of a classfer adopted may worsen when too many features are used to tran the classfer. Ths s due to the fact that some features are redundant for constructng the classfer. Therefore, many feature selecton or feature extracton methods have been proposed to mnmze the cons of the rrelevant or redundant features. Feature selecton methods am to select the most relevant features, whle feature extracton methods attempt to transform features nto a new (and may be reduced) set of more representatve features. Several ICA-based methods have been proposed and used for feature extracton n pattern classfcaton. The frst one may be referred to as the drect ICA feature extracton method, n whch ICA s drectly used to transform orgnal features nto a new set of features for classfcaton use. Snce ICA assumes that the varables after the transformaton are ndependent of each other, the features obtaned from the drect ICA feature extracton method are as ndependent wth each other as possble. As a result, the new features obtaned seem to be more consstent wth the assumpton of the naïve Bayes classfer compared to the orgnal features. Therefore, the classfcaton performance of the naïve Bayes classfer could be mproved usng the ICA features (Zhang et al., 1999). Nevertheless, the strong ndependence assumpton used n the ICA computaton may not be approprate for some real-world datasets. To overcome ths 4

17 Chapter 1 Introducton lmtaton, Hyvarnen et al. (2001a) proposed topographc ndependent component analyss (TICA) by relaxng the strong ndependence assumpton. TICA uses contrast functons ncludng the hgher-order correlatons between the components to acheve the relaxaton of the strong ndependence assumpton. However, n practce the emprcal contrast functons are dffcult to construct. Though the strong ndependence assumpton s napproprate for some realworld datasets, t may offer the advantages for some specfc classfers such as the NB model. Snce the strong ndependence assumpton of ICA makes the new features as ndependent as possble, the features obtaned from ICA may be more consstent wth the underlyng assumpton of nave Bayes classfer. Furthermore, Bressan and Vtra (2002) proposed the CC-ICA feature extracton method that apples ICA wthn each class, whch can help to extract the representatve features from the orgnal features wthn each class. Ther emprcal studes showed that the CC-ICA feature extracton method may be more sutable than the drect ICA feature extracton method for the NB classfer. A lmtaton of the CC-ICA feature extracton method s that t requres more tranng data than the drect ICA feature extracton method n mplementaton. Usually, the number of samples should not be less than the number of features wthn each class for the CC-ICA feature extracton method, whle for the drect ICA feature extracton method the number of samples for all the classes s requred to be not less than the number of features. However, there may not be enough tranng data for some real-world applcatons such as mcroarray data analyss due to the very hgh data collecton cost. Therefore, t s meanngful to extend CC-ICA and develop new ICA- 5

18 Chapter 1 Introducton based feature extracton method so that t s applcable to the case of small datasets. Snce ICA-based feature extracton methods are manly used for addressng snglelabel classfcaton problems, t would also be very useful to nvestgate the usefulness of ICA as a feature extracton method n solvng mult-label classfcaton problems. 1.3 Research scope and objectves The man objectve of ths thess s to address several methodologcal and applcaton ssues n applyng ICA for feature extracton, whch could be helpful to those who expect to use t to mprove the performance of the naïve Bayes classfer n solvng both sngle-label and mult-label classfcaton problems. In many cases ICA can extract more useful nformaton than prncpal component analyss (PCA) for the succeedng classfers snce ICA can make use of hgh-order statstcs nformaton. However, a feature extracton method cannot always perform better than others for all applcaton domans and all classfers. It s therefore meanngful to compare varous feature extracton methods wth respect to the classfcaton performance of the succeedng classfer. Our comparatve study found that CC-ICA s often advantageous over PCA and ICA n mprovng the performance of naïve Bayes classfer. However, the CC- ICA requres more tranng data to ensure that there are enough tranng data for each class. In the case where the sample sze s much less than the number of features, e.g. n mcroarray data analyss, the drect mplementaton of CC-ICA may become nfeasble. Therefore, we propose a sequental feature extracton approach for naïve Bayes classfcaton of mcroarray data. In the sequental feature extracton approach, stepwse regresson s frst appled for feature selecton and CC-ICA s then used for 6

19 Chapter 1 Introducton feature transformaton. It s expected that the proposed approach could be adopted by researchers to solve such classfcaton problems wth small sample sze but extremely large attrbute sze n dfferent domans ncludng mcroarray data analyss. For some mcroarray datasets, there may be only few samples for some classes so that CC-ICA cannot be appled after feature selecton. Therefore, we extend CC- ICA and propose partton-condtonal ndependent component analyss (PC-ICA) for naïve Bayes classfcaton of mcroarray data. In ths research, we appled mnmum redundancy maxmum relevance (MRMR) prncple based on mutual nformaton to select nformatve features and appled PC-ICA for feature transformaton for each partton. Compared to ICA and CC-ICA, PC-ICA represents an n-between concept. If each class has enough samples to do ICA, there s no need to combne the samples nto parttons and PC-ICA wll become CC-ICA. If all the classes are grouped nto one partton, CC-ICA wll collapse to ICA. PC-ICA could make full use of samples n the parttons ncludng several classes to mprove the performance of naïve Bayes classfer. It s expected that PC-ICA could help to solve the mult-class problems even f the number of tranng examples s small. For mult-label classfcaton problems, feature extracton s also essental for mprovng classfcaton performance. Based on the experence of ICA for snglelabel problems, ICA transformaton could make the features more approprate for mult-label naïve Bayes classfcaton. However, none of the prevous studes dealt wth the use of ICA as a feature method for mult-label naïve Bayes (MLNB) classfer. Therefore, we propose the ICA-MLNB scheme for solvng mult-label classfcaton problems. It s expected that ICA-MLNB could not only expand the 7

20 Chapter 1 Introducton applcaton of ICA n pattern classfcaton but also be adopted by researchers who are nterested n applyng naïve Bayes to solve mult-label problems. 1.4 Contrbutons of ths thess The man contrbutons of the work presented n ths thess can be summarzed from the pont of vew of methodologcal and applcaton as follows. In terms of methodology, we have proposed a new sequental feature extracton method for naïve Bayes classfcaton of mcroarray data. Ths method reduces the number of features by the stepwse regresson and transforms the features to a small set of ndependent features. Despte the smplcty of the proposed method, our expermental results showed that t can mprove the performance of the classfer sgnfcantly. In addton, we proposed PC-ICA for solvng mult-class problems. Instead of applyng ICA wthn each class n CC-ICA, PC-ICA uses ICA to do feature extracton wthn each partton whch may consst of several small-sze classes. Expermental results on several mcroarray datasets have shown that PC-ICA usually leads to better performance than ICA for naïve Bayes classfcaton of mcroarray data. In terms of applcaton, we frst compared the ICA, PCA and CC-ICA feature extracton methods for the NB classfer. It s found that all the three methods keep mprovng the performance of the naïve Bayes classfer wth the ncrease of the number of attrbutes. Although CC-ICA has been found to be superor to PCA and ICA n most cases, t may not be sutable for the case where the sample sze of each class s not suffcently large. Ths s the motvaton of the sequental feature extracton method and PC-ICA presented n ths thess. Snce none of the prevous 8

21 Chapter 1 Introducton studes dealt wth the use of ICA for mult-label naïve Bayes classfcaton, we nvestgate the usefulness of ICA as a feature extracton method for mult-label naïve Bayes classfer and propose the ICA-MLNB scheme for solvng mult-label classfcaton problems. Our expermental results demonstrate the effectveness of the scheme n mprovng the performance of mult-label naïve Bayes classfcaton. 1.5 Organzaton of the thess Ths thess focuses on the study of ICA-based feature extracton methods for the naïve Bayes classfer n solvng sngle and mult -label classfcaton problems. It conssts of seven chapters. Fgure 1.1 shows the man content of each chapter and the relatonshps among dfferent chapters. Chapter 2 revews the use of ICA as a feature extracton tool n pattern classfcaton. Dfferent ICA feature extracton methods and ther applcatons are summarzed and examned. Compared wth other feature extracton methods, the superorty of ICA based feature extracton methods les n ther ablty of utlzng hgh-order statstcs and ther sutablty for the non-gaussan case. Our lterature revew also found that ICA s partcularly sutable for the naïve Bayes classfer but there are stll several lmtatons worth further nvestgatng. In Chapter 3, we frst ntroduce the naïve Bayes model and three feature extracton methods, namely PCA, ICA and CC-ICA. Then we emprcally compare them for the naïve Bayes classfer wth regards to the classfcaton performance. Our expermental results have shown that all three methods can mprove the performance of the naïve Bayes classfer. In general, CC-ICA outperforms PCA and ICA n terms 9

22 Chapter 1 Introducton of the classfcaton accuracy. However, CC-ICA requres more tranng data to ensure that there are enough tranng data for each class. Chapter 4 presents a sequental feature extracton approach for naïve Bayes classfcaton of mcroarray data. The proposed feature extracton approach starts from gene selecton by stepwse regresson, whch s a smple but effectve dmenson reducton technque followng the MRMR prncple. The data on the genes selected are then transformed by CC-ICA, whch makes the new features after transformaton become as ndependent as possble. In Chapter 5, we extend CC-ICA and propose PC- ICA for naïve Bayes classfcaton of mcroarray data. CC-ICA apples ICA for each class, whle PC-ICA uses ICA to do feature extracton wthn each partton consstng of several small-sze classes. As such, t represents a compromse between ICA and CC-ICA. The effectveness of PC-ICA has been demonstrated by our expermental studes on several mcroarray datasets. Whle Chapters 4 and 5 deal wth sngle-label classfcaton problems, Chapter 6 s manly concerned wth the use of ICA n mult-label naïve Bayes classfcaton problems. In Chapter 6, we apply ICA to mult-label naïve Bayes and propose the ICA-MLNB scheme for mult-label classfcaton. The results obtaned from our expermental studes have shown the effectveness of the ICA-MLNB scheme and also demonstrate the usefulness of ICA as a feature extracton method n solvng mult-label classfcaton problems. Chapter 7 gves the concluson of ths thess as well as some potental future research topcs. 10

23 Chapter 1 Introducton 1. Introducton 2. Lterature revew 3. Comparng PCA, ICA and CC- ICA for naïve Bayes classfer 4. A sequental feature extracton approach for NB classfcaton of mcroarray data 5. PC-ICA for NB classfcaton of mcroarray data 6. ICA for mult-label naïve Bayes classfcaton 7. Conclusons and future research Fg. 1.1 Structure of the thess 11

24 Chapter 2 Lterature Revew CHAPTER 2 LITERATURE REVIEW 2.1 Introducton Pattern classfcaton problems are usually very complex and cannot be well solved by only one procedure (Jan et al., 2000). For the purpose of reducng computatonal costs and mprovng classfcaton performance, certan preprocessng procedure s often adopted to select the most nformatve features or to approprately transform the orgnal data nto a new set of data. The preprocessng procedure s often termed as feature selecton or feature extracton. Prevous researchers have proposed a number of feature extracton methods for mprovng the performance of classfcaton. Among the varous feature extracton methods, ICA has receved ncreasng attenton due to ts usefulness n helpng extract representatve features for classfcaton. As mentoned n Chapter 1, ICA s a relatvely new statstcal technque for fndng hdden factors or components to gve a novel representaton of multvarate data. It was orgnally proposed by Jutten and Herault (1991) for solvng the blnd source separaton (BSS) problems. In ths applcaton, ICA can help to fnd the underlyng ndependent components, whch may provde valuable nformaton for data analyss. As a feature extracton technque, ICA may be vewed as a generalzaton of PCA. PCA tres to fnd uncorrelated varables to represent the orgnal multvarate data, whereas ICA attempts to obtan statstcally ndependent varables to represent the orgnal multvarate data, especally n the case of non- Gaussan dstrbuton. 12

25 Chapter 2 Lterature Revew Theoretcally, ICA s a computatonal algorthm to search for a lnear transformaton that mnmzes the statstcal dependence between the components of a multvarate varable. Many mportant theoretcal landmarks n ICA, e.g. Common (1994), Bell and Sejnowsk (1995), Amar et al. (1996), Cardoso and Laheld (1996), and Hyvarnen and Oja (1997), were establshed n the 1990s. Snce then, ICA has ganed more and more popularty n a wde spectrum of areas, e.g. bomedcal sgnal processng, mage recognton, fault dagnoss, data mnng and fnancal tme seres analyss. In most of the prevous studes, ICA was taken as an effectve preprocessng procedure for further data analyss. It s therefore not surprsng that ICA has also receved much attenton n pattern classfcaton as a feature extracton method. Ths chapter provdes a revew of the most commonly used ICA-based feature extracton methods for pattern classfcaton. The basc ICA model s frst brefly ntroduced n Secton 2.2. Secton 2.3 presents the drect ICA feature extracton method wth more emphases on supervsed classfcaton, whch s followed by several other ICA-based feature extracton methods presented n Sectons 2.4 and 2.5. Secton 2.6 summarzes the concludng comments. 2.2 Basc ICA model ICA was orgnally developed to deal wth BSS problems whch are closely related to the classcal cocktal-party problem. Assume that there are three mcrophones used to record tme sgnals n dfferent locatons n one room. The ampltudes of the three sgnals are respectvely denoted as ( t) x ( t) 1, and x 3( t), x 2 where t s the tme ndex. Further assume that each sgnal s a weghted sum of three 13

26 Chapter 2 Lterature Revew dfferent source sound sgnals whch are respectvely denoted as ( t) s ( t) 1, and s 3( t ). s 2 The relatonshp between the three source sound sgnals and the three mcrophones sound sgnals may be descrbed as x x x 1 ( t) = a11s1 ( t) + a12 s2( t) + a13s3 ( t) ( t) = a21s1( t) + a22s2( t) + a23s3( t) ( t) = a s ( t) + a s ( t) + a s ( t) (2.1) where a j (, j = 1,2, 3 ) represent the unknown weghts that reflect the dstances of the mcrophones from the sound sources. The problem s to separate the three ndependent sound sources only based on the three mcrophones records. The smple BSS problem wth three sources can be generalzed to the case of n sources. Suppose that there are n observed random varables x, x, 2, x 1 L n, whch are modeled as the lnear combnatons of n random source varables s, s, 2, s 1 L n. Mathematcally, t can be expressed as x = a s + a s + L+ a n n, 1,2, L, n s = (2.2) where a j (, j = 1,2, L, n ) represents the mxng coeffcents, and s ( = 1,2, L, n ) are assumed to be mutually statstcally ndependents. Equaton (2.2) can also be represented n the vector-matrx form as follows: x = As (2.3) 14

27 Chapter 2 Lterature Revew where x s the random column vector wth elements x, x, 2, x 1 L n, s s the random column vector wth elements s, s, 2, s 1 L n, and A s the mxng matrx wth elements a j. In ICA, Eq. (2.3) s often re-wrtten as y = Wx (2.4) where 1 W = A s the demxng matrx and T = [ y1, y2, L, y n denotes the y ] ndependent components. The task s to estmate the demxng matrx and ndependent components only based on the mxed observatons, whch can be done by varous ICA algorthms bult upon a certan prncple. There are varous prncples to solve the ICA model, such as maxmum lkelhood, nongaussanty maxmzaton, and mutual nformaton mnmzaton. In computaton, each of the prncples generates a specfc objectve functon and ts optmzaton wll enable the ICA estmaton. Varous optmzaton algorthms may be appled to solve the optmzaton problems and obtan the ndependent components. 2.3 Drect ICA feature extracton method In pattern classfcaton, prncpal component analyss (PCA) and lnear dscrmnant analyss (LDA) are two popular feature extracton methods. Lke PCA and LDA, ICA can also be drectly used for feature extracton. Gven the varables x,, x, 1 2 L x n, the underlyng ndependent varables s,, 1 s2, sm ( m n) L and the demxng matrx W can be obtaned by dfferent ICA algorthms. Then the 15

28 Chapter 2 Lterature Revew ndependent varables s, s, 2 L, sm ( m ) 1 n obtaned can be drectly used to tran the classfer. Meanwhle, the demxng matrx W can be drectly appled to transform the test data for classfcaton. Snce ths method nvolves the drect applcaton of ICA, we here refer to t as the drect ICA feature extracton method. Fgure 2.1 shows the flow chart of the drect ICA feature extracton method for pattern classfcaton. As shown n Fg. 2.1, to construct an approprate classfer we usually need to frst splt the dataset avalable nto tranng and test datasets. The datasets are preprocessed by certan feature selecton procedures. For the tranng dataset after feature transformaton, ICA s used to do the feature extracton and obtan the demxng matrx W, whch can then be used to do feature transformaton for the test data after feature selecton. Meanwhle, the tranng and test datasets after ICA-based feature extracton can be used to construct an approprate classfer by learnng ts parameters and examnng ts classfcaton performance. In pattern classfcaton, the drect ICA feature extracton method has been wdely adopted n both supervsed classfcaton and unsupervsed classfcaton. In the followng, we shall frst gve a revew of some relevant studes dvded nto supervsed and unsupervsed classfcatons, where there are more studes n the supervsed classfcaton group. Then we brefly dscuss the ssue of classfer selecton as the drect ICA feature extracton method may be ntegrated wth varous classfers. 16

29 Chapter 2 Lterature Revew Dataset Tranng Preprocessng Test Feature extracton by ICA Feature transformaton by W Learnng and testng classfer Fg Flow chart of the drect ICA feature extracton method for classfcaton Supervsed classfcaton Supervsed classfcaton refers to the type of classfcaton n whch the label for each sample s known n advance. In the tranng process, a classfer s constructed from the features and labels of sample data, n whch the drect ICA feature extracton method plays a major role. In the test process, the label for a new gven sample wll be predcted by the classfer obtaned. Applcaton areas of the supervsed classfcaton based on the drect ICA feature extracton method nclude face recognton, sgnal analyss, mage analyss, text categorzaton, etc. (1) Face recognton Face recognton s a major applcaton area n whch the drect ICA feature extracton method has ganed n popularty. In ths applcaton, the earlest study could 17

30 Chapter 2 Lterature Revew be attrbuted to Bartlett and Sejnowsk (1997) who proposed an ICA representaton of face mages and compared t wth the PCA representaton of the same face mages. Ther study showed that ICA provdes a better representaton than PCA because n the latter only the second-order statstcs are decorrelated. Guan and Szu (1999) compared the drect ICA and PCA feature extracton methods for the nearest neghbor classfer for face recognton. Ther study found that ICA outperforms PCA when one tranng mage per person s used. It ndcates that the drect ICA feature extracton method may be a better alternatve when only few tranng samples are avalable. Also usng the nearest neghbor classfer, Donato et al. (1999) showed that ICA representaton performed as well as the Gabor representaton and better than PCA representaton, whch are popular representaton methods n classfyng facal actons. Km et al. (2004) proposed an ICA based face recognton scheme, whch was found to be robust to the llumnaton and pose varatons. An nterestng fndng by Km et al. (2004) s that n the resdual face space ICA provdes a more effcent encodng n terms of redundancy reducton than PCA. In face recognton, the algorthms based only on the vsual spectrum are not robust enough to be used n uncontrolled envronments. Motvated by ths queston, Chen et al. (2007) proposed to fuse nformaton from vsual spectrum and nfrared magery to acheve better results. Ther scheme also employs ICA as a feature extracton method for the support vector machne (SVM) classfer. Ther expermental results show that the scheme mproves recognton performance substantally. 18

31 Chapter 2 Lterature Revew Based on an applcaton of the drect ICA feature extracton method to Yale Face Databases and AT&T Face Databases, Kwak et al. (2002) found that ICA transformaton can make new features as ndependent wth each other as possble. Smlar to earler studes, the study by Kwak et al. (2002) also showed that ICA outperforms PCA and LDA as feature extracton method for face recognton. Subsequently, Kwak and Cho (2003) further extended the work by Kwak et al. (2002) by developng a stablty condton for the earler study. The two earler studes mentoned above focused on the two-class face recognton problems. More recently, Kwak (2008) extended the use of the drect ICA feature extracton method to the case of mult-class face recognton usng the nearest neghborhood classfer. The expermental results for several face databases demonstrated the usefulness of the drect ICA feature extracton method n solvng mult-class face recognton problems. (2) Sgnal analyss Sgnal analyss s also a major applcaton area where the drect ICA feature extracton method has been wdely used. Applcatons of the drect ICA feature extracton method to sgnal analyss nclude data analyss of functonal magnetc resonance magng (fmri), electroencephalography (EEG), and electrocardogram (ECG). Prevous studes have shown that the drect ICA feature extracton method can help to extract task-related components and reduce the nose of sgnals effectvely (Stone, 2004). Laubach et al. (1999) compared PCA and ICA for quantfyng neuronal ensemble nteractons, and found that ICA performs better than PCA n terms of the classfcaton performance. The study by Hoya et al. (2003) attempted to classfy the 19

32 Chapter 2 Lterature Revew EEG sgnals of letter magery tasks by combnng ICA and probablstc neural network. It was found that the ncluson of ICA n the classfer led to an mprovement of classfcaton accuracy rate by around 17-30%. Melssant et al. (2005) studed the EEG measurements for detectng Alzhemer s dsease, and found that the classfcaton results for the group wth severe Alzhemer s dsease usng ICA are comparable to the best classfcaton results n the lterature. In addton, the drect ICA feature extracton method has also been appled to the dscrmnaton of mental tasks for EEG-based bran computer nterface systems. It was found that ICA ntegrated wth the SVM classfer may produce good classfcaton performance, whch could be attrbuted to the fact that the temporal nformaton from a wndow of data s effectvely extracted by ICA. The drect ICA feature extracton method has also been appled to heartbeat classfcaton. Herrero et al. (2005) used ICA and machnng pursuts to do feature extracton for heartbeat classfcaton. Ther concluson s that ICA could mprove the system s ablty of dscrmnatng varous beat sgnals, whch s partcularly useful n clncal use. More recently, Yu and Chou (2008) proposed to ntegrate ICA and neural networks for ECG beat classfcaton. Ther expermental results showed that the scheme of ntegratng ICA and neural networks s of great potental n the computeraded dagnoss of heart dseases based on ECG sgnals. (3) Image analyss Image analyss usually requres effectve feature extracton through varous feature extracton methods such as ICA. Hoyer and Hyvarnen (2000) nvestgated the use of ICA n decomposng natural color and stereo mages. They found that the 20

33 Chapter 2 Lterature Revew features extracted by ICA could be drectly used for pattern recognton of color or stereo data. Karvonen and Smla (2001) also found that the ICA representaton of data s useful to mprove the classfcaton performance n sea ce Synthetc aperture radar (SAR) mage analyss. Fortuna et al. (2002) showed that ICA performs better than PCA as a feature extracton method n object recognton under varyng llumnaton. Leo and Dstante (2003) proposed a comparatve study of wavelet and ICA for automatc ball recognton usng the back propagaton neural network. Borgne et al. (2004) appled ICA to extract features from natural mages, and use the new features for a K-nearest neghborhood (KNN) classfcaton paradgm. Ther expermental results demonstrated the effectveness of the drect ICA feature extracton method n classfyng natural mages. Based on a large set of consumer photographs, the Fourertransformed mages, Boutell and Luo (2005) appled the drect ICA feature extracton method to derve ther sparse representatons for classfcaton. The emprcal analyss results showed the superorty of ICA over PCA as a feature extracton technque. In addton to the tradtonal ICA model, other types of ICA models have also been drectly used for feature extracton n mage analyss. For nstance, Cheng et al. (2004) showed the effectveness of kernel ndependent component analyss (KICA) for texture feature extracton. The study by Luo and Boutell (2005) used overcomplete ICA for the heurstc and support vector machne classfcaton of Fourer-transformed mages and demonstrated ts effectveness as a feature extracton method. 21

34 (4) UCI machne learnng repostory Chapter 2 Lterature Revew Some researchers have also appled the drect ICA feature extracton method to analyze the data from the UCI machne learnng repostory. Kwak et al. (2001) added class nformaton to the Wsconsn Breast Cancer Dagnoss and Chess End- Game datasets, whch plays an mportant role n extractng useful features for classfcaton. Expermental results showed that the features extracted by ICA are more useful than the orgnal features n classfcaton. Usng the nne contnuous datasets from the UCI machne learnng repostory, Prasad et al. (2004) evaluated the ntegraton of the drect ICA feature extracton method wth naïve Bayes, nstance based learnng and decson trees. Ther expermental results showed that naïve Bayes classfer outperforms other classfers for fve of the nne datasets. For the remanng four datasets, naïve Bayes classfer s comparable wth other classfers. It could be attrbuted to the fact that the naïve Bayes classfer s known to be optmal when attrbutes are ndependent wth each other gven the class. Based on another nne datasets from the UCI machne learnng repostory, Sanchez-Poblador et al. (2004) examned the applcablty of ICA as a feature extracton technque for decson trees and multlayer perceptrons. It was found that for some datasets the drect ICA feature extracton would beneft the classfcaton, whle for others the beneft was mnor. The concluson was that the use of ICA as a preprocessng technque may mprove the classfcaton performance when the feature space has a certan structure. 22

35 (5) Mcroarray data analyss Chapter 2 Lterature Revew Accurate classfcaton of mcroarray data s very mportant for successful dagnoss and treatment of dseases such as cancer. Recently, some researchers have also appled the drect ICA feature extracton method to help mprove the classfcaton performance of mcroarray data analyss. For nstance, Zheng et al. (2006) combned ICA wth the sequental floatng forward technque to do feature extracton for classfyng the DNA mcroarray data. Ther study showed the effectveness of the drect ICA feature extracton method n classfyng mcroarray data. More recently, Lu et al. (2009a,b) developed a genetc algorthm/ica based ensemble learnng system to help mprove the performance of mcroarray data classfcaton. Ther expermental results further demonstrated the usefulness of the drect ICA feature extracton method n mcroarray data analyss. (6) Mscellaneous In addton to the applcaton areas descrbed above, the drect ICA feature extracton method has also been used to help solve the classfcaton problems n other applcaton areas. Here we shall only gve two examples on the use of ICA n text categorzaton and fault dagnoss. Text categorzaton s based on statstcal representatons of documents that usually consst of a huge dmenson. It s necessary to fnd an effectve dmenson reducton for a better representaton of word hstograms. In ths applcaton context, Kolenda et al. (2002) appled the drect ICA feature extracton method and found that the ICA representaton s better than PCA representaton n explanng the group 23

36 Chapter 2 Lterature Revew structure. The study by Wdodo et al. (2007) ntegrated ICA and SVM for ntellgent faults dagnoss of nducton motors, whch showed the advantage of ICA over PCA as a feature extracton technque Unsupervsed classfcaton In contrast to supervsed classfcaton, unsupervsed classfcaton does not requre user to nput sample classes n performng classfcaton. It uses certan technques to determne whch features are related wth each other and whch samples can be grouped nto a class. In classfcaton process, the user can specfy the desred number of output classes. The applcablty of the drect ICA feature extracton method n unsupervsed classfcaton has also been wdely explored. Lee et al. (2000) proposed the ICA mxture model (ICAMM) for unsupervsed classfcaton of non- Gaussan classes. Its classfcaton performance was found to be comparable to or advantageous over those obtaned by AutoClass that uses a Gaussan mxture model. The ICAMM has been used for unsupervsed mage classfcaton, segmentaton, and enhancement (Lee and Lewck, 2002). Several other researchers, ncludng Hashmoto (2002) and Shah et al. (2002, 2003, 2004), also appled the ICAMM to solve other mage classfcaton problems usng dfferent algorthms. These earler studes showed that n mage analyss the unsupervsed classfcaton based on ICAMM could produce hgher accuracy than the K-means algorthm, whch llustrates the benefts of employng hgher order statstcs n classfcaton. In Bae et al. (2000), the ICAMM has also been appled for blnd sgnal separaton n teleconferencng. The authors found that ICAMM could learn well the 24

37 Chapter 2 Lterature Revew unmxng matrces gven the number of classes. However, f no optmal number of classes were gven, ICAMM would lkely result n a local optmum n most cases. Therefore, Olvera and Romero (2004) proposed the Enhanced ICAMM to modfy the learnng algorthm based on a gradent optmzaton technque. Ths new model mproves the performance of the orgnal ICAMM to some degree. In future, other estmaton prncples and algorthms are expected to be explored n order to further mprove the classfcaton performance of ICAMM. Unsupervsed classfcaton has also been used n mcroarray data analyss. An example s the study by Lee and Batzoglou (2003), whch appled lnear and nonlnear ICA to project mcroarray data nto statstcally ndependent components that correspond to putatve bologcal processes. Then the genes can be grouped nto clusters based on the ndependent components obtaned. It has been found that ICA outperformed methods such as PCA, K-means clusterng and the Plad model, n constructng functonally coherent clusters on mcroarray datasets. Szu (2002) proposed a spectral ICA-based unsupervsed classfcaton algorthm for space-varant magng for breast cancer detectons, whch may offer an unbased, more senstve, accurate, and generally more effectve way to track the development of breast cancer. Sur (2003) also compared ICA and PCA for detectng coregulated gene groups n mcroarray data, and found that ICA may be more useful than PCA n fndng coregulated gene groups. 25

38 Chapter 2 Lterature Revew Comparsons between varous feature extracton methods and classfers In pattern classfcaton, there are many other feature extracton methods for use n addton to ICA. Some researchers have therefore conducted studes on comparng the drect ICA feature extracton method wth other feature extracton methods such as PCA. For example, Cao and Chong (2002) compared PCA, Kernel PCA (KPCA) and ICA for SVM classfcaton. They found that SVM ntegrated wth PCA, KPCA or ICA performs better than that wthout any feature extracton methods n terms of classfcaton accuracy. Furthermore, the KPCA and ICA feature extracton methods seem to be more sutable than PCA for the SVM classfer. Denz et al. (2003) conducted a comparson of classfcaton performance between PCA and ICA for SVM n face recognton. Ther experment results showed that PCA and ICA are comparable, whch may be due to the fact that the SVM classfer s nsenstve to the representaton space. As the tranng tme for ICA was more than that for PCA, Denz et al. (2003) suggested the use of PCA feature extracton method f the SVM classfer s adopted. Fortuna and Capson (2004) also compared the PCA and ICA feature extracton methods for face recognton based on SVM. Dfferent from the study by Denz et al. (2003), Fortuna and Capson (2004) drew the concluson that ICA outperformed PCA n ts generalzaton ablty by mprovng the margn and reducng the number of support vectors. Yang et al. (2005) used the SAR mage data to compare PCA and ICA feature extracton methods for KNN and SVM classfers. Ther concluson s that PCA and ICA are comparable wth each other. 26

39 Chapter 2 Lterature Revew Snce the drect ICA feature extracton method may be ntegrated wth varous classfers, t s meanngful to compare the performance of varous classfers wth the drect ICA feature extracton method. Jan and Huang (2004a) ntegrated ICA and LDA for gender classfcaton of face recognton. Ther study showed a sgnfcant mprovement n gender classfcaton accuracy rate after the drect ICA feature extracton method s used. Furthermore, Jan & Huang (2004b) appled ICA representaton of facal mages to nearest neghbor classfer, LDA and SVM for gender dentfcaton. The expermental results showed that SVM wth ICA may have better classfcaton performance than the other two. Kocsor and Toth (2004) compared the performance of artfcal neural networks (ANN), SVM and Gaussan mxture modelng (GMM) wth feature extracton methods such as PCA, ICA, LDA and sprngy dscrmnant analyss (SDA) for phoneme classfcaton. Ther expermental results showed that SVM ntegrated wth ICA has better classfcaton performance than other schemes. Glmore et al. (2004) appled ICA for mage feature extracton and compared the performance of vector quantzaton, neural network and Fsher classfer. Although the performance of all the three classfers has been mproved by ICA, the Fsher classfer seems to have the best classfcaton performance among the three classfers. Prasad et al. (2004) tested the performance of naïve Bayes, C4.5 and Seeded K-means ntegrated wth ICA through the classfcaton of Emphysema n Hgh Resoluton Computer Tomography (HRCT) mages. It s found that naïve Bayes n the ICA space acheved the best classfcaton performance. Ths s not surprsng as the ndependence assumpton between attrbutes n ICA space s consstent wth the underlyng assumpton of naïve Bayes. 27

Feature Reduction and Selection

Feature Reduction and Selection Feature Reducton and Selecton Dr. Shuang LIANG School of Software Engneerng TongJ Unversty Fall, 2012 Today s Topcs Introducton Problems of Dmensonalty Feature Reducton Statstc methods Prncpal Components

More information

The Research of Support Vector Machine in Agricultural Data Classification

The Research of Support Vector Machine in Agricultural Data Classification The Research of Support Vector Machne n Agrcultural Data Classfcaton Le Sh, Qguo Duan, Xnmng Ma, Me Weng College of Informaton and Management Scence, HeNan Agrcultural Unversty, Zhengzhou 45000 Chna Zhengzhou

More information

Support Vector Machines

Support Vector Machines /9/207 MIST.6060 Busness Intellgence and Data Mnng What are Support Vector Machnes? Support Vector Machnes Support Vector Machnes (SVMs) are supervsed learnng technques that analyze data and recognze patterns.

More information

Classifier Selection Based on Data Complexity Measures *

Classifier Selection Based on Data Complexity Measures * Classfer Selecton Based on Data Complexty Measures * Edth Hernández-Reyes, J.A. Carrasco-Ochoa, and J.Fco. Martínez-Trndad Natonal Insttute for Astrophyscs, Optcs and Electroncs, Lus Enrque Erro No.1 Sta.

More information

Learning the Kernel Parameters in Kernel Minimum Distance Classifier

Learning the Kernel Parameters in Kernel Minimum Distance Classifier Learnng the Kernel Parameters n Kernel Mnmum Dstance Classfer Daoqang Zhang 1,, Songcan Chen and Zh-Hua Zhou 1* 1 Natonal Laboratory for Novel Software Technology Nanjng Unversty, Nanjng 193, Chna Department

More information

Outline. Discriminative classifiers for image recognition. Where in the World? A nearest neighbor recognition example 4/14/2011. CS 376 Lecture 22 1

Outline. Discriminative classifiers for image recognition. Where in the World? A nearest neighbor recognition example 4/14/2011. CS 376 Lecture 22 1 4/14/011 Outlne Dscrmnatve classfers for mage recognton Wednesday, Aprl 13 Krsten Grauman UT-Austn Last tme: wndow-based generc obect detecton basc ppelne face detecton wth boostng as case study Today:

More information

Content Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers

Content Based Image Retrieval Using 2-D Discrete Wavelet with Texture Feature with Different Classifiers IOSR Journal of Electroncs and Communcaton Engneerng (IOSR-JECE) e-issn: 78-834,p- ISSN: 78-8735.Volume 9, Issue, Ver. IV (Mar - Apr. 04), PP 0-07 Content Based Image Retreval Usng -D Dscrete Wavelet wth

More information

Classification of Face Images Based on Gender using Dimensionality Reduction Techniques and SVM

Classification of Face Images Based on Gender using Dimensionality Reduction Techniques and SVM Classfcaton of Face Images Based on Gender usng Dmensonalty Reducton Technques and SVM Fahm Mannan 260 266 294 School of Computer Scence McGll Unversty Abstract Ths report presents gender classfcaton based

More information

Subspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;

Subspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points; Subspace clusterng Clusterng Fundamental to all clusterng technques s the choce of dstance measure between data ponts; D q ( ) ( ) 2 x x = x x, j k = 1 k jk Squared Eucldean dstance Assumpton: All features

More information

Classifying Acoustic Transient Signals Using Artificial Intelligence

Classifying Acoustic Transient Signals Using Artificial Intelligence Classfyng Acoustc Transent Sgnals Usng Artfcal Intellgence Steve Sutton, Unversty of North Carolna At Wlmngton (suttons@charter.net) Greg Huff, Unversty of North Carolna At Wlmngton (jgh7476@uncwl.edu)

More information

Cluster Analysis of Electrical Behavior

Cluster Analysis of Electrical Behavior Journal of Computer and Communcatons, 205, 3, 88-93 Publshed Onlne May 205 n ScRes. http://www.scrp.org/ournal/cc http://dx.do.org/0.4236/cc.205.350 Cluster Analyss of Electrcal Behavor Ln Lu Ln Lu, School

More information

Term Weighting Classification System Using the Chi-square Statistic for the Classification Subtask at NTCIR-6 Patent Retrieval Task

Term Weighting Classification System Using the Chi-square Statistic for the Classification Subtask at NTCIR-6 Patent Retrieval Task Proceedngs of NTCIR-6 Workshop Meetng, May 15-18, 2007, Tokyo, Japan Term Weghtng Classfcaton System Usng the Ch-square Statstc for the Classfcaton Subtask at NTCIR-6 Patent Retreval Task Kotaro Hashmoto

More information

Journal of Chemical and Pharmaceutical Research, 2014, 6(6): Research Article. A selective ensemble classification method on microarray data

Journal of Chemical and Pharmaceutical Research, 2014, 6(6): Research Article. A selective ensemble classification method on microarray data Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(6):2860-2866 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 A selectve ensemble classfcaton method on mcroarray

More information

CHAPTER 3 SEQUENTIAL MINIMAL OPTIMIZATION TRAINED SUPPORT VECTOR CLASSIFIER FOR CANCER PREDICTION

CHAPTER 3 SEQUENTIAL MINIMAL OPTIMIZATION TRAINED SUPPORT VECTOR CLASSIFIER FOR CANCER PREDICTION 48 CHAPTER 3 SEQUENTIAL MINIMAL OPTIMIZATION TRAINED SUPPORT VECTOR CLASSIFIER FOR CANCER PREDICTION 3.1 INTRODUCTION The raw mcroarray data s bascally an mage wth dfferent colors ndcatng hybrdzaton (Xue

More information

Smoothing Spline ANOVA for variable screening

Smoothing Spline ANOVA for variable screening Smoothng Splne ANOVA for varable screenng a useful tool for metamodels tranng and mult-objectve optmzaton L. Rcco, E. Rgon, A. Turco Outlne RSM Introducton Possble couplng Test case MOO MOO wth Game Theory

More information

BOOSTING CLASSIFICATION ACCURACY WITH SAMPLES CHOSEN FROM A VALIDATION SET

BOOSTING CLASSIFICATION ACCURACY WITH SAMPLES CHOSEN FROM A VALIDATION SET 1 BOOSTING CLASSIFICATION ACCURACY WITH SAMPLES CHOSEN FROM A VALIDATION SET TZU-CHENG CHUANG School of Electrcal and Computer Engneerng, Purdue Unversty, West Lafayette, Indana 47907 SAUL B. GELFAND School

More information

Detection of an Object by using Principal Component Analysis

Detection of an Object by using Principal Component Analysis Detecton of an Object by usng Prncpal Component Analyss 1. G. Nagaven, 2. Dr. T. Sreenvasulu Reddy 1. M.Tech, Department of EEE, SVUCE, Trupath, Inda. 2. Assoc. Professor, Department of ECE, SVUCE, Trupath,

More information

Classification / Regression Support Vector Machines

Classification / Regression Support Vector Machines Classfcaton / Regresson Support Vector Machnes Jeff Howbert Introducton to Machne Learnng Wnter 04 Topcs SVM classfers for lnearly separable classes SVM classfers for non-lnearly separable classes SVM

More information

NAG Fortran Library Chapter Introduction. G10 Smoothing in Statistics

NAG Fortran Library Chapter Introduction. G10 Smoothing in Statistics Introducton G10 NAG Fortran Lbrary Chapter Introducton G10 Smoothng n Statstcs Contents 1 Scope of the Chapter... 2 2 Background to the Problems... 2 2.1 Smoothng Methods... 2 2.2 Smoothng Splnes and Regresson

More information

Outline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like:

Outline. Self-Organizing Maps (SOM) US Hebbian Learning, Cntd. The learning rule is Hebbian like: Self-Organzng Maps (SOM) Turgay İBRİKÇİ, PhD. Outlne Introducton Structures of SOM SOM Archtecture Neghborhoods SOM Algorthm Examples Summary 1 2 Unsupervsed Hebban Learnng US Hebban Learnng, Cntd 3 A

More information

Machine Learning 9. week

Machine Learning 9. week Machne Learnng 9. week Mappng Concept Radal Bass Functons (RBF) RBF Networks 1 Mappng It s probably the best scenaro for the classfcaton of two dataset s to separate them lnearly. As you see n the below

More information

Investigating the Performance of Naïve- Bayes Classifiers and K- Nearest Neighbor Classifiers

Investigating the Performance of Naïve- Bayes Classifiers and K- Nearest Neighbor Classifiers Journal of Convergence Informaton Technology Volume 5, Number 2, Aprl 2010 Investgatng the Performance of Naïve- Bayes Classfers and K- Nearest Neghbor Classfers Mohammed J. Islam *, Q. M. Jonathan Wu,

More information

A Fast Visual Tracking Algorithm Based on Circle Pixels Matching

A Fast Visual Tracking Algorithm Based on Circle Pixels Matching A Fast Vsual Trackng Algorthm Based on Crcle Pxels Matchng Zhqang Hou hou_zhq@sohu.com Chongzhao Han czhan@mal.xjtu.edu.cn Ln Zheng Abstract: A fast vsual trackng algorthm based on crcle pxels matchng

More information

Face Recognition University at Buffalo CSE666 Lecture Slides Resources:

Face Recognition University at Buffalo CSE666 Lecture Slides Resources: Face Recognton Unversty at Buffalo CSE666 Lecture Sldes Resources: http://www.face-rec.org/algorthms/ Overvew of face recognton algorthms Correlaton - Pxel based correspondence between two face mages Structural

More information

Announcements. Supervised Learning

Announcements. Supervised Learning Announcements See Chapter 5 of Duda, Hart, and Stork. Tutoral by Burge lnked to on web page. Supervsed Learnng Classfcaton wth labeled eamples. Images vectors n hgh-d space. Supervsed Learnng Labeled eamples

More information

Lecture 5: Multilayer Perceptrons

Lecture 5: Multilayer Perceptrons Lecture 5: Multlayer Perceptrons Roger Grosse 1 Introducton So far, we ve only talked about lnear models: lnear regresson and lnear bnary classfers. We noted that there are functons that can t be represented

More information

A Binarization Algorithm specialized on Document Images and Photos

A Binarization Algorithm specialized on Document Images and Photos A Bnarzaton Algorthm specalzed on Document mages and Photos Ergna Kavalleratou Dept. of nformaton and Communcaton Systems Engneerng Unversty of the Aegean kavalleratou@aegean.gr Abstract n ths paper, a

More information

Determining the Optimal Bandwidth Based on Multi-criterion Fusion

Determining the Optimal Bandwidth Based on Multi-criterion Fusion Proceedngs of 01 4th Internatonal Conference on Machne Learnng and Computng IPCSIT vol. 5 (01) (01) IACSIT Press, Sngapore Determnng the Optmal Bandwdth Based on Mult-crteron Fuson Ha-L Lang 1+, Xan-Mn

More information

FEATURE EXTRACTION. Dr. K.Vijayarekha. Associate Dean School of Electrical and Electronics Engineering SASTRA University, Thanjavur

FEATURE EXTRACTION. Dr. K.Vijayarekha. Associate Dean School of Electrical and Electronics Engineering SASTRA University, Thanjavur FEATURE EXTRACTION Dr. K.Vjayarekha Assocate Dean School of Electrcal and Electroncs Engneerng SASTRA Unversty, Thanjavur613 41 Jont Intatve of IITs and IISc Funded by MHRD Page 1 of 8 Table of Contents

More information

Edge Detection in Noisy Images Using the Support Vector Machines

Edge Detection in Noisy Images Using the Support Vector Machines Edge Detecton n Nosy Images Usng the Support Vector Machnes Hlaro Gómez-Moreno, Saturnno Maldonado-Bascón, Francsco López-Ferreras Sgnal Theory and Communcatons Department. Unversty of Alcalá Crta. Madrd-Barcelona

More information

Face Recognition Based on SVM and 2DPCA

Face Recognition Based on SVM and 2DPCA Vol. 4, o. 3, September, 2011 Face Recognton Based on SVM and 2DPCA Tha Hoang Le, Len Bu Faculty of Informaton Technology, HCMC Unversty of Scence Faculty of Informaton Scences and Engneerng, Unversty

More information

A Fast Content-Based Multimedia Retrieval Technique Using Compressed Data

A Fast Content-Based Multimedia Retrieval Technique Using Compressed Data A Fast Content-Based Multmeda Retreval Technque Usng Compressed Data Borko Furht and Pornvt Saksobhavvat NSF Multmeda Laboratory Florda Atlantc Unversty, Boca Raton, Florda 3343 ABSTRACT In ths paper,

More information

CS 534: Computer Vision Model Fitting

CS 534: Computer Vision Model Fitting CS 534: Computer Vson Model Fttng Sprng 004 Ahmed Elgammal Dept of Computer Scence CS 534 Model Fttng - 1 Outlnes Model fttng s mportant Least-squares fttng Maxmum lkelhood estmaton MAP estmaton Robust

More information

Improvement of Spatial Resolution Using BlockMatching Based Motion Estimation and Frame. Integration

Improvement of Spatial Resolution Using BlockMatching Based Motion Estimation and Frame. Integration Improvement of Spatal Resoluton Usng BlockMatchng Based Moton Estmaton and Frame Integraton Danya Suga and Takayuk Hamamoto Graduate School of Engneerng, Tokyo Unversty of Scence, 6-3-1, Nuku, Katsuska-ku,

More information

S1 Note. Basis functions.

S1 Note. Basis functions. S1 Note. Bass functons. Contents Types of bass functons...1 The Fourer bass...2 B-splne bass...3 Power and type I error rates wth dfferent numbers of bass functons...4 Table S1. Smulaton results of type

More information

Collaboratively Regularized Nearest Points for Set Based Recognition

Collaboratively Regularized Nearest Points for Set Based Recognition Academc Center for Computng and Meda Studes, Kyoto Unversty Collaboratvely Regularzed Nearest Ponts for Set Based Recognton Yang Wu, Mchhko Mnoh, Masayuk Mukunok Kyoto Unversty 9/1/013 BMVC 013 @ Brstol,

More information

Using Neural Networks and Support Vector Machines in Data Mining

Using Neural Networks and Support Vector Machines in Data Mining Usng eural etworks and Support Vector Machnes n Data Mnng RICHARD A. WASIOWSKI Computer Scence Department Calforna State Unversty Domnguez Hlls Carson, CA 90747 USA Abstract: - Multvarate data analyss

More information

Discriminative Dictionary Learning with Pairwise Constraints

Discriminative Dictionary Learning with Pairwise Constraints Dscrmnatve Dctonary Learnng wth Parwse Constrants Humn Guo Zhuoln Jang LARRY S. DAVIS UNIVERSITY OF MARYLAND Nov. 6 th, Outlne Introducton/motvaton Dctonary Learnng Dscrmnatve Dctonary Learnng wth Parwse

More information

An Entropy-Based Approach to Integrated Information Needs Assessment

An Entropy-Based Approach to Integrated Information Needs Assessment Dstrbuton Statement A: Approved for publc release; dstrbuton s unlmted. An Entropy-Based Approach to ntegrated nformaton Needs Assessment June 8, 2004 Wllam J. Farrell Lockheed Martn Advanced Technology

More information

Support Vector Machines

Support Vector Machines Support Vector Machnes Decson surface s a hyperplane (lne n 2D) n feature space (smlar to the Perceptron) Arguably, the most mportant recent dscovery n machne learnng In a nutshell: map the data to a predetermned

More information

SVM-based Learning for Multiple Model Estimation

SVM-based Learning for Multiple Model Estimation SVM-based Learnng for Multple Model Estmaton Vladmr Cherkassky and Yunqan Ma Department of Electrcal and Computer Engneerng Unversty of Mnnesota Mnneapols, MN 55455 {cherkass,myq}@ece.umn.edu Abstract:

More information

Review of approximation techniques

Review of approximation techniques CHAPTER 2 Revew of appromaton technques 2. Introducton Optmzaton problems n engneerng desgn are characterzed by the followng assocated features: the objectve functon and constrants are mplct functons evaluated

More information

User Authentication Based On Behavioral Mouse Dynamics Biometrics

User Authentication Based On Behavioral Mouse Dynamics Biometrics User Authentcaton Based On Behavoral Mouse Dynamcs Bometrcs Chee-Hyung Yoon Danel Donghyun Km Department of Computer Scence Department of Computer Scence Stanford Unversty Stanford Unversty Stanford, CA

More information

CAN COMPUTERS LEARN FASTER? Seyda Ertekin Computer Science & Engineering The Pennsylvania State University

CAN COMPUTERS LEARN FASTER? Seyda Ertekin Computer Science & Engineering The Pennsylvania State University CAN COMPUTERS LEARN FASTER? Seyda Ertekn Computer Scence & Engneerng The Pennsylvana State Unversty sertekn@cse.psu.edu ABSTRACT Ever snce computers were nvented, manknd wondered whether they mght be made

More information

Fuzzy Modeling of the Complexity vs. Accuracy Trade-off in a Sequential Two-Stage Multi-Classifier System

Fuzzy Modeling of the Complexity vs. Accuracy Trade-off in a Sequential Two-Stage Multi-Classifier System Fuzzy Modelng of the Complexty vs. Accuracy Trade-off n a Sequental Two-Stage Mult-Classfer System MARK LAST 1 Department of Informaton Systems Engneerng Ben-Guron Unversty of the Negev Beer-Sheva 84105

More information

Concurrent Apriori Data Mining Algorithms

Concurrent Apriori Data Mining Algorithms Concurrent Apror Data Mnng Algorthms Vassl Halatchev Department of Electrcal Engneerng and Computer Scence York Unversty, Toronto October 8, 2015 Outlne Why t s mportant Introducton to Assocaton Rule Mnng

More information

Skew Angle Estimation and Correction of Hand Written, Textual and Large areas of Non-Textual Document Images: A Novel Approach

Skew Angle Estimation and Correction of Hand Written, Textual and Large areas of Non-Textual Document Images: A Novel Approach Angle Estmaton and Correcton of Hand Wrtten, Textual and Large areas of Non-Textual Document Images: A Novel Approach D.R.Ramesh Babu Pyush M Kumat Mahesh D Dhannawat PES Insttute of Technology Research

More information

y and the total sum of

y and the total sum of Lnear regresson Testng for non-lnearty In analytcal chemstry, lnear regresson s commonly used n the constructon of calbraton functons requred for analytcal technques such as gas chromatography, atomc absorpton

More information

APPLICATION OF MULTIVARIATE LOSS FUNCTION FOR ASSESSMENT OF THE QUALITY OF TECHNOLOGICAL PROCESS MANAGEMENT

APPLICATION OF MULTIVARIATE LOSS FUNCTION FOR ASSESSMENT OF THE QUALITY OF TECHNOLOGICAL PROCESS MANAGEMENT 3. - 5. 5., Brno, Czech Republc, EU APPLICATION OF MULTIVARIATE LOSS FUNCTION FOR ASSESSMENT OF THE QUALITY OF TECHNOLOGICAL PROCESS MANAGEMENT Abstract Josef TOŠENOVSKÝ ) Lenka MONSPORTOVÁ ) Flp TOŠENOVSKÝ

More information

Incremental Learning with Support Vector Machines and Fuzzy Set Theory

Incremental Learning with Support Vector Machines and Fuzzy Set Theory The 25th Workshop on Combnatoral Mathematcs and Computaton Theory Incremental Learnng wth Support Vector Machnes and Fuzzy Set Theory Yu-Mng Chuang 1 and Cha-Hwa Ln 2* 1 Department of Computer Scence and

More information

A Modified Median Filter for the Removal of Impulse Noise Based on the Support Vector Machines

A Modified Median Filter for the Removal of Impulse Noise Based on the Support Vector Machines A Modfed Medan Flter for the Removal of Impulse Nose Based on the Support Vector Machnes H. GOMEZ-MORENO, S. MALDONADO-BASCON, F. LOPEZ-FERRERAS, M. UTRILLA- MANSO AND P. GIL-JIMENEZ Departamento de Teoría

More information

Meta-heuristics for Multidimensional Knapsack Problems

Meta-heuristics for Multidimensional Knapsack Problems 2012 4th Internatonal Conference on Computer Research and Development IPCSIT vol.39 (2012) (2012) IACSIT Press, Sngapore Meta-heurstcs for Multdmensonal Knapsack Problems Zhbao Man + Computer Scence Department,

More information

Proper Choice of Data Used for the Estimation of Datum Transformation Parameters

Proper Choice of Data Used for the Estimation of Datum Transformation Parameters Proper Choce of Data Used for the Estmaton of Datum Transformaton Parameters Hakan S. KUTOGLU, Turkey Key words: Coordnate systems; transformaton; estmaton, relablty. SUMMARY Advances n technologes and

More information

Associative Based Classification Algorithm For Diabetes Disease Prediction

Associative Based Classification Algorithm For Diabetes Disease Prediction Internatonal Journal of Engneerng Trends and Technology (IJETT) Volume-41 Number-3 - November 016 Assocatve Based Classfcaton Algorthm For Dabetes Dsease Predcton 1 N. Gnana Deepka, Y.surekha, 3 G.Laltha

More information

IMAGE FUSION BASED ON EXTENSIONS OF INDEPENDENT COMPONENT ANALYSIS

IMAGE FUSION BASED ON EXTENSIONS OF INDEPENDENT COMPONENT ANALYSIS IMAGE FUSION BASED ON EXTENSIONS OF INDEPENDENT COMPONENT ANALYSIS M Chen a, *, Yngchun Fu b, Deren L c, Qanqng Qn c a College of Educaton Technology, Captal Normal Unversty, Bejng 00037,Chna - (merc@hotmal.com)

More information

Wishing you all a Total Quality New Year!

Wishing you all a Total Quality New Year! Total Qualty Management and Sx Sgma Post Graduate Program 214-15 Sesson 4 Vnay Kumar Kalakband Assstant Professor Operatons & Systems Area 1 Wshng you all a Total Qualty New Year! Hope you acheve Sx sgma

More information

Parallelism for Nested Loops with Non-uniform and Flow Dependences

Parallelism for Nested Loops with Non-uniform and Flow Dependences Parallelsm for Nested Loops wth Non-unform and Flow Dependences Sam-Jn Jeong Dept. of Informaton & Communcaton Engneerng, Cheonan Unversty, 5, Anseo-dong, Cheonan, Chungnam, 330-80, Korea. seong@cheonan.ac.kr

More information

X- Chart Using ANOM Approach

X- Chart Using ANOM Approach ISSN 1684-8403 Journal of Statstcs Volume 17, 010, pp. 3-3 Abstract X- Chart Usng ANOM Approach Gullapall Chakravarth 1 and Chaluvad Venkateswara Rao Control lmts for ndvdual measurements (X) chart are

More information

Three supervised learning methods on pen digits character recognition dataset

Three supervised learning methods on pen digits character recognition dataset Three supervsed learnng methods on pen dgts character recognton dataset Chrs Flezach Department of Computer Scence and Engneerng Unversty of Calforna, San Dego San Dego, CA 92093 cflezac@cs.ucsd.edu Satoru

More information

Discriminative classifiers for object classification. Last time

Discriminative classifiers for object classification. Last time Dscrmnatve classfers for object classfcaton Thursday, Nov 12 Krsten Grauman UT Austn Last tme Supervsed classfcaton Loss and rsk, kbayes rule Skn color detecton example Sldng ndo detecton Classfers, boostng

More information

A new Unsupervised Clustering-based Feature Extraction Method

A new Unsupervised Clustering-based Feature Extraction Method A new Unsupervsed Clusterng-based Feature Extracton Method Sabra El Ferchch ACS Natonal School of Engneerng at Tuns, Tunsa Salah Zd AGIS lle Unversty of Scence and Technology, France Kaouther aabd ACS

More information

Comparison Study of Textural Descriptors for Training Neural Network Classifiers

Comparison Study of Textural Descriptors for Training Neural Network Classifiers Comparson Study of Textural Descrptors for Tranng Neural Network Classfers G.D. MAGOULAS (1) S.A. KARKANIS (1) D.A. KARRAS () and M.N. VRAHATIS (3) (1) Department of Informatcs Unversty of Athens GR-157.84

More information

SLAM Summer School 2006 Practical 2: SLAM using Monocular Vision

SLAM Summer School 2006 Practical 2: SLAM using Monocular Vision SLAM Summer School 2006 Practcal 2: SLAM usng Monocular Vson Javer Cvera, Unversty of Zaragoza Andrew J. Davson, Imperal College London J.M.M Montel, Unversty of Zaragoza. josemar@unzar.es, jcvera@unzar.es,

More information

Outline. Type of Machine Learning. Examples of Application. Unsupervised Learning

Outline. Type of Machine Learning. Examples of Application. Unsupervised Learning Outlne Artfcal Intellgence and ts applcatons Lecture 8 Unsupervsed Learnng Professor Danel Yeung danyeung@eee.org Dr. Patrck Chan patrckchan@eee.org South Chna Unversty of Technology, Chna Introducton

More information

Tsinghua University at TAC 2009: Summarizing Multi-documents by Information Distance

Tsinghua University at TAC 2009: Summarizing Multi-documents by Information Distance Tsnghua Unversty at TAC 2009: Summarzng Mult-documents by Informaton Dstance Chong Long, Mnle Huang, Xaoyan Zhu State Key Laboratory of Intellgent Technology and Systems, Tsnghua Natonal Laboratory for

More information

A PATTERN RECOGNITION APPROACH TO IMAGE SEGMENTATION

A PATTERN RECOGNITION APPROACH TO IMAGE SEGMENTATION 1 THE PUBLISHING HOUSE PROCEEDINGS OF THE ROMANIAN ACADEMY, Seres A, OF THE ROMANIAN ACADEMY Volume 4, Number 2/2003, pp.000-000 A PATTERN RECOGNITION APPROACH TO IMAGE SEGMENTATION Tudor BARBU Insttute

More information

EYE CENTER LOCALIZATION ON A FACIAL IMAGE BASED ON MULTI-BLOCK LOCAL BINARY PATTERNS

EYE CENTER LOCALIZATION ON A FACIAL IMAGE BASED ON MULTI-BLOCK LOCAL BINARY PATTERNS P.G. Demdov Yaroslavl State Unversty Anatoly Ntn, Vladmr Khryashchev, Olga Stepanova, Igor Kostern EYE CENTER LOCALIZATION ON A FACIAL IMAGE BASED ON MULTI-BLOCK LOCAL BINARY PATTERNS Yaroslavl, 2015 Eye

More information

MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION

MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION MULTISPECTRAL IMAGES CLASSIFICATION BASED ON KLT AND ATR AUTOMATIC TARGET RECOGNITION Paulo Quntlano 1 & Antono Santa-Rosa 1 Federal Polce Department, Brasla, Brazl. E-mals: quntlano.pqs@dpf.gov.br and

More information

Human Face Recognition Using Generalized. Kernel Fisher Discriminant

Human Face Recognition Using Generalized. Kernel Fisher Discriminant Human Face Recognton Usng Generalzed Kernel Fsher Dscrmnant ng-yu Sun,2 De-Shuang Huang Ln Guo. Insttute of Intellgent Machnes, Chnese Academy of Scences, P.O.ox 30, Hefe, Anhu, Chna. 2. Department of

More information

RECOGNIZING GENDER THROUGH FACIAL IMAGE USING SUPPORT VECTOR MACHINE

RECOGNIZING GENDER THROUGH FACIAL IMAGE USING SUPPORT VECTOR MACHINE Journal of Theoretcal and Appled Informaton Technology 30 th June 06. Vol.88. No.3 005-06 JATIT & LLS. All rghts reserved. ISSN: 99-8645 www.jatt.org E-ISSN: 87-395 RECOGNIZING GENDER THROUGH FACIAL IMAGE

More information

Machine Learning. Support Vector Machines. (contains material adapted from talks by Constantin F. Aliferis & Ioannis Tsamardinos, and Martin Law)

Machine Learning. Support Vector Machines. (contains material adapted from talks by Constantin F. Aliferis & Ioannis Tsamardinos, and Martin Law) Machne Learnng Support Vector Machnes (contans materal adapted from talks by Constantn F. Alfers & Ioanns Tsamardnos, and Martn Law) Bryan Pardo, Machne Learnng: EECS 349 Fall 2014 Support Vector Machnes

More information

Backpropagation: In Search of Performance Parameters

Backpropagation: In Search of Performance Parameters Bacpropagaton: In Search of Performance Parameters ANIL KUMAR ENUMULAPALLY, LINGGUO BU, and KHOSROW KAIKHAH, Ph.D. Computer Scence Department Texas State Unversty-San Marcos San Marcos, TX-78666 USA ae049@txstate.edu,

More information

Unsupervised Learning

Unsupervised Learning Pattern Recognton Lecture 8 Outlne Introducton Unsupervsed Learnng Parametrc VS Non-Parametrc Approach Mxture of Denstes Maxmum-Lkelhood Estmates Clusterng Prof. Danel Yeung School of Computer Scence and

More information

Recognizing Faces. Outline

Recognizing Faces. Outline Recognzng Faces Drk Colbry Outlne Introducton and Motvaton Defnng a feature vector Prncpal Component Analyss Lnear Dscrmnate Analyss !"" #$""% http://www.nfotech.oulu.f/annual/2004 + &'()*) '+)* 2 ! &

More information

The Nottingham eprints service makes this work by researchers of the University of Nottingham available open access under the following conditions.

The Nottingham eprints service makes this work by researchers of the University of Nottingham available open access under the following conditions. Da, Wujao and Lu, Bn and Meng, Xaoln and Huang, D. () Spato-temporal modellng of dam deformaton usng ndependent component analyss. Survey Revew, 6 (9). pp. 7-. ISSN 7-76 Access from the Unversty of Nottngham

More information

Modular PCA Face Recognition Based on Weighted Average

Modular PCA Face Recognition Based on Weighted Average odern Appled Scence odular PCA Face Recognton Based on Weghted Average Chengmao Han (Correspondng author) Department of athematcs, Lny Normal Unversty Lny 76005, Chna E-mal: hanchengmao@163.com Abstract

More information

The Study of Remote Sensing Image Classification Based on Support Vector Machine

The Study of Remote Sensing Image Classification Based on Support Vector Machine Sensors & Transducers 03 by IFSA http://www.sensorsportal.com The Study of Remote Sensng Image Classfcaton Based on Support Vector Machne, ZHANG Jan-Hua Key Research Insttute of Yellow Rver Cvlzaton and

More information

A New Feature of Uniformity of Image Texture Directions Coinciding with the Human Eyes Perception 1

A New Feature of Uniformity of Image Texture Directions Coinciding with the Human Eyes Perception 1 A New Feature of Unformty of Image Texture Drectons Concdng wth the Human Eyes Percepton Xng-Jan He, De-Shuang Huang, Yue Zhang, Tat-Mng Lo 2, and Mchael R. Lyu 3 Intellgent Computng Lab, Insttute of Intellgent

More information

REFRACTIVE INDEX SELECTION FOR POWDER MIXTURES

REFRACTIVE INDEX SELECTION FOR POWDER MIXTURES REFRACTIVE INDEX SELECTION FOR POWDER MIXTURES Laser dffracton s one of the most wdely used methods for partcle sze analyss of mcron and submcron sze powders and dspersons. It s quck and easy and provdes

More information

Image Representation & Visualization Basic Imaging Algorithms Shape Representation and Analysis. outline

Image Representation & Visualization Basic Imaging Algorithms Shape Representation and Analysis. outline mage Vsualzaton mage Vsualzaton mage Representaton & Vsualzaton Basc magng Algorthms Shape Representaton and Analyss outlne mage Representaton & Vsualzaton Basc magng Algorthms Shape Representaton and

More information

A Bilinear Model for Sparse Coding

A Bilinear Model for Sparse Coding A Blnear Model for Sparse Codng Davd B. Grmes and Rajesh P. N. Rao Department of Computer Scence and Engneerng Unversty of Washngton Seattle, WA 98195-2350, U.S.A. grmes,rao @cs.washngton.edu Abstract

More information

ISSN: International Journal of Engineering and Innovative Technology (IJEIT) Volume 1, Issue 4, April 2012

ISSN: International Journal of Engineering and Innovative Technology (IJEIT) Volume 1, Issue 4, April 2012 Performance Evoluton of Dfferent Codng Methods wth β - densty Decodng Usng Error Correctng Output Code Based on Multclass Classfcaton Devangn Dave, M. Samvatsar, P. K. Bhanoda Abstract A common way to

More information

Relevance Assignment and Fusion of Multiple Learning Methods Applied to Remote Sensing Image Analysis

Relevance Assignment and Fusion of Multiple Learning Methods Applied to Remote Sensing Image Analysis Assgnment and Fuson of Multple Learnng Methods Appled to Remote Sensng Image Analyss Peter Bajcsy, We-Wen Feng and Praveen Kumar Natonal Center for Supercomputng Applcaton (NCSA), Unversty of Illnos at

More information

Lecture 4: Principal components

Lecture 4: Principal components /3/6 Lecture 4: Prncpal components 3..6 Multvarate lnear regresson MLR s optmal for the estmaton data...but poor for handlng collnear data Covarance matrx s not nvertble (large condton number) Robustness

More information

NUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS

NUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS ARPN Journal of Engneerng and Appled Scences 006-017 Asan Research Publshng Network (ARPN). All rghts reserved. NUMERICAL SOLVING OPTIMAL CONTROL PROBLEMS BY THE METHOD OF VARIATIONS Igor Grgoryev, Svetlana

More information

Correlative features for the classification of textural images

Correlative features for the classification of textural images Correlatve features for the classfcaton of textural mages M A Turkova 1 and A V Gadel 1, 1 Samara Natonal Research Unversty, Moskovskoe Shosse 34, Samara, Russa, 443086 Image Processng Systems Insttute

More information

A Unified Framework for Semantics and Feature Based Relevance Feedback in Image Retrieval Systems

A Unified Framework for Semantics and Feature Based Relevance Feedback in Image Retrieval Systems A Unfed Framework for Semantcs and Feature Based Relevance Feedback n Image Retreval Systems Ye Lu *, Chunhu Hu 2, Xngquan Zhu 3*, HongJang Zhang 2, Qang Yang * School of Computng Scence Smon Fraser Unversty

More information

An Image Fusion Approach Based on Segmentation Region

An Image Fusion Approach Based on Segmentation Region Rong Wang, L-Qun Gao, Shu Yang, Yu-Hua Cha, and Yan-Chun Lu An Image Fuson Approach Based On Segmentaton Regon An Image Fuson Approach Based on Segmentaton Regon Rong Wang, L-Qun Gao, Shu Yang 3, Yu-Hua

More information

The Discriminate Analysis and Dimension Reduction Methods of High Dimension

The Discriminate Analysis and Dimension Reduction Methods of High Dimension Open Journal of Socal Scences, 015, 3, 7-13 Publshed Onlne March 015 n ScRes. http://www.scrp.org/journal/jss http://dx.do.org/10.436/jss.015.3300 The Dscrmnate Analyss and Dmenson Reducton Methods of

More information

CSCI 5417 Information Retrieval Systems Jim Martin!

CSCI 5417 Information Retrieval Systems Jim Martin! CSCI 5417 Informaton Retreval Systems Jm Martn! Lecture 11 9/29/2011 Today 9/29 Classfcaton Naïve Bayes classfcaton Ungram LM 1 Where we are... Bascs of ad hoc retreval Indexng Term weghtng/scorng Cosne

More information

Feature Selection as an Improving Step for Decision Tree Construction

Feature Selection as an Improving Step for Decision Tree Construction 2009 Internatonal Conference on Machne Learnng and Computng IPCSIT vol.3 (2011) (2011) IACSIT Press, Sngapore Feature Selecton as an Improvng Step for Decson Tree Constructon Mahd Esmael 1, Fazekas Gabor

More information

Solving two-person zero-sum game by Matlab

Solving two-person zero-sum game by Matlab Appled Mechancs and Materals Onlne: 2011-02-02 ISSN: 1662-7482, Vols. 50-51, pp 262-265 do:10.4028/www.scentfc.net/amm.50-51.262 2011 Trans Tech Publcatons, Swtzerland Solvng two-person zero-sum game by

More information

Learning a Class-Specific Dictionary for Facial Expression Recognition

Learning a Class-Specific Dictionary for Facial Expression Recognition BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 16, No 4 Sofa 016 Prnt ISSN: 1311-970; Onlne ISSN: 1314-4081 DOI: 10.1515/cat-016-0067 Learnng a Class-Specfc Dctonary for

More information

A Novel Term_Class Relevance Measure for Text Categorization

A Novel Term_Class Relevance Measure for Text Categorization A Novel Term_Class Relevance Measure for Text Categorzaton D S Guru, Mahamad Suhl Department of Studes n Computer Scence, Unversty of Mysore, Mysore, Inda Abstract: In ths paper, we ntroduce a new measure

More information

High resolution 3D Tau-p transform by matching pursuit Weiping Cao* and Warren S. Ross, Shearwater GeoServices

High resolution 3D Tau-p transform by matching pursuit Weiping Cao* and Warren S. Ross, Shearwater GeoServices Hgh resoluton 3D Tau-p transform by matchng pursut Wepng Cao* and Warren S. Ross, Shearwater GeoServces Summary The 3D Tau-p transform s of vtal sgnfcance for processng sesmc data acqured wth modern wde

More information

A Multivariate Analysis of Static Code Attributes for Defect Prediction

A Multivariate Analysis of Static Code Attributes for Defect Prediction Research Paper) A Multvarate Analyss of Statc Code Attrbutes for Defect Predcton Burak Turhan, Ayşe Bener Department of Computer Engneerng, Bogazc Unversty 3434, Bebek, Istanbul, Turkey {turhanb, bener}@boun.edu.tr

More information

Modeling Inter-cluster and Intra-cluster Discrimination Among Triphones

Modeling Inter-cluster and Intra-cluster Discrimination Among Triphones Modelng Inter-cluster and Intra-cluster Dscrmnaton Among Trphones Tom Ko, Bran Mak and Dongpeng Chen Department of Computer Scence and Engneerng The Hong Kong Unversty of Scence and Technology Clear Water

More information

Local Quaternary Patterns and Feature Local Quaternary Patterns

Local Quaternary Patterns and Feature Local Quaternary Patterns Local Quaternary Patterns and Feature Local Quaternary Patterns Jayu Gu and Chengjun Lu The Department of Computer Scence, New Jersey Insttute of Technology, Newark, NJ 0102, USA Abstract - Ths paper presents

More information

Feature Selection for Target Detection in SAR Images

Feature Selection for Target Detection in SAR Images Feature Selecton for Detecton n SAR Images Br Bhanu, Yngqang Ln and Shqn Wang Center for Research n Intellgent Systems Unversty of Calforna, Rversde, CA 95, USA Abstract A genetc algorthm (GA) approach

More information

Online Detection and Classification of Moving Objects Using Progressively Improving Detectors

Online Detection and Classification of Moving Objects Using Progressively Improving Detectors Onlne Detecton and Classfcaton of Movng Objects Usng Progressvely Improvng Detectors Omar Javed Saad Al Mubarak Shah Computer Vson Lab School of Computer Scence Unversty of Central Florda Orlando, FL 32816

More information