Human recognition based on ear shape images using PCA-Wavelets and different classification methods

Transcription

1 Meical Devices an Diagnostic Engineering Review Article ISSN: Human recognition base on ear shape images using PCA-Wavelets an ifferent classification methos Ali Mahmou Mayya1* an Mariam Mohamma Saii PhD stuent, Computer Engineering, Tishreen University, Syria Abstract A new approach of human recognition using ear images is introuce. It consists of two basic steps which are the ear segmentation an ear recognition. In the first one, Likelihoo skin etector is use to etermine the skin areas in the sie face images. Then, some of the morphological operations are applie to etermine the ear region. This region is extracte using image processing techniques. The ear recognition step epens on the segmente ear images as inputs. A hybri PCA_Wavelet algorithm is use to extract the ear features from ear. Finally, the fee forwaring back propagation neural network is traine using the feature vectors. Tests which applie on 460 images, which have been taken uring 4 months an uner ifferent illumination an pose variations, show that the system achieve a rate of 96.73% for ear extraction an 98.9% for recognition. More experiments are one to specify the best wavelet level, the best number of features, the best classification metho, an the best threshol value. The stuy is also compare with other ones at the area of ear recognition. Introuction Ear recognition is an example of the Human recognition using biometrics epening on the human biological properties. This kin of recognition has been recently use because of the ear s istinctive properties such as its invariant shape. The ear is efine by unique shape that contains ifferent parts (in location an size), an obvious curve lines that facilitate the feature extraction phase. Even the ears of ientical twins iffer in some respects [3,7,15,16], also Ear is less susceptible to istortions than the fingerprint or hanprint (fingerprint may suffer from burns, wouns or other eformities) so, ear is use for recognition especially in air planes accients [5,14]. In aition to that, ear images is easier to be collecte than any other biometrics since it oesn t require any tools to be inserte in human boy to collect samples as in DNA, an this is why ear is consiere non-invasive. We can also observe that ear images are smaller than face, han or leg ones which makes the recognition process fast an easy [14]. The most important thing is that ear maintains its shape between 8 an 70 of ages so, its changes are very slow an little [11]. Relate work A. Ianarelli [10] was the first actual person who use ear to recognize people. He compare 10,000 ears rawn from a ranomly selecte sample in California. Although his stuy was important, it stills ifficult to apply on computer because of the ifficulty of localizing the anatomical point which serves as the origin of the measurement of the system [1]. Burge an Burger [1] propose the use of Voronoi iagrams. Their technique use an ajacency graph built from Voroni regions of earcurve segments. Although their paper ha no recognition results, it prompte a range of further stuies into the effectiveness of ears as a biometric [2]. Their system in t take in account illumination an pose variation. Morone et al. [13] trie three neural networks approaches to recognize people base on ear images. These approaches were (Bora, Bayesian, an Weighte Bayesian combination) which epene on the macrofeatures extracte from the ear. As a result, they got 93% recognition rate. The genetic search was a powerful approach use by Yuizono [18] to esign a robust ear recognition system. Automatic extraction using template matching was use to obtain ear images. The propose system inclue 6 images from vieo frame for each iniviual; the first three images use for atabase while the secon three images use as test images. No illumination or pose variations were taken in account. Occlusion by hair or earrings also was not consiere. The recognition rate was 100%. Later, Hurley [9] applie the force fiel transform an PCA to extract the force features like energy lines, in the classification stage they use the Eucliian istance classifier to obtain 99.2% recognition rate. Mahbubur Rahman [15] use the Generalize Hugh Transform (GHT) to extract the ear s features from the ear image which was extracte from the acquire image using Masking. The extracte features along with the subjects i were store in the atabase for testing for a match. As a result they got almost 89% recognition rate. Recently, Islam [11] ha propose using AaBoost algorithm to extract ear images. They also traine their system with rectangular haar-like features epening on a set of training images with ifferent Corresponence to: Ali Mahmou Mayya, Computer an Automatic Control Engineering epartment, Tishreen University, Syria, alimia1988@yahoo.com Key wors: ear recognition, ear segmentation, ear images, back propagation, PCA, Wavelet Receive: August 14, 2016; Accepte: September 12, 2016; Publishe: September 16, 2016

2 age an gener an uner various illumination an pose conitions. Later, they upate their system by means of ICP algorithm. They obtaine a rank one recognition rate of 93% while testing with the University of Notre Dame Biometrics Database. Zhichun Mu an Zhaoxia Xie [17] compare the use of locally linear embeing (LLE) an PCA algorithms; they foun that the first algorithm is better in recognition an when they applie the improve version IDLLE, the recognition rate increase precisely. They also foun that recognition ecrease with pose variation, but they commonly got 80% recognition rate. In a recent publication of Kumar an Wu [12] they present an ear recognition approach, which uses the phase information of Log-Gabor filters for encoing the local structure of the ear. The encoe phase information is store in normalize grey level images. The rank-e performance for the Log-Gabor approaches ranges between 92.06% an 95.93% on a atabase which contains 753 images from 221 subjects. Our work The propose ear recognition system consists of two main parts shown in Figure 1. The first part, ear segmentation, inclues three steps which are the skin etection, morphological operations an ear extraction. The secon part, ear recognition, inclues the feature extraction steps (Wavelet approximation plus PCA) an training the neural network classifier. Each part of these steps is escribe in the following sections. Ear segmentation part The input of this step is the sie face images which will be segmente using the likelihoo skin etector an the morphological operations. Skin etection In this stage, each pixel of sie face image is stuie iniviually to be classifie to either skin or non-skin pixels epening on the likelihoo ratio. We use a skin etector evelope by Ciarán Ó Conaire. [6] that calculates the most likely skin pixels base on a previously compute skin moel. Their non-parametric histogram-base moels were traine using manually-annotate skin pixels (14,985,845 pixels) an non-skin pixels (304,844,751 pixels). Image pixels are classifie accoring to the value P(Skin C) which represents the probability that the pixel whose color is C belongs to the skin region. Because the computations of all probabilities are not possible, the Bayes rule of computing probability of observing skin, given c color P(Skin C) is given as follows [8]: ( ) P Skin C P ( C Skin) P ( Skin) ( ) ( ) + ( ) ( ) = P C Skin P Skin P C N Skin P N Skin After computing the posterior probability P(Skin C), it must be compare with θ which is efine as follows: 0 θ 1 If this probability is greater than θ, the pixel is consiere as a skin pixel; on other han, the pixel is labele as non-skin pixel. After obtaining the likelihoo images, these images are transforme to the binary format. Ear etection This stage uses the morphological operations to process the resultant images from the previous stage which contain many unesire white an black points which must be elete. To achieve this, two morphological operations are one. The first is an operation of filling holes which is performe to remove the black points insie the white regions. This operation prouces an assemble white region representing the ear area. However; if the resultant image still contains some noisy white points on the borers, it will be remove by clearing boarers operation. The secon operation is the erosion operation which is one to ecrease the white areas in the image to obtain the right ear region. Figure 2 illustrates the etaile skin an ear etection phases explaine previously. Ear extraction The propose approach of obtaining the ear region epens on cropping this region from the original sie face image. So, it nees the start point coorinates (xmin, ymin), the with, an the height of the croppe rectangle which will be obtaine from the original image by one of two propose ways. The first way is experimental one, which requires vertical scanning for the first white pixel in the image. Then, the with an height are efine experimentally because the istance between sie face an camera is almost cm an the ear region mostly resies at the left angle of the image. The secon way, the measurement-base way, fins four basic points illustrate in Figure 3A. First, an up-own scanning is one to ientify the start point (first white pixel), an the en point (last white pixel). Secon, a scanning Figure 1. Block iagram of the propose ear recognition system. Figure 2. Ear etection phases. A: original image, B: likelihoo image of image (A), C: binary image of image (B), D: fille holes image of image (C), E: (D) image after erosion, F: output image (image with colore ear region)

3 Figure 3. Ear extraction: (A) The measurement metho (B) The output image. from left to right is performe to ientify the first assistant point (first white pixel) an the secon assistant point (last white point). The with an height can be efine by means of simple subtraction. For more unerstaning, the following example is mentione: Suppose that 1st point is: s(m1,n1); 2n point is: a1(m2,n2); 3r point a2(m3,n3); 4th point e(m4,n4), then the cropping rectangle will have the upper-left point (n2,m1), n3-n2 with an m4-m1 height. Figure 3 illustrates the way of efining the rectangular area which will contain the ear region base on the measurement-base metho. Ear recognition part This step epens on the output ear images of the previous part. These images are use as input to the feature extraction step. Feature extraction Feature extraction of the ear images is very significant stage to obtain the most important information of original images in orer to reuce the next step s processing time. There are ifferent algorithms which can be use to extract features, but the propose approach suggests using wavelet transform to obtain the approximation coefficients (ca) which will be normalize to match the [0-1] form. This normalize (ca) image will store as (.jpg) image, an this will make its gray values in the range [0-255]. Next, principle component analysis algorithm is use to get the final feature vector of each image. Wavelet 2D transform: The wavelet transform concentrates the energy of the image into a small number of wavelet coefficients. The importance of this step is to minimize the size of the original ear images, an extract the most important values of them. In this stuy, the 2D wavelet transform of level 2 is use to ecompose the ear image into coefficients. Then, the approximation coefficient (ca) is normalize by iviing its values by maximum of (ca). This process insures that the maximum value of this matrix will be 1, an the pixel s range is [0-1]. In the next step, the normalize image is store as JPG format, an the gray scale range becomes [0-255]. At the en of this step, the image s size will be almost four times smaller than the original size. Using two imensional wavelet transforms, an image f(x,y) can be represente as follows [4]: f( xy, ) = SJφJ ( xy, ) + v v jϕj ( xy, ) + h h jϕj ( xy, ) + jϕj ( xy, ) = Sj + v Dj + h Dj + Dj (3) J J J J J J j j j j j j = 1 = 1 = 1 = 1 = 1 = 1 Where the two imensional wavelets are the tensor prouct of the one imensional wavelets as below [4]: (, ) ( ) ( ) ( xy, ) ( x) ( y) φj xy = φ x φ y (4) v ϕ = φ ϕ (5) ( xy, ) ( x) ( y) ( xy, ) ( x) ( y) h ϕ = ϕ φ (6) ϕ = ϕ ϕ (7) Where J represents the number of wavelet levels. The first stage is calle the approximation coefficients, where the image s energy is concentrate. The other components are calle the etaile coefficients which are the horizontal, vertical an iagonal images (ch,cv,cd). Principle component analysis: After the ear images have been transforme an normalize in the previous stage, a principle component analysis (PCA) is performe to extract the final feature vectors. PCA is a technique for reucing the imension of feature vectors while preserving the variation in the ataset. A low imension space calle Eigen space, which is efine by a set of Eigen vectors of ataset is use in classification. In ear biometric, the eigenvalues an eigenvectors are compute for the set of training images, an an ear space is selecte base on the eigenvectors. After computing the eigenvectors an eigenvalues of each approximate image, the eigenvectors correspons to the eigenvalues that exceees the threshol (0.5e+008) are selecte, while the others are eliminate. This process will prouce the last feature vectors. Figure 4A shows feature vectors of two ifferent ear images relate to iniviuals who are twins. The plot shows the intersection between the two vectors. Although twins have similar ears, the vector features of them are little ifferent. So, it can be sai that the propose feature extraction metho are effective even in the case of twins images. On the other han, Figure4.B illustrates the variance between feature vectors of two ifferent ear images relate to non-relative iniviuals. Back propagation neural networks The propose metho suggests using fee forwaring back propagation neural networks FFBPNN as a classifier. The FFBPNN propose topology is illustrate in Figure 5. The chosen FFBPNN has one hien layer consiste of 1000 neurons, 200 neurons output layer, ( tansig, logsig ) functions for the hien an output layer, an trainscg as learning function as shown in Figure 5. Experiments an results Experiments have been performe to evaluate the efficiency an robustness of the system using our ear atabase. The atabase contains Figure 4. Feature vectors of two images: (A): relative iniviuals (B): non-relative iniviuals.

4 460 sie face images corresponing to 55 persons of age between 12 an 60. All the illumination variations, pose variations, ay variations an even night-ay variations are taken in account uring imaging. Ear segmentation tests The system succeee in segmenting ear image correctly from 445 sie face images, but it faile in 15 images, an this achieve 96.73% rate. Figure 6 illustrates examples of the sie face images succeee in segmentation phase. It can be notice from Figure 6 that images have ifferent illumination an pose variations. Some images suffer from egraation Figure 5. Neuron Moel. ue to the low level of brightness or occlusion by hair an earrings. Variable istances from camera an various sie faces skin color are taken in account. In Figure 7, there are four images which have ba ear area because of occlusion of the most ear area (A), occlusion an ba imaging (B), ba pose from camera (C,D). Ear recognition tests 200 samples (4 to each iniviual) of correctly segmente ear images were chosen to buil the ear atabase an train FFBP classifier. The system achieve 98.9% recognition rate when it was teste on 170 ear images which on t belong to the ear atabase. Here are some of the ear images which recognize correctly. Figure 8 inclues ifferent ear images which have ifferent illumination an poses. Some of them are covere by hair or earrings, the others suffer from egraation ue to camera motion or to ba imaging. All of these images are classifie correctly. On the other han, Figure 9 illustrates the four samples which recognize incorrectly, an this is because of big egraation that cause by occlusion of the most ear area (A, B), an ba segmente ear (C, D). Determining the best network parameters: Experiments were performe to etermine the best FFBP network to use. Figure10 introuces two ifferent network performance analyses escribing the importance of choosing the appropriate network parameters (training function, layer function, number of layers, an number of neurons etc.). (A) tansig, purelin, traing, (B) use tansig, logsig, trainscg In Figure 10A, it can be notice that the performance can t match the goal (0.02) ue to the selective layer an training (learning) functions which are tansig, purelin, traing respectively. In contrast, the performance is very goo an matchs the goal at the 24th epoch when we use tansig, logsig, trainscg functions in Figure 10B, while it Figure 6. Neural Networks Moel. Figure 8. Examples of correctly segmente samples. Figure 7. The propose back propagation network topology. Figure 9. Example of faile samples.

5 spens 1000 epochs in Figure 10A with lower performance. Another parameter which plays such important role in network performance is the number of neurons in each layer. Figure 11 explains the network s performance accoring to number of neurons. Figure 11A shows network s performance accoring to [ ] neurons of the hien an output layer respectively. It can be notice that this performance is very similar to another one in Figure 11B where network has [ ] form. The last option of 500 neuron of the hien layer requires the least number of epochs, an the network s error rops fast, Figure 11C. (A) 2000 for 1st layer, 200 for 2n one (B) 1000 for 1st layer, 200 for 2n one (C) 500 for 1st layer, 10 for 2n one Table 1 inclues the recognition rates of ifferent FFBP neural networks. The networks iffer in number of epochs, layers, neurons an training time. Accoring to previous analysis, the FFBPNN with one hien layer consiste of 1000 neurons, 200 neurons output layer, ( tansig, logsig ) functions for the hien an output layer, an trainscg as learning function is the best back propagation topology to use Figure 12 inclues the performance of the two selecte FFBP networks accoring to ifferent numbers of epochs. It can be conclue that the secon choice, the blue curve, ( tansig, logsig ) is better than the first, the re one, ( tansig, purelin ). So, sections 4.2.2, an will be introuces in the term of the this selective topology. Determining the wavelet s level: To etermine the best wavelet s level, some of the experiments were one, an the back propagation classifier was evaluate by ifferent feature vectors which have been extracte uner ifferent wavelet s levels. From Table 2, it can be conclue that approximation components (ca) of the secon level give the best recognition rate. So, the level 2 was selecte. Choosing the appropriate threshol: Choosing the best threshol is very important problem to make system capable of refusing the strange iniviuals an accept the system s persons. The number of samples that faile ue to the threshol constraint must be efine, an this will help to get the best threshol. To make a best selection, 20 ifferent samples of stranger iniviuals which on t belong to the system are introuce to test the system s rejection performance. Table 3 escribes the number of faile samples accoring to each threshol value. Table 3 shows that the best threshol value with least number of false positives is 0.008, but at the same time, it can be notice that the performance ecrease to 94.32%. Figure 13 inclues the false acceptance an rejection. Choosing the appropriate number of elements of feature vector: We suggest using the first 30th elements of feature vector, the first 90th an the first 150th ones to test system instea of using the hall feature vectors. Table 4 inclues the results of using those choices. It can be conclue that using less features may reuce the training time but le to lower performance. For more accurate analysis, we compute the mean square error (MSE) of each test sample introuce to the Table 1. Experimental results for ifferent FFBPNN. Epochs Layers Neurons Training Time Rate % [ ] 5: [ ] 3: [500 10] 0: [ ] 0: [ ] 2: Figure 10. Examples of correctly recognize ear images. Table 2. The recognition rate accoring to ifferent wavelet s levels. Wavelet level Feature extraction time (minutes) FFN epochs Rate % Figure 11. The incorrectly recognize ear images. Figure 12. Performance analysis of ffbpnn accoring to layers an training functions. Table 3: Experimental results of changing threshol values. Threshol False Rejection False Acceptance Rate %

6 Figure 13. Performance analysis of ffbpnn accoring to number of neurons. Figure 14. Comparative of ifferent propose FFBP networks rate. Figure 16. False acceptance (re) an False rejection (blue). Table 4. Experimental results of ifferent number of features. No. features Training Time NO. true samples Rate % 30 2: : : All (300) 3: Table 5: Experimental results of ifferent classifier performance. Figure 15. Comparative of ifferent propose neural networks rate. FFBPNN of the three cases (30th,90th,150th) of feature vector. The result are shown in Figures 14,15 an 16 respectively. If we set the threshol value on 0.01, then the number of false rejecte samples on chart MSE1 will be 8. However this number will be 1 if the threshol becomes In MSE2 chart; if the threshol is 0.01, then the number of false rejecte samples is 5, an becomes 1 if the threshol is In MSE3 chart; if the threshol value is 0.05, then the number of Net Type epochs neurons Training Time Rate % FFBP 1000 [ ] 5: FFBP 524 [ ] 3: FFBP 255 [500 10] 0: SOM : SOM : SOM : LVQ :13: LVQ :11: LVQ :32: K-NN rejecte samples is 2, while this number becomes 1 if the threshol set on 0.1. Determine the best classifier Selecting the best classifier epens on many things such as recognition rate, training time, classifier parameters etc. In the

7 Table 6: Comparative stuy. Researcher Database Size Use Approach Weak points Rate A. Ianarelli [10], images Manual Ear measurement Very limite an manual - Burge an Burger [1], Neighborhoo graph from Voronoi iagrams the eges etecte from ear image can be very ifferent even in presence Not mentione of relatively small changes in camera-to-ear orientation or lighting Moreno [13], images (6 images for 28 iniviuals) Yuizono [18], images for 110 iniviuals Compression Networks(Neural Network) Genetic search Does not Take in account illumination an pose variations Does not Take in account occlusion by hair or earning Does not Take in account illumination an pose variations Does not Take in account occlusion by hair or earning Victor [16], images PCA Does not Take in account occlusion by hair or earning Ear 40% Face 80% Chang [3], image training sets PCA EIGEN-FACES AND EIGEN-EARS Does not Take in account occlusion by hair or earning Ear 71.6% Face 70.5% Both 90.9% Choras [5], images geometrical approaches Does not Take in account occlusion by hair or earning 100% S. M. S. Islam [11], 200 images from 100 AaBoost, Algorithm, Iterative Closest Point Although system takes in account illumination an pose variations, 93% 2008 iniviuals (ICP) The ICP algorithm can't eal with big pose variations. Depening on prepare atabase. S. A. Daramola, O. D. Oluwaninyo, [7], images from 350 iniviuals Haar wavelet transform, Back propagation neural network Does not Take in account illumination an pose variations Does not Take in account occlusion by hair or earning Kumar, [12] images Phase encoing with Log Gabor lters % Our work 445 images Likelihoo skin etector an morphological operation, Hybri PCA-Wavelets Features, FFBPNN (for reviewer) 98.90% 93% 100% 98% following, a simple comparative is introuce between ifferent classifiers use in our system in orer to specify the best classification metho (Table 5). It can be sai that the best classification metho is the fee forwaring back propagation neural networks. SOM has a goo clustering effectiveness ue to its competitive single layer where the similar neurons clustere together, but it still require long training time. LVQ networks have a linear layer (as output) in aition to the competitive one, this will look goo for our system, but the result shows that LVQ gives low rates. Nearest neighbor classifier achieve very goo performance an can be use instea of neural network to avoi the long training time. However, the FFBPNN has better rate than K-nn. Regaring to the neural networks, Figure17 illustrates how performance iffers using three ifferent networks which are the FFBPNN, SOM an LVQ. Comparative stuy We compare our stuy with similar ones in the area of ear recognition. Table 6 illustrates the main ifferences between our system an recently ear recognition ones. Conclusion In this paper, a full human recognition base ear images was introuce. The ear images were obtaine by means of likelihoo skin etector an morphological operations. The features were extracte using PCA-wavelet algorithm, then a FFBPNN was traine by these features. The experiments were one on 370 test images an the system achieve 98.9% recognition rate. References 1. Burge M, Burger W (1998) Ear biometrics, In Biometrics: Personal Ientification in Networke Society. In: Jain A (E s.,) Kluwer Acaemic Publishers. 2. Bustar JD, Nixon MS (2010) Towar Unconstraine Ear Recognition from Two- Dimensional Images, Ieee Transactions On Systems, Man, an Cybernetics-Part A: Systems an Humans 40: Chang K, Bowyer KW, Sarkar S, Victor B (2003) Comparison an Combination of Ear an Face Machine Image in Appearance-Base Biometrics, IEEE Transaction on pattern Analysis an machine Intelligence 25: Chi-Fa Chen (2003) Combination of PCA an Wavelet Transforms for Face Recognition on 2.5D Images, Palmerston North: Choras M (2005) Ear Biometrics Base on Geometrical Feature Extraction, Electronic Letters on Computer Vision an Image Analysis. pp: Conaire CO, O Connor NE, Smeaton AF (2007) Detector aaptation by maximising agreement between inepenent ata sources, In CVPR. IEEE Computer Society. 7. Daramola SA, Oluwaninyo OD (2011) Automatic Ear Recognition System using Back Propagation Neural Network, International Journal of Vieo & Image Processing an Network Security IJVIPNS-IJENS, 11: Elgammal A, Muang C, Hu D (2009) Skin Detection - a Short Tutorial, Encyclopeia of Biometrics by Springer-Verlag Berlin Heielberg. 9. Hurley DJ, Nixon MS, Carter JN (2005) Force fiel feature extraction for ear biometrics, Computer Vision an Image Unerstaning 98: Iannarelli A (1989) Ear Ientification, Forensic Ientification Series, Paramount Publishing Company, Fremont, California. 11. Islam SMS, Bennamoun M, Mian AS, Davies R (2008) A Fully Automatic Approach for Human Recognition from Profile Images Using 2D an 3D Ear Data, Proceeings of 3DPVT 08 - the Fourth International Symposium on 3D Data Processing 20: Kumar A, Wu C (2012) Automate human ientification using ear imaging. Pattern Recognition: Moreno B, Sanchez A, Velez JF (1999) On the Use of Outer Ear Images for Personal Ientification in Security Applications, IEEE 33r Annual International Carnahan Conference on Security Technology: Nanni L, Brahnam S (2000) A Genetic Algorithm for Creating a Set of Color Spaces for Ear Authentication, Int l Conf. IP, Comp. Vision, an Pattern Recognition IPCV : Rahman M, Islam R, Bhuiyan NI, Ahme B, Islam A (2007) Person Ientification Using Ear Biometrics, International Journal of the Computer, the Internet an Management 15: Victor B, Bowyer KW, Sarkar S (2002) An Evaluation of Face an Ear Biometrics Proc. Int l Conf. Pattern Recognition:

8 17. Xie Z, Mu Z (2008) Ear Recognition Using LLE an IDLLE Algorithm, ICIC LNCS 1: Yuizono T, Wang Y, Satoh K, Nakayama S (2002) Stuy on Iniviual Recognition for Ear Images by using Genetic Local Search, In Proc. Of the 2002 Congress on Evolutionary Computation: Copyright: 2016 Mayya AM. This is an open-access article istribute uner the terms of the Creative Commons Attribution License, which permits unrestricte use, istribution, an reprouction in any meium, provie the original author an source are creite.