HAND GESTURE RECOGNITION

Transcription

1 HAND GESTURE RECOGNITION Azeem Hafeez1, Naima Munir2, Rabia Naheed3,Shireen Khan4, Arfa Saif5 1,2,3,4,5 Electrical Engineering Department, FAST - NUCES. 852-Block B, Faisal Town, Lahore, Pakistan 1 azeem.hafeezz@gmail.com 2 naima_munir_9@live.com 3 rabia_naheed@hotmail.co.uk 4 shireeeen_khan@gmail.com 5 arfa.saif92@gmail.com Abstract: With the advancement in technology, the applications of hand gesture recognition are increasing enormously. In this paper, a method of detection and recognition of hand gestures is presented. Hand gestures are detected by the Ellipse detection method and recognized by the normalized correlation method. The presented algorithm has five steps: Firstly the palm was detected using the Ellipse Detection Method, and then the Region of Interest (ROI) was extracted so that only the fingers of the hand remained. Next, the gesture was shifted in the image so that the orientations of the database image and the test image becomes same. Finally, the gesture was recognized based on the normalized correlation method. This method was successfully implemented and tested in MATLAB. The success rate of the Ellipse detection algorithm was approximately 96 percent and the success rate of Normalized Correlation Method was approximately 70.1 percent. Keywords: Hand gesture recognition, Ellipse Detection, Image Shifting, Normalized Correlation, Covariance, Image cropping. 1. Introduction Gesture recognition is an active region of research in the field of Computer Vision and Image Processing. The basic motivation of implementing this gesture recognition system was to serve the humanity and make interaction between normal human beings and speech challenged people convenient. We used a very simple and efficient algorithm to implement this system. There are numerous applications of this system. This system can be used for interaction between humans and machines, for e.g. to control the home appliances without making much effort, just by the simple movement of hands. Different methods of hand gesture recognition have already been proposed and implemented. But those methods required additional hardware such as the data glove or the colour markers. This hand recognition system required just bare hands as the input media but the distance of hands from the sensing device should be same for all the gestures. The algorithm has five steps: Firstly the gesture was detected using the Ellipse Detection Method, and then the ROI was extracted from the image so that only the image of fingers remained to minimize the background effect. Next, the gesture was shifted in the image so that the orientations of the database image and the test image become same and finally the gesture was recognized based on the normalized correlation method. This method was implemented in MATLAB. We tested our system on static gestures and the results were very accurate. Ellipse Detection algorithm was approximately 96 percent accurate. Normalized correlation was 70.1 percent accurate. The details of implementation of algorithms and the result analysis are described in detail in the following sections. 2. Materials and methods This section describes in detail the implementation and the steps of the different algorithms used in this hand recognition system. The basic flow chart is shown below:

2 2.1 Ellipse Detection Method Figure1. Flow Chart of the Hand Recognition System The main steps that were followed are as follows: 1. Normalizing the R and G components of the image of a palm. 2. Plotting an ellipse of these components. 3. Setting a standard ellipse. 4. Categorizing all the pixels that fall within the ellipse as the pixels representing skin These steps are explained as follows: 1) Normalizing R and G components of an image: As every color in an image is a combination of three primary colors which are Red, Blue and Green, using this property of an image we normalized the R and G component of every pixel to get the values within a certain range. The following formula was used for normalization. 2) Plotting an ellipse of r and g r=r/(r+g+b) g=g/(r+g+b) A graph was plotted of r against g. The points formed an ellipse as shown in the figure below. The equation used to plot an ellipse of r and g components is as follows: rg=((x-x 0 ) /a) 2 + ((y-y 0 ) /b) 2 Here, x 0 = 0.32, y 0 =0.4, a=0.02 and b=0.06.these values were assigned to the respective variables after various tests on images of different palm samples. 3) Setting a standard ellipse: Figure2. Plotting an ellipse of r and g

3 This ellipse was taken as the standard ellipse and this algorithm was tested with images of different palms. It was seen that most of the palm data fell within the ellipse. 4) Categorizing all the pixels that fall within the ellipse as the pixels representing skin: This algorithm was further tested on images containing pixels representing skin color and other colors too. The pixels representing the skin were not changed and the rest of the image was turned black as shown in the figure. Ellipse Detection was tested on 126 images and the tests were successful. The success rate of this algorithm is 96%. Time Complexity is a drawback of this algorithm. An image and its resultant image after ellipse detection are shown below: Figure3. Original image and its Ellipse Detected image. 2.2 Cropping the image After the hand gesture was detected using the ellipse detection method, the image was cropped so that only the fingers remain and also the background is removed, all other skin pixels except the palm were removed. This was done to increase the efficiency of normalized correlation method. Firstly, the sum of pixel rows and columns was computed. If the sum of pixel values of a row exceeded the threshold value the starting row was known, and if the sum of pixel values of a column exceeded the threshold value the starting column of the gesture was known. Considering the starting row and column of the gesture, the original pixel values were copied to a new 2D-array of dimension 330x250. In this way the image was cropped to increase the efficiency of the normalized correlation method. The following figures show the ellipse detected gesture and the cropped gesture. Figure4. Ellipse Detected images after cropping.

4 This algorithm was also tested on 126 images and 114 images were cropped successfully. Thus the success rate of this algorithm was 90 percent. 2.3 Shifting the Gestures When the image was cropped, the cropped gesture was aligned and shifted to the left so that each gesture had the same orientation. This step was also done to increase the efficiency of the normalized correlation method. This algorithm was also tested on 126 images, it yielded good desired results. 2.4 Normalized Correlation Method Figure5. Cropped Images after Shifting Correlation is the basic operation used in this algorithm. This a simple but very useful operation. In this algorithm, the shifted images were used as an input, the dimensions of each image was calculated. The images were converted from 2D to 1D. The size of all the images was made same by appending zeroes. These 1D arrays of gesture were then correlated. After performing tests on various images the threshold value was determined. The images that had correlation close to 1 were of the same gesture and the images whose correlation was less than 0.85 were of different gesture. Taking into consideration the correlation coefficient the gestures were recognized. The formula used for correlation is as follows: Crr=Σxy/[ΣN( ΣxΣy)] Here x, y are the random variables, N is the total number of random variables. Crr for the above two images: Figure6. Set1 of images for testing of Normalized Correlation Method Crr of the above two images: Figure7. Set2 of images for testing Normalized Correlation Method.

5 It was a very efficient algorithm. This algorithm was tested on 126 images and 96 images were recognized correctly. Thus the success rate of this algorithm is 70.1 percent. 3. Results and Discussion After having performed different experiments on 126 images the success rate of various algorithms was calculated as tabulated below: S.No Algorithm Success Rate 1. Ellipse Detection 96.0% 2. Image Cropping 87.2% 3. Image Shifting 95.3% 4. Normalized Correlation Method 70.1% Table1. Analysis of algorithms used for Hand Recognition System The results of Ellipse Detection are most accurate. The results of image cropping and image shifting are also quite good. The success rate of Normalized Correlation Method is 70.1% because the slight orientation or angle difference in the database image and the test image of the same gesture resulted in correlation value less than 0.85, so the gestures were not recognized as same. But the success rate of Normalized Correlation Method is also good enough. This algorithm of hand gesture recognition is very simple and the efficient in time. 4. Conclusion It is a very simple and efficient algorithm for hand gesture recognition. The success rate of hand gesture detection is 96 percent and the success rate of hand gesture recognition by normalized correlation method is 70.1 percent. For accurate results the background must be uniform and the lighting conditions should be good enough. The background must not be of skin color. Also the hand must not be very close to the sensing device i.e., a camera. The time complexity is a drawback of the normalized correlation method but the simple approach can be easily implemented. The efficiency of this algorithm can be increased by pre-processing the images to further reduce the background effect from the gesture images. 5. References [1] Hand Sign Recognition through Palm Gesture and Movement, Spring Oleg Rumyantsev, Matt Merati, Vasant, Ramachandran,Stanford University,Spring 2012 [2] Hand Gesture Recognition: A Comparative Study -Prateem Chakraborty, Prashant Sarawgi, Ankit Mehrotra, Gaurav Agarwal, Ratika Pradhan, March 2008 [3] HAND GESTURE RECOGNITION Gradient orientation histograms and eigenvectors methods, dramdst,june 2006