Comparative Study of Hand Gesture Recognition Techniques

Transcription

1 Reg. No.: DOI:V2I4P16 Comparative Study of Hand Gesture Recognition Techniques Ann Abraham Babu Information Technology Department University of Mumbai Pillai Institute of Information Technology Sector 16, New Panvel , Navi Mumbai, Maharashtra, India. ABSTRACT Hand gesture recognition is a growing and very vast field of research. It has a wide area of application in human computer interaction and sign language. Compared with the traditional interaction approaches, such as keyboard or mouse, gesture based interaction is more natural and efficient. This paper presents two different techniques to recognize hand gestures. These techniques are based on detection of some shape based features like centroid, orientation of hand, presence of fingers and thumb in image. The hand gesture recognition process is divided into four phases namely image capture phase to capture input image. The second phase is image preprocessing which prepares the image so as to extract features in next phase. The third phase is feature extraction which extracts features that can be used to classify the given gesture. And finally recognition and interpretation of different hand gestures. Keywords: Hand gesture recognition techniques, Human computer interaction, Shape based features. 1. INTRODUCTION Gestures are the physical actions that convey some meaningful information. It is a form of non-verbal communication in which bodily actions are used to communicate important messages, either in place of speech or together in parallel with spoken words. They are a powerful tool of communication among humans & are of particular interest to the deaf & dumb community. If we ignore the world of computers for a while and consider interaction among human beings, we can simply realize that we are utilizing a wide range of gestures in our daily personal communication. Thus the significant use of gestures in our daily life as a mode of interaction motivates the use of gestural interface in modern era. There is a great emphasis on using hand gesture in various applications of computers. Hand gesture represents ideas and actions using different hand shapes, orientation or finger patterns. These hand gestures are given as an input to gesture recognition system. But recognizing hand gestures is really a challenging task as one of main goal of hand gesture recognition is to identify hand gestures and classify them as accurately as possible. Once the gesture is recognized by the system, various actions can be linked to different gestures to perform various events by the machine. The remainder of this paper is organized as follows. Section 2 presents related work. System overview is presented in Section 3. Detailed description of methods and Comparison are presented in section 4 and section 5 respectively. 2. RELATED WORK In early, many gesture recognition techniques have been developed for recognizing and tracking various hand gestures i.e. Instrumented gloves, optical markers and some advanced techniques based on image features, color based, vision based techniques are available for hand gesture recognition, all have their advantages and drawbacks. An instrumented

2 glove contains number of sensors which gives information about hand location, orientation and tips of fingers. These data gloves provide high accuracy results but they are too expensive and required wired connection with system. Optical tracking system based on markers work with infrared (IR) light. The markers project IR light and reflect this light on the screen. These systems require a complex configuration. based techniques requires processing of image textures or color features. If we are working on skin color based detection, there is a wide variation in skin tones from one continent of earth to another [2] [4]. The method [7] described for gesture recognition requires compactness calculation, which is one of the shape based descriptor and it is calculated as the ratio of squared perimeter of the shape to its area. That means if two different hand shapes with same perimeter to area ratio exists, it classify these two different shapes as same, in this way this approach limits the number of gesture pattern that can be classified using these three shape based descriptors and only 10 different patterns have been recognized. The methods discussed in this paper are based on shape based features which can recognize any number of the hand shape based on their orientation and generate different encoding bits. 3. SYSTEM OVERVIEW The components of hand gesture recognition system [3] is shown in figure 1 below 3.2 Pre-processing The main function of this phase is to prepare the image so as to extract the features in the next phases. It is the process of dividing the input image into regions separated by boundaries. 3.3 Feature Extraction This phase finds and extracts features that can be used to classify the given gesture. When the input data to an algorithm is too large to be processed and it is suspected to be redundant (much data, but not much information) then the input data is transformed into a reduced representation set of features (also named features vector). Transforming the input data into the set of features is called feature extraction. If the features extracted are carefully chosen it is expected that the features set will extract the relevant information from the input data in order to perform the desired task using this reduced representation instead of the full size input. 3.4 Hand Gesture Recognition Recognition or classification of hand gestures is the last phase of the recognition system. The classification represents the task of assigning a feature vector or a set of features to some predefined classes in order to recognize the hand gesture. 4. METHODS The Implemented Algorithm for method I and method II is shown in figure 2 below Capture Preprocessing Feature Extraction Input Hand Gesture Recognition Preprocessing Orientation Detection Figure 1: Block diagram of hand gesture recognition system. 3.1 Capture This is the first phase of hand gesture recognition process. The task of this phase is to acquire an image, which is then processed in the next phases. is captured using a camera or web cam. Features Extraction Classification and bits generation Recognize Hand Gestures Figure 2: Flowchart of Algorithm Reg. No.: DOI:V2I4P16Page:2

3 4.1 Method I Preprocessing In this phase, first RGB image is captured. Since RGB color space is sensitive to various light conditions. It is then converted into YCbCr color space [5]. Since luminance component in YCbCr color space creates problem. YCbCr image is converted into binary images [5]. We find boundary contours for locating hand by scanning the image. Perform vertical scan by scanning the image from left to right (top to bottom).the first white pixel encounter is treated as the left side of the hand. Then we start scanning of image from right to left (top to bottom).the first white pixel encounter is set as the right side of the hand. Now we perform a horizontal scan within the vertical boundaries defined earlier from left to right and top to bottom. The first encounter white pixel is set as top of the hand. As the hand extends from the bottommost part of the image, there is no cropping required to locate the end of hand Orientation Detection After locating the hand, we find the edges of hand image by extracting those pixels in the image where pixels value changes rapidly from 0 to 1. After finding the edges of hand in the image, next we find the orientation of hand image to determine whether hand is horizontal or vertical by scanning the edges of hand in the binary image [1] Feature Extraction In this phase, features of hand image are extracted to classify given hand gestures. First centroid [1] of hand image is calculated via image moment. Second feature extracted include peaks (tip) of the fingers [1] using Euclidean distance formula. And the last feature extracted include thumb of the hand image which is determined by counting the number of white pixels in the hand image. If there are less than 0.069% of total white pixels then we assume thumb is present otherwise thumb is not present in the hand image [1] [4] Classification In this phase, we classify the peaks into significant and insignificant peaks using thresholding property. and encode them as 1 and 0 respectively. Leftmost bit in the bit sequence of peaks is considered for thumb. If thumb is present, leftmost bit will be 1 otherwise 0. For example, [01110] will be categorized as three fingertips with no thumb, which represent the finger pattern of the hand gesture [1]. 4.2 Method II Preprocessing segmentation is typically performed to locate the hand object and boundaries in image. The captured RGB image is converted into binary image [5]. The hand segmentation is performed using K- means algorithm. The K-means clustering algorithm is an iterative technique which is used to partition the image into K clusters. The output of K-means clustering algorithm is formation of two clusters. Cluster 1 which represents hand, has all pixel values set as 1 and cluster which represents background has 0 intensity pixels. We find boundary contours for locating hand object by scanning the image. Perform horizontal and vertical scan to find boundary contours of the hand image [2] Orientation Detection The previous approach described in method I for orientation detection was limited by the constraint that the hand object should not touch the corner of the image it should lie somewhere near the middle and it depends on tracing of boundary matrices or edges of hand in binary image. So to overcome from this limitation, method II introduced the current approach for orientation detection of hand in image by computing the length and width of bounding box with an assumption that if the hand is vertical then length of the bounding box is greater than the width of bounding box and if width of bounding box is greater than the length of bounding box then the image contains horizontal hand. We compute the ratio of length to width of the bounding box if it is greater than 0.9 then it is vertical otherwise horizontal. Reg. No.: DOI:V2I4P16Page:3

4 4.2.3 Feature Extraction In this phase, first we calculate centroid [2] of segmented hand image via image moment which is weighted average of pixels intensities. The second feature extracted is thumb [2] which is a significant shape feature to classify various hand gestures. For thumb detection, we calculate bounding box of hand image and divide this box into left side and right side. By taking 30 pixels as a width from each side of the bounding box we crop this bounding box in two regions. So one is the left box represented by green boundary and the other is right box represented by blue boundaries in the image shown below. After getting these two boxes, we count the total number of white pixels present in binary image which also represents the hand object in image. We count the number of white pixels in each box. If less than 6.9% of total white pixels exist in any of the left box or right box, we consider that the thumb is present in that box. If both boxes have less than 6.9% of total number of white pixels exists in image then thumb is not present in any of the box because thumb is only one and it cannot be found at both sides for the same hand shape pattern. And if both boxes having more than 6.9 percent of total white pixels in the image, then thumb is not present in any of the box. The third feature extracted is number of fingers raised which is calculated by computing extrema points [2] Classification Classification of different hand gestures is based on generation of 7 bit binary sequence. First bit of 7 bit sequence represent the type of orientation, if the hand is vertical first bit will be 1 otherwise 0. Second bit is allotted for thumb presence, if thumb is present in the hand gesture then it is set as 1 otherwise 0. Next three bits are assigned for total number of raised fingers in hand such that if 1 finger is raised then it will be represented by its 3 bit binary representation and coded as 001, if two fingers are raised then it will be coded as 010 which is the binary representation of 2. As we know a hand gesture can have maximum four fingers raised in image so these three bits can have the maximum bit pattern as 100. Last two bits of 7 bit sequence are very important because they differentiate among the hand patterns which have equal number of fingers. These two bits set as 1 or 0 by tracing the location of extreme points of each extrema and centroid [2]. 5. COMPARISON Below table shows comparison between two methods based on various parameters such as segmentation, Orientation Detection, Classification, Recognition Rate and Average Computation Time. Method I II Segmentation Orientation Detection Trace boundary of hand object by scanning image Depend on values of x- boundary and y- boundary Classification Uses 5 bit binary sequence Recognition Rate Average Computation Time 6. CONCLUSION K-means clustering and Trace boundary of hand object by scanning image Depend on ratio of length to width of the bounding box Uses 7 bit binary sequence 92% 94% 2.76 sec 0.60 sec A comparative study is given on two hand gesture recognition techniques based on detection of some shape based features like orientation, centroid, thumb detection, presence of fingers in terms of raised or folded fingers of hand and their respective location in image. Based on this comparison we can conclude that method II which uses K-means clustering for segmentation, gives approximate recognition rate of 94% and takes only fractional part of a second to recognize the hand gesture is computationally Reg. No.: DOI:V2I4P16Page:4

5 efficient as compared to method I which gives approximate recognition rate of 92% with an average computation time of 2.76 second. 7. ACKNOWLEDGEMENTS I thank the Lord Almighty for his grace and blessings which help me to complete this study. I would like to express my sincere thanks to my guide, Head of IT Department Dr. Satishkumar Varma and to all the professors of Department of Information Technology Engineering, Pillai s Institute of Information Technology, New Panvel for their guidance and support throughout. [7] A. Jinda-Apiraksa, W. Pongstiensak, and T. Kondo, Shape-Based Finger Pattern Recognition using Compactness and Radial Distance, The 3rd International Conference on Embedded Systems and Intelligent Technology(ICESIT 2010), Chiang Mai, Thailand, February REFERENCES [1] Meenakshi Panwar and Pawan Singh Mehra, Hand Gesture Recognition for Human Computer Interaction, IEEE International Conference on Information Processing (ICIIP 2011), Waknaghat, India, Nov [2] Meenakshi Panwar, Hand Gesture based Interface for Aiding visually Impaired, IEEE International Conference on Information Processing (ICIIP 2012), Noida, India. [3] Krishnakant C. Mule & Anilkumar N. Holambe, Hand Gesture Recognition using PCA and Histogram Projection, International Journal on Advanced Computer Theory and Engineering (IJACTE),Osmanabad, India. [4] Meenakshi Panwar, Hand Gesture Recognition based on Shape Parameters, International Conference on Computing, Communication and Applications (ICCCA), [5] Creative Cow s egas Color Space compression.ru download articles color space ch03.p df [6] Zaman Khan and Ibraheem Hand Gesture Recognition: A Literature Review International Journal of Artificial Intelligence, July Reg. No.: DOI:V2I4P16Page:5