Volume 7, Issue 5, May 2017 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Hand Gestures Recognition for ATM Machine Interface Using k-nn Algorithm Prof. P. S. Sadaphule, Amol Khadse, Shivendu Amale, Vikram Singh Chanda Department of Computer Engineering, All India Shree Shivaji Memorial Society s Institute of Information Technology, Pune, Maharashtra, India DOI: 10.23956/ijarcsse/SV7I5/0235 Abstract In Hand gesture recognition technology, a camera reads the movements of the human body and communicates the data to a computer that uses the gestures as input to control devices. The approach consists of identifying hand pixels in each frame, extract features and use those features to recognize a specific hand pose. In this paper we are using k-nearest neighbor algorithm and the idea here is to select k-nearest input of the certain input from a training database and then assign it to the output that cast a majority vote among the one associated to the selected inputs. In order to obtain always the majority vote the k-parameter is usually the odd number since even can cause a tie in case of two-class classification problem. The results obtained proved that the k-nn had a very good performance and that the feature selection and data preparation is an important phase in the all process. Gesture recognition can be seen as a way for computers to begin to understand human body language, thus building a richer bridge between machines and humans. Gesture recognition enables humans to interface with the machine and interact naturally without any mechanical devices. Keywords Binary Linked Object, Binary silhouette, Fingertip Detection, Hand Gesture Recognition, k-nn algorithm. I. INTRODUCTION Body language is an important way of communication among humans. People perform various gestures in their daily lives. Gestures can help us save more in less time. Thus gesture recognition could be used for improving human-machine interaction. Gesture recognition is an active research area in machine learning and gaming interface. Vision based gestures recognition has been attracting more attention due to no hardware requirement except camera, which is very suitable for ubiquitous computing and emerging applications. Among all gestures that we perform, hand gesture plays an important role. Thus in this project we focused on hand gestures recognition. Interactive system based on gesture recognition needs real-time implementation to work with acceptable performance. Our main goal is to provide a complete system, which can efficiently deal with three important sub-problems i. e. isolate recognition, gesture verification, and real-time gesture spotting. Our approach features a simple, flexible design and targets high-computational efficiency and recognition accuracy, for real-time applications. It is based K- nearest neighbor (k-nn), which significantly reduces processing time with negligible accuracy drop. II. DIFFERENT APPROACH In the literature of gesture recognition, there are two important definitions need to be cleared: hand posture and hand gestures. A hand posture is defined solely by the static hand configuration and hand location without any movements involved. A hand gesture refers to a sequence of hand postures connected by continuous motions (global hand motion and local finger motion) over a short time span. A hand gesture is a composite action constructed by a series of hand postures that act as transition states. For hand postures, the repeatability is usually poor due to the high degree of freedom of the hand as well as the difficulty of duplicating the same working environment such as the background and the lighting condition. To solve the problem, we use a statistical approach based on a set of Haar-like features that focus more on the information within a certain area of the image rather than each single pixel. To improve the classification accuracy and achieve the real-time performance, we employ the AdaBoost (Adaptive Boost) learning algorithm that can adaptively select the best features in each step and combine them into a strong classifier. The training algorithm based on AdaBoost learning algorithm takes a set of "positive" samples, which contain the object of interest (in our case: hand postures), and a set of "negative" samples, i. e. images do not contain objects of interest. Neural networks can be used to determine the gesture of hands. The starting point using Neural Network approach is to create database with all the images that would be used for training and testing. Two operations were carried out in all of the images. They were converted to grayscale and the background was made uniform. Histograms are created of the images and edges are found. The algorithm has been developed using supervised learning. As the inputs are applied to the network, the network outputs are compared to the targets. The learning rule is then used to adjust the weights and 2017, IJARCSSE All Rights Reserved Page 326
biases of the network in order to move the network outputs closer to the targets. Neural networks have already proven efficient for classification, have been used and lead to very satisfying results: the rate of successful recognition can reach 95%. III. COLOR DETECTION AND FILTRATION Color detection and filtration module is used to identify an individual which is wrapped in the finger. The colored marker in the finger will detect and identify each color in the finger from group of RGB colors. In this proposed system, black projection technique is used to separate the color. Colors are detected and then it is filtered to find the position of the fingers accordingly. The following algorithm has been designed to detect the color in the module. A. Algorithm 1. Detect the video by capturing the frames. 2. Separate the individual colors from the group of RGB colors in the frames. 3. Get the range of the colors and set the threshold values. 4. For each pixel a. Compute distance between detected pixels (D) and the threshold value(v) b. If D < V Accept current pixels else reject 5. Convert the sequence of frames to gray scale except detected pixels. 6. Repeat steps 2 to 5 until the color is detected. 7. Filter and locate the range of color in XY coordinates. 8. End IV. FINGERTIP DETECTION The fingertip detection algorithm includes four steps. First of all a camera capture a real time video of moving hand in front of system and hand segmentation is done based on skin filter. In second step wrist end is detected, based on histogram of skin pixels and after this performs hand cropping using different parameters in current image frame of video. In the third step, Hand Cropping is carried out where noisy pixels are removed. Finally fingertips will be detected in the cropped hand image, which is a continuous process for different image frames in the video. A. Skin Filter The skin filter is used on the current input image frame of video. The skin filters are used to create a binary image with background in black color and the hand region in white. In the next step the binary image need to be smoothed using the averaging filter. There can be many errors in the output image of skin filter step because of wrong pixel detection or some skin pixels in the background of hand. To remove these errors, the biggest BLOB (Binary Linked Object) is considered as the hand and rest the background. The biggest BLOB represents hand coordinates in 1 and 0 to the background. The filtered out hand is after removing all errors. The only limitation of this filter is that the BLOB for hand should be the biggest one. B. Wrist end detection After scan of image, we choose the maximum value of on pixels coming out of all scanned ( 1 in the binary silhouette). It was noted that maximum value of on pixels represents the wrist end and opposite end of this scan would represent the finger end. C. Hand cropping Hand cropping minimizes the number of pixels to be taken into account for processing which leads to minimization of computation time. In the next step Histogram would be generated from the binary silhouette of the image. It was observed from the histogram that at the point where the wrist ends. As starting point of image where inclination is found, and then the points correspond to the first on pixel scanning from other three sides are found, which gives the coordinates where the image should be cropped. D. Fingertip Detection Now in the cropped hand image, fingertips will be figured out. Again scanning the cropped binary image and calculate the number of pixels for each row or column based on the hand direction in up-down or left right. Then intensity values for each pixel are assigned from 1 to 255 in increasing order from wrist to finger end by proportionality. So, each on pixel on the edges of the fingers would be assigned a high intensity value of 255. Now detection of the edge of the fingers is done by just detecting pixels having, intensity of 255. V. K-NN ALGORITHM The function of the k Nearest Neighbors (k-nn) algorithm is to use a database in which the data points are separated into several separate classes to predict the classification of a new sample point. Data points are separated using detection of fingertips whose pixel intensity is high. The way in which the algorithm decides which of the points from the training set are similar enough to be considered when choosing the class to predict for a new observation is to pick the k closest 2017, IJARCSSE All Rights Reserved Page 327
data points to the new observation, and to take the most common class among these. This is why it is called the k Nearest Neighbors algorithm. A. Algorithm The algorithm can be summarized as: 1. A positive integer k is specified, along with a new sample. 2. We select the k entries in our database which are closest to the new sample. 3. We find the most common classification of these entries. 4. This is the classification we give to the new sample. VI. IMPLEMENTATION In our paper, we have implemented an ATM machine which takes input from user in form of hand gesture. The idea is to provide ease of access for users. The system accepts input from the user by hand gesture, recognizes the gesture and returns the respective output. In our implementation we have used three hand gestures where user moves his finger forward to deposit money in his account, moves his finger to right to withdraw money from the account and moves his finger to left to issue a mini statement. Fig. 1 Initial condition where system detects the finger. This is considered HALT condition where system waits for the gesture. In Fig. 1, user moves his finger to the center of the screen. The finger is detected by the system and converted in grayscale to map the contrast of the finger to the background. The density of the background noise is kept adjustable and also the width of the finger can be changed. Fig. 2. Condition where system detects the finger moved forward. This is considered AMOUNT DEPOSIT condition where system allows the user to deposit money. 2017, IJARCSSE All Rights Reserved Page 328
In Fig. 2, the user moves his finger forward to make a gesture to deposit money. In this system, we have maintained a circle of active pixels. These are the set of points which recognizes the gesture. As the user moves his finger forward his finger goes beyond the circle and we can recognize hand gesture. Fig. 3. Condition where system detects the finger moved towards right. This is considered WITHDRAW AMOUNT condition where system allows the user to deposit money. In Fig. 3, the user moves his finger towards right to make a gesture to deposit money. The camera, as it faces opposite the user, recognizes the gesture as finger moved towards our left. As the user moves his finger towards right finger goes beyond the circle towards our left and we can recognize the hand gesture. Fig. 4. Condition where system detects the finger moved towards left. This is considered MINI STATEMENT condition where system displays last ten transactions user performed. In Fig. 4, the user moves his finger towards left to make a gesture to deposit money. The camera, as it faces opposite the user, recognizes the gesture as finger moved towards our right. As the user moves his finger towards right finger goes beyond the circle towards our right and we can recognize the hand gesture. As soon as the gesture is recognized, a mini statement popup is displayed on the screen. Thus, in this system we can recognize three gestures from the user. The user has to only move his finger to either side to carry out his transactions. The circle is the state space where points are mapped of the finger. Nearest neighbor of these points classifies the points to intended gesture. VII. CONCLUSION The importance of gesture recognition lies in building efficient human machine interaction. Its applications range from sign language recognition through medical rehabilitation to virtual reality. The major tools for this purpose include 2017, IJARCSSE All Rights Reserved Page 329
HMMs, particle filtering and condensation algorithm, FSMs, and KNNs. A hybridization of KNNs and FSMs can increase the reliability and accuracy of gesture recognition systems. The dynamic movement of hand has been modeled by HMMs and FSMs. The similarity of a test hand shape may be determined with respect to prototype hand contours, using fuzzy sets. Similarity-based matching of the retrieved images may be performed on clusters of databases, using concepts from approximate reasoning, searching, and learning. Thus, gesture recognition promises wide-ranging applications in fields from photojournalism through medical technology to bio metrics. REFERENCES [1] T. Hninn and H. Maung, "Real-Time Hand Tracking and Gesture Recognition System Using Neural Networks," International journal of Computer, Electrical, Automation, Control and Information Engineering, vol. 3, no. 2, pp. 315 319, 2009. [2] A. Chaudhary, K. Das, J. L. Raheja, and S. Raheja, "Intelligent Approaches to interact with Machines using Hand Gesture Recognition in Natural way: A Survey,"International Journal of Computer Science & Engineering Survey (IJCSES), vol. 2, no. 1, pp.122 133, Feb. 2011. [3] Q.Chen, N. Georganas, and E. Petriu, "Real-time Vision-based Hand Gesture Recognition Using Haar-like Features," Instrumentation and Measurement Technology Conference - IMTC 2007 Warsaw, Poland, pp. 1 6, May 2007. [4] G. Simion, V. Gui, and M. Otesteanu, "Vision Based Hand Gesture Recognition: A Review," INTERNATIONAL JOURNAL OF CIRCUITS, SYSTEMS AND SIGNAL PROCESSING, vol. 6, no. 4, pp. 275 282, 2012. [5] R. Elakkiya, K. Selvamani, S. Kanimozhi, R. Velumadhava, and A. Kannan, "Intelligent system for human computer interface using hand gesture recognition,"procedia Engineering, vol. 38, pp. 3180 3191, 2012. [6] J.Liu, L. Zhong, J. Wickramasuriya, and V. Vasudevan,"UWave:Accelerometer-based personalized gesture recognition and its applications,"pervasive and Mobile Computing, vol. 5, no. 6, pp. 657 675, Dec. 2009. [7] J.L. Raheja, K. Das, and A. Chaudhary, "An Efficient Real Time Method of Fingertip Detection," International Conference on Trends in Industrial Measurements and Automation TIMA 2011, pp. 447 450, 2011. [8] Y. Fang, K. Wang, J. Cheng, and H. Lu, "A REAL-TIME HAND GESTURE RECOGNITION METHOD," IEEE ICME, pp. 995 998, 2007. 2017, IJARCSSE All Rights Reserved Page 330