Hand Gesture Optimization using Structure from Motion

Transcription

1 Available online at Pancaran Pendidikan FKIP Universitas Jember Vol. 07, Issue, 2, pp, , May, 2018 ISSN X e-issn X Pancaran Pendidikan DOI: /pancaran.v7i2.172 Hand Gesture Optimization using Structure from Motion Muhammad Sonnie Bestian 1, Risanuri Hidayat 1, I Wayan Mustika 1, 1 University Gadjah Mada, Yogyakarta, Indonesia sonnie11.mti13@mail.ugm.ac.id ARTICLE INFO Article History: Received 1 st Maret 2018 Received in revised form 10 th April 2018 Accepted 17 th April 2018 Published online 1 st May 2018 Key Words: Hand gesture, matching, SIFT threshold, structure from motion ABSTRACT Utilization human body with unique way to distinguish between one person and another already widely used in research. Biometric is a method for recognizing humans based on one or more physical characteristics or unique behaviors. Many previous researches of hand gestures used to control devices and identification. In This paper, we discusses the use of structure from motion method for optimization hand gesture. Structure from Motion ( SFM ) is the process to estimating the structure of 3-D shape from the series of 2-D images. The goal is to estimate the pose of the camera calibration results bits of the image to be reconstructed into a 3-D structure, from the scene and the scale factor is not known, then recover the actual scale factor by displaying an object of known size. The results from the trial by reconstructing four (4) drawing hand pieces and matched by detecting a hand gesture (3-D), from the stage of the process generated frame 2 + 3: match rate slightly (0) => relax SIFT threshold to 0.7 to 0 matches => SIFT relax the threshold to 0.8 with 2 matches => SIFT relax the threshold to 0.9 with 18 matches => SIFT relax threshold to 0.95 with 42 matches. From the test results obtained by the degree of fit pieces of the picture (2-D) to the whole image (3-D) resulted in 53,85% accuracy value. Copyright Muhammad, 2018, this is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited INTRODUCTION Several research related with hand gesture entitled "Introduction of Dynamic Hand Gestures Using Neurol Network Back propagation Learning Method", the system was made to recognized of hand gesture. The training process with back propagation

2 174 Pancaran, Vol. 07, No. 2, , May, 2018 method is done with initial image input and normalization result to get the weights on artificial neural network. The identification process is done by calculating the output generated by the input image to be identified as a command to be executed by the computer. [1] [2] Conducted research on the use of hand gestures for physicians to view collection of medical images in a sterile environment. In the study [3] apply hand gesture for videogame controller. [4] carried out research on the use of hand gestures to control remote robots. In research [5] hand gestures replace mechanical devices of the mouse or keyboard. Electronic devices in the car can be controlled using a simple hand gesture [6]. The use of hand gestures for Smart TV systems is examined by [7] which focuses on the introduction of numbers given through hand gestures. From the previous research has not done the research use Structure From Motion to improve the image structure with the object of hand gesture, so it becomes important in this research to do hand gesture optimization research using SFM (Structure From Motion) method single. METHODS Generally, the staps of hand gesture optimization research using SFM (Structure From Motion) method, consists of several steps, there are image capture from webcam, image acquisition, image processing, Structure From Motion. Figure 1 shows the steps of the research. Figure 1. The Steps of Sructure From Motion Webcams are used to capture movements in real time. Camera capture results from real space into image form can be shown in Figure 2. Figure 2. camera coordinat from real form into image form

3 Muhammad et al: Hand Gesture Optimization using Structure from Motion 175 The object used in the research is a hand gesture object, as shown in Figure 3. Figure 3 hand gesture image The object generated from the camera is a 3D object, assumed with variable X. Can be shown in equation 1 [13] Object resulted in 3D form saved into 2D form, where shows in figure 4. Figure 4. 3D object to 2D object And the equaion shows in equation bellow. Structure from motion (SFM) is the process of calculating camera orientation and 3D positioning automatically by analyzing a set of photos. The SFM method is a process for finding point correspondence on stereo images using SIFT feature point extraction. Stages in the SFM method are divided into three parts, there are correspondence search, incremental reconstruction and reconstruction. The first step is correspondence search to find the structure in the input image and identify the projection of the same points in the image structure. The resulting output is a set of verifiable verifiable geometric images with pairs and an image projection graph for each point. At this stage is divided into three stages, 3.1 Correspondence search. When the camera captures the object in question, a matrix as shown in equation 4 [13].

4 176 Pancaran, Vol. 07, No. 2, , May, 2018 Where : X = object 3D x = object 2D P = parameter projection matric camera K = intrinsic [R I t]= extrinsic For taking two objects with the same camera is produced as shown in equation 5 [13] Where : X = object 3D x = object 2D K = intrinsic = extrinsic The display for the three camera objects is shown in Figure 3.5 Previously published in book multiview Geometry in computer vicion Figure 5. Object from same camera position Source : Richard hartly and Andrew Zisserman At the time of object retrieval in the real field, the captured object camera consists of several types of objects, where each object has different location and location

5 Muhammad et al: Hand Gesture Optimization using Structure from Motion 177 Incremental reconstruction The initialization of points (points) that are connected to each object in sequence in accordance with the number of objects identified. From the initialization results will be found points between the appropriate objects and objects that are not appropriate. Reconstruction The resulting graph serves as the basis for the reconstruction phase, the model carefully selected between the two reconstruction view, before gradually registering the new image, triangulation forming points, outlier filtering, and perfecting the reconstruction using the Bundle Adjustment (BA). RESULTS AND DISCUSSION Image Aquitotion In the process of image acquisition conducted data retrieval for handgasture test object. The process in the early stages of the system invokes the camera menu (webcam) used and chooses the camera type, shown in Figure 6 Figure 6. Image acquisition from camera Capture the hand gesture use capture menu in aplication to get the mage of handgesture, and The result is shown in Figure 7 Figure 7. Camera Capture Results Just crop handgesture to reduce the noise of the image, Performed the ROI process as shown in Figure 8.

6 178 Pancaran, Vol. 07, No. 2, , May, 2018 Figure 8. ROI Results Repeat the capture process in 3 times and in different position from the right side, middle, and left side, with the same camera type and resolution shown in Figure 9 Figure 9. HandGesture image from 3 position SFM process After the next acquisition process carried out the process SFM (Structure From Motion), the first process produced is shown in Figure 10

7 Muhammad et al: Hand Gesture Optimization using Structure from Motion 179 Figure 10. Correspondence Search Process Matching the image of object 1 to object 2 produced the following results: Frame 1 + 2: too few matching (0) => relax SIFT threhsold to 0.7 with 5 matching => relax SIFT threshold to 0.9 with 12 matching => relax SIFT threhsold to 0.95 with 29 Matching Figure 11. The result of extraction process features object 1 and object 2 The initialization results are as follows: 12 inliers / 29 SIFT matches = 41.38% Remove 6 outliers outof 12 points with reprojection error bigger than pixels Remove 3 outliers outof 6 points with reprojection error bigger than pixels The process is continued with the 2nd object and the 3rd object, the result shown in Figure 12.

8 180 Pancaran, Vol. 07, No. 2, , May, 2018 Figure 12. Object 2 + Object 3 Matching the object 2 with object 3 is generated by the following result: Frame 2 + 3: too few matching (1) => relax SIFT threshold to 0.7 with 2 matching => relax SIFT threhsold to 0.8 with 6 matching => relax SIFT threhsold to 0.95 with 26 Matching The result of the process is shown in Figure 13 Figure 13. The result of extraction process features object 2 and object 3 The initialization results are as follows: 14 inliers / 26 SIFT matches = 53.85%

9 Muhammad et al: Hand Gesture Optimization using Structure from Motion 181 Remove 12 outliers outof 14 points with reprojection error bigger than pixels Remove 2 outliers outof 2 points with reprojection error bigger than pixels CONCLUSION In this research, the way to optimize the quality of objects with the same camera and lighting levels. The results showed that the calibration of cameras performed while taking the object can affect the accuracy of the results. Data retrieval in different times with the same moving object affects the results of reconstruction and accuracy results. REFERENCES Lukito, Y. dan Harjoko,A., 2013, Pengenalan Hand Gesture Dinamis Menggunakan JST Metode Pembelajaran Backpropagation, Makalah Seminasik Wachs, J., Stern, H., Edan, Y., Gillam, M., Feied, C., Smith, M. dan Handler, J., 2007, Real-Time Hand Gesture Interface for Browsing Medical Images, J. Intel. Comp. Med., 1, Manresa, C., Varona, J., Mas, R. dan Perales, F.J., 1999, Real-Time Hand Tracking and Gesture Recognition for Human-Computer Interaction, J. Electronic Letters on Computer Vision and Image Analysis, 2, 1-7. Tara, R. Y., Centroid Distance Fourier Description for Shape-Based Hand Gesture Recognition In Mobile Robot Teleoperation, Tesis, Magister Teknologi Informasi, Universitas Gadjah Mada, Yogyakarta. Kurata, T., Okuma, T., Kourogi, M. dan Sakaue, K., 2001, The Hand Mouse: GMM Hand-color Classification and Mean Shift Tracking, Prosiding IEEE ICCV Workshop on RATFGRTS, Vancouver. Zobl, M., Geiger, M., Schuller, B., Lang, M. dan Rigoll, G., 2003, A Realtime System for Hand Gesture Controlled Operation of In-Car Devices, Prosiding International Conference on Multimedia and Expo (ICME 03), Baltimore. Wu, H., Chen, X. dan Li, G., 2012, Simultaneous Tracking and Recognition of Dynamic Digit Gestures for Smart TV Systems, Prosiding Fourth International Conference on Digital Home, Guangzhou. Rachmat,A., Wirayuda A.B., dan Sulliyo Sistem Identifikasi Biometrik Ruas Jari Tangan Manusia Menggunakan Metode Principal Component Analysis (PCA) dan Learning Vector Quantification (LVQ), Jurnal Fakultas Teknik Informatika, Universitas Telkom. Munir, Rinaldi, 2004, Pengolahan Citra Digital dengan Pendekatan Algoritmik Bandung : INFORMATIKA Basuki, A., Palandi, J.F., Fatchurrohman, Pengolahan Citra Digital Menggunakan Visual Basic, Penerbit Graha Ilmu, Gonzales, R.C., and Wood, R.E., Digital Image Processing, Addison-Wesley Co, 2008.

10 182 Pancaran, Vol. 07, No. 2, , May, 2018 Prasetyo, E., 2011, Pengolahan Citra Digital dan Aplikasinya menggunakan Matlab. Yogyakarta: Andi Offset. Hartley,R. and Zisserman,A.,(2003), Multiple View Geometry in Computer Vision, Secon Edition, UK: Cambridge University Press. Sayyid,A.R., Sanjaya,M,. and Satya, Y.P., 2015, Kontrol Mobil Robot Menggunakan Hand Gesture Recognition Dengan Metode Adaptive Neuro-Fuzzy Inference System (ANFIS), Alhazen Journal of Physics, Volume 2, Nomor 1, Issue 1, Juli ISSN