Computer Aided Drafting, Design and Manufacturing Volume 25, Number 3, September 2015, Page 10

Transcription

1 Computer Aided Drafting, Deign and Manufacturing Volume 5, umber 3, September 015, Page 10 CADDM Reearch of atural Geture Recognition and Interactive Technology Compatible with YCbCr and SV Color Space YE Wen-yu, FEG Kai-ping, LUO a, PA Yang Guangdong Univerity of Technology, Guangzhou igher Education Mega Center, Guangzhou , China. Abtract: In view of the current geture recognition algorithm baed on kin color egmentation i not flexible and ha weak reitance to the environment, thi paper put forward a new method of kin color modeling to improve the adaptability of geture egmentation when it face to different tate. The modeling built by double color pace intead of only one i compatible both in YCbCr and SV color pace to training the Gauian model which can update the threhold value for binarization. Finally, thi paper deigned a natural geture recognition and interactive ytem baed on the double color pace model. It ha hown that the ytem ha a good interactive experience in different environment. Key word: human-machine interaction; geture recognition; kin color egmentation; feature extraction 1 Introduction atural geture recognition refer to ingle image or multiple action equence of the hand without wearing any ening device recorded by camera, the target geture contour i then extracted out from the captured image equence and identify the claification of the geture characteritic. It ha broad application propect [1-3]. The direction of development of the next generation of humancomputer interaction technology will be that the machine a a multi-function perceptron, receive from the viion-baed body movement to replace the traditional keyboard, moue and other device to achieve freedom, mart, efficient, divere, peopleoriented and interactive experience between human and computer [4-6]. Uing ingle camera and image proceing algorithm to realize geture recognition i the mot popular method currently. It i uing kin egmentation algorithm to eparate the hand area, however one of the bigget challenge in thi method i the interference which come from different environment. To olve thi problem, a lot of reearche optimize the formula in kin color model uch a elliptical boundary model and twodimenional Gauian ditribution [7-8], but the promotion i limited epecially in complex environment. Baed on kin color egmentation of natural geture recognition technology generally i only applied in one ingle color pace, whoe primary raion i the recognition rate depending on the urrounding environment, uch a lighting condition, background color, the pixel of the camera, geture and camera ditance, jitter, cla color interference and o on. To olve thi problem, thi paper propoe a color modeling method which i compatible both YCbCr and SV color pace. Thi method can obtain real-time color ample taken by the uer, to find the initial value under the kin probability value generated by the Gauian model in each color pace, and the formation mechanim of an adaptive threhold to find out the bet threhold, and finally combine of the image proceing algorithm for color egmentation. With thi method, the paper develop a natural geture recognition, interactive ytem to enhance the adaptability to different environment and improve the efficiency of the geture recognition. Sytem Architecture and Modular Diviion Thi paper recognize the input, and explore the human-machine interactive application for identifying geture, and add the ytem command repone to the identified input, the ytem flow chart i hown in Fig. 1. Correponding author: FEG Kai-ping, Male, Profeor, fengkp@gdut.edu.cn.

2 YE Wen-yu et al., Reearch of atural Geture Recognition and Interactive Technology Compatible with 11 Uer Input Geture Trigger Command Webcam capture hand geture Recognition Claification Fig. 1. Image PreProceing Contour Matching Sytem flow chart. Skin Color Segmentation Motion Tracking In Fig. 1, the geture input wa input to trigger command erie of proceing and eventually reached the experience of computer interaction effect with geture, the ytem i a typical Pipe and Filter ytem. Thi paper tudie the image egmentation, photography, the color of kin track, identification and claification, the feature extraction. The ytem i divided into different module according to the proce. The reult of the previou module wa tranferred to the next module. The input data of ytem i geture video recorded by camera. According to the difference of tatic and dynamic geture, the output i divided into two categorie: 1 For tatic geture, the output i the template of matching number correponding to the input geture; For dynamic geture, different geture trigger different command according to different application. To optimize the overall tructure, enhance the independence of each module and reduce redundant data, the tructure framework of ytem i divided into three part, which i hown in Fig.. Through uch diviion, each module can be independent ealed, and in thi way the reuability of module will be improved. The eparation of recognition and interactive i ueful in the development of new interactive application. Video Capture Data Collection Module Image PreProceing atural and Geture Recognition Sytem Geture Recognition Sytem Skin Segmentation Adaptive Threhold Motion Tracking Adaptive Motion Contour Extraction Filter Template Matching uman Computer Interaction Module Drawing, Moue Simulating, Picture Browing Fig.. Sytem tructure diagram. 3 Key Technology 3.1 Adaptive Skin Model In order to extract geture outline from the image, the geture area of the image hould be located firtly. In order to adapt to the recognition of tatic and dynamic geture, thi paper tudie the kin color egmentation method to extract kin area from the image. Baed on the reearch of RGB, comparing the egmentation effect of YCbCr and SV color pace, the kin model compatible with the YCbCr and SV i developed to ditinguih the background and geture, and provide the foundation of decribing the characteritic of geture. (1) YCbCr color pace The human eye i more enitive to brightne than color, o many color pace eparated brightne and color component, YCbCr i one of them. It Y channel denote brightne component, Cb channel denote blue color, Cr channel denote red color, and it i widely ued in digital televiion, video decoding application; the ditribution of kin color including black, white, yellow of three different race in thi color pace i more focued. The probability ditribution of kin i hown in Fig. 3. The horizontal axi repreent Cb component, the vertical axi repreent Cr component [9-10].

3 1 Computer Aided Drafting, Deign and Manufacturing (CADDM), Vol.5, o.3, Sep. 015 μ E(k) E( k )( k ) T According to the calculation of the μ and fitting of Gauian model i hown in Fig. 5., (a) Black (b) White (c) Yellow Fig. 3. (d) Synthei of 3 kind Skin probability ditribution. Fig. 3(d) i convered to three dimenion pace, which i hown in Fig. 4. Fig. 5. () SV color model Gauian model. SV color pace contain hue, aturation, brightne. SV model i repreented by ix ide vertebral. SV color model divided the aturation and brightne component, and wa uitable for color egmentation algorithm, and S pace ditribution hitogram are hown in Fig. 6 and Fig. 7 repectively [11]. Fig. 4. Three-dimenion color ditribution. Fig. 6. pace hitogram. According to the ditribution feature of the above, it i known that ditribution of the color of kin Cb and Cr component i Gauian ditribution. Suppoed that hue of color i k=[cb,cr] T, the probability denity function of kin can be calculated according to mean vector and covariance, and etablih kin model for: 1 1 exp ( k ] r( k ) p( k / kin) (1) Fig. 7. S pace hitogram.

4 YE Wen-yu et al., Reearch of atural Geture Recognition and Interactive Technology Compatible with 13 Fig. 6 and Fig. 7 how and S hitogram all together in mall interval, the interval of i [134,158], and the interval S i [19,48]. The both hitogram are the ingle-peak, o then can be fitted by the ingle-peak gauian model, formula a M (, μ, μ ) () where μ μ 1 i i0 1 S i i0 S S S i0 i0 S ( ( S S S i i μ μ ) i0 ) ( S μ )( μ ). i S i By above model, the color probability of kin can be get a Eq. 3. P(, S) exp 0.5( S, ) T S S 1 ( S, ) (3) The algorithm of adaptive threhold value method can be repreented a following: the kin probability value generated by Gauian model i et up a the initial threhold, with initial threhold to tet the target area, and a unit i reduced every time. Then judge the area ize of kin color detected by the current threhold, if the color of kin area i increaing, record color of kin incremental and threhold value at thi time, if the color of kin area i reducing, record all the kin after incremental and correponded threhold from big to mall according to color of kin by incremental ort, when the incremental i maximum, the threhold correponding i the bet [1-15]. Skin model flow chart i hown in Fig. 8. Skin Sample Image Set Skin Threhold Separate YCbCr Channel Separate SV Channel Reduce Skin Threhold Y Cb Cr S V Train Gau Model M(Cb,Cr) Train Gau Model M(,S) Increae Skin Area Y Reduce Threhold by increation Skin Probability P(Cb,Cr) Skin Probability P(,S) The mallet increation, The optimal Threhold Fig. 8. Skin model. 3. Contour Matching After pre-proceing, color egmentation i applied to capture the target geture. In order to enure real-time repone, thi article ue the twodimenional template matching method. Twodimenional contour of the cutom template, extract target geture contour match with the template outline. The moment of the contour i calculated the outline profile characteritic point, the invariant ditance propoed by u, i defined on the normalized center ditance of even invariant ditance, they have rotation, tranlation, caling invariance [11]. Thi paper tudie the geture and calculating target of the template u moment imilarity;

5 14 Computer Aided Drafting, Deign and Manufacturing (CADDM), Vol.5, o.3, Sep. 015 pecifically need to calculate the geometric moment and ma center, normalized center ditance at the ame moment. Finally calculate both the imilarity of invariant moment, and et threhold to determine the match i ucceful or not. 3.3 Feature Extraction (1) Determine of finger The algorithm of determining whether the target geture i finger or not and the direction of finger toward i hown a follow: Start: calculate the center of gravity of the object contour and the external center of the rectangle. Switch: compared the vertical axi with the threhold. Cae 1. if the vertical axi in the center of gravity i lower than threhold, then there i no finger geture; Cae. if the center of gravity in the center of the bottom indicate that the finger i pointing upward; Cae 3. if the center of gravity in the center of the top indicate that finger i pointing below. () Extraction of fingertip The algorithm to determine the fingertip location idea from the target geture contour i a follow: Step 1. Calculate the contour of the center of ma M(x, y); Step. Calculate the contour at every point of the polar angle, convert each two-dimenional polar coordinate, M(x, y) for the converion center at all point by polar angle and polar radiu ize of the ort; Step 3. Calculate the contour polar coordinate point to the M(x, y) the average of the ditance k; Step 4. The ize of the polar angle in the numerical range [5, 315 ] within the polar radiu of the contour point et to 0, in order to achieve the filtering of the writ and arm; Step 5. The ize of the polar angle le than the average k i the polar radiu of the contour point et to 0, to achieve the palm of your hand i removed and the remaining point of the fingertip region. 4 Interactive Application Thi ytem can match a variety of predefined and uer-defined template. The recognition effect i hown in Fig. 9, the left one how a current profile and matching the template number, the middle one how the ource image, contour extraction and matching template number, and the right how a kin color egmentation effect. Fig. 10 how the drawing interaction effect by our ytem, the left one how a finger to draw the graphic, the middle one how a ource of image and contour fingertip poitioning, the right image how the color egmentation reult. Fig. 9. Geture matching effect of our ytem. Fig. 10. Finger drawing effect of our ytem. Fig. 11 how the application that uing geture to browe the photo through our ytem, the ytem define the fingertip move to left direction to viit the previou picture, and the fingertip move to the right to browe to the next picture. The left of Fig. 11 how the currently diplayed picture and picture number, the middle part how a trajectory of the fingertip poitioning and fingertip movement, the right part how kin color egmentation. 3 Fig. 11. Browe picture effect of our ytem.

6 YE Wen-yu et al., Reearch of atural Geture Recognition and Interactive Technology Compatible with 15 5 Teting and Evaluation In order to ae the effect of the ytem identification, multiple indexe are defined hown in Table 1, the indexe include: Profile = number of the outline of the image color egmentation to extract; Contour hit rate = the number of ucceful match / current contour; Geture hit rate = the number of ucceful match / total number of frame; From Table 1, it i hown that the outline of the ytem hit rate i gradually enhanced from 50% to 94% from the initialization of the ytem to a period of running, the geture hit rate gradually rie from 5% to 85%, howing the contour extraction and template matching efficiency comparion well, adaptive proce to identify the efficiency of teady improvement in compliance with the deign expected. Table 1. Experimental Data Acquiition Current contour / μm Good contour / μm Good contour rate /% Succeful match / μm Good contour match /% Contour hit rate /% Geture hit rate /% Fig. 1. Fig. 13. Effect of YCbCr model. Effect of SV model. Thi paper realize an adaptive color model of the two color pace, and define two mode: predefined threhold and real-time acquiition of color information to generate the threhold, the threhold of the two mode contain the two type of YCbCr pace and SV pace through real-time acquiition, and the color model can improve the adaptability of the ytem in different environment, two color pace egmentation of color in different environment are hown in Fig. 1 and Fig. 13, the background of the two map i more complex, SV kin color egmentation i better than YCbCr, their repective following two imple background but high light, the color egmentation reult the YCbCr uperior to of SV; It can be een that both the pro and con i not abolute, under different circumtance, the effect of the two method differ by real-time contrat, the uer can choice egmentation method for geture recognition and interaction. For the verification ytem of the geture recognition rate, we tet ten predefined geture which predefined by the ytem in different context to elect 10 ample video, and the reult of the tatitical tet are a follow: (1) Background i imple and no color interference can extract a clear outline, a hown in Fig.14. The contour edge are mooth, flawle interference. The

7 16 Computer Aided Drafting, Deign and Manufacturing (CADDM), Vol.5, o.3, Sep. 015 recognition rate i higher; the tatitical reult are hown in Table. geture recognition rate i from 67% to 9%, dynamic geture recognition rate i from 80 percent to 90 percent, with 0.5 to 1 econd delay. The reult of our ytem are atifactory with the deign goal. Fig. 14. Simple background geture recognition reult. Table. Geture template number Simple Background Environment Recognition Rate Statitic (Unit: %) YCbCr recognition rate SV recognition rate Average 90 9 () Complex background, without the interference of kin color and high light, the extraction efficiency i moderate, a hown in Fig.15. The outline of the edge i not mooth, there are minor flaw, YCbCr identification rate i lightly reduced and SV largely decline. The tatitical reult are hown in Table 3. Under the condition with no imilar color, high-light interference and the appropriate ditance from people to camera (0.5 m to.5 m), the tatic Fig. 15. Complex background geture recognition reult of our ytem. Under the condition with no imilar color, high-light interference and the appropriate ditance from people to camera (0.5 to.5 m), the tatic geture recognition rate i from 67% to 9%, dynamic geture recognition rate i from 80 percent to 90 percent, with 0.5 to 1 econd delay. The reult of our ytem are atifactory with the deign goal. Table 3. Geture template number Complex Background Environment Recognition Rate Statitic ( %) YCbCr recognition rate Average SV recognition rate Table 4. Recognition Rate Statitic Between Popular Algorithm in Different Environment (%) Algorithm Low illumination Backlit Complex background igh peed Adaptive kin egmentation in YUV model Adaptive kin egmentation in SV model Subtraction method Kinect ytem Thi paper At lat, thi reearch how the accuracy of recognition in popular algorithm under different tate in Table 4. It how that thi paper i more accurate than ingle color model in all condition below, but lower than Kinect Sytem for the infrared reflection module. Uing ingle camera to reduce the experience gap which come from Kinect mut be the next objective in thi reearch.

8 YE Wen-yu et al., Reearch of atural Geture Recognition and Interactive Technology Compatible with 17 6 Concluion Thi article focued on OpenCV development librarie, our method uing a ingle camera i compatible with dual-color pace of tatic geture and dynamic geture recognition and interaction. The main function include: Thi paper analyzed the comparion of YCbCr and SV color pace, combined of Gauian model to etimate the initial color value, repectively. baed on adaptive threhold in YCbCr and SV color to realize egmentation model; through real-time adaptive threhold mechanim enhanced adaptability of our ytem to the environment; The experiment how that the model algorithm i imple, fat real-time egmentation of any reolution of the camera; in interaction deign, thi paper give different output for tatic geture and dynamic geture. For tatic geture, after the ucce of matching, the ytem will output the template of number; with dynamic geture, finger drawing and browe the picture can be realized. The experiment how that the interaction effect of the tatic geture recognition rate and dynamic geture to achieve the deired goal. The ytem can quickly extracted kin color from a imple background, but with complicated background the color egmentation i not o atifactory. The focu of future reearch i to improve the ytem adaptability, tability, and the three-dimenional cene interaction. Reference [1] Lv Y. Camera Geture Interaction Sytem [D] Qingdao: Qingdao Univerity, 009. [] Daggotar F, Sarrafzadeh A, Meom C. Multi-layered hand and face tracking for real-time geture recognition [C]// the 008 International Conference on eural Information Proceing, Auckland, ew Zealand, LCS, 009, 5506: [3] Vafadar M, Behrad A. uman hand geture recognition uing motion orientation hitogram for interaction of handicapped peron witll computer [C]// the 008 International Conference on Image and Signal Proceing, Cherbourg, France, LCS, 008, 5099: [4] Wilkowki A. A MM-baed ytem for real-time geture recognition in movie equence [C]// Proceeding of the 008 Conference on uman Sytem Interaction, Krakow, Poland, 008: [5] Xu J F, Jun S Q, Shi Y W, et al. Implementation of digital chime-bell interaction ytem driven by hand geture [C]// the 008 International Conference on Pervaive Computing and Application, Alexandria, Egypt, 008: [6] akaya Y, akakuki T. A practical approach for recognition of hand geture and ditinction of it ingularity [C]// Automation and Logitic (ICAL), 010 IEEE International Conference on, 010: [7] Lee J Y, Yoo S I. An elliptical boundary model for kin color detection [C]// Proceeding of the 00 International Conference on Imaging Science, Sytem, and Technology, 00. [8] Zhao J L, Chen T D. An approach to dynamic geture recognition for real time interaction [C]// Proceeding of ls009: [9] Szénái S. Ditributed Implementation of Cell uclei Detection Algorithm, Recent Advance in Image, Audio and Signal Proceing [M]. WSEAS Pre, Budapet, 013: [10] Zarit B D, Super B J. Comparion of five color model in kin pixel claification [C]// Proceeding of the International Workhop on Recognition, Analyi, and Tracking of Face and Geture in Real-time Sytem, Corfu, Greece,1999: [11] Jayaram S, Schmugge S. Effect of color pace tranformation, the illuminance component and color modeling on kin detection [C]// Proceeding of the IEEE Computer Society Conference on Computer Viion and Pattem Reeognition,Wahington, DC, USA,004: [1] Wang. Baed on Viual Real-time Tracking and Recognition and Geture in the Application of uman-computer Interaction [D]. angzhou: Zhejiang Univerity, 008. [13] Xia Z Y. uman Skin Diviion Technology Reearch and Application [D]. Changha: Central South Univerity, 009. [14] Tenilion J C, Shirazi M, Fukamachi. Comparative performance of different kin chrominance model and chrominance pace for the automatic detection of human face in color image [C]// Proceeding of Conference of Automatic Face and Geture Recognition, Grenoble, France, 003: [15] Qi S M. Geture Tracking and Recognition Technology in the Sequence of Image [D]. Suzhou: Suzhou Univerity, 008. YE Wen-yu, mater degree candidate for Guangdong Univerity of Technology, the main of reearch field i computer viion and virtual reality. FEG Kai-ping, a profeor and mater tutor for Guangdong Univerity of Technology, the main of reearch field are virtual reality, computer viion, interaction technology and application baed on web and computer animation. fengkp@ gdut.edu.cn. LUO a, mater degree candidate for Guangdong Univerity of Technology, the main of reearch field i computer viion and virtual reality.