A Study of Movement Attitude Capture System Based on Kinect

Transcription

1 A Study of Movement Atttude Capture System Based on Knect Abstract Hongka Zhou Pngdngshan Unversty, Pngdngshan467000, Chna In ths paper, we propose a human-robot vsual percepton algorthm for humanod robot that mproves the accuracy of human moton data captured usng Knect as a vsual nput devce. Frst, the captured jont dsplacement nformaton s transformed nto angle nformaton by the nverse knematcs equaton. Then, based on the change of angular velocty and angular acceleraton, the long tme movement s automatcally dvded nto ndependent segments, and the hgh precson angular trajectory s estmated by the prncple of correlaton vector machne. Fnally, the algorthm s evaluated by usng the spatal smlarty, the tme smlarty and the smoothness of the angular trajectory. The algorthm s valdated on the AO robot platform. Expermental results show that ths algorthm effectvely mproves the spatal-temporal smlarty and track smoothness of moton percepton, whch lays the foundaton for hgh-precson moton smulaton. Keywords:Human-computer nteracton, Vsual percepton, Knect, Inverse knematcs, Correlaton Vector Machne. 1. ITRODUCTIO Acton smulaton s an mportant part of human-robot nteracton, and the realstc premse of realstc smulaton s the good capture and percepton of human actons. Teachng method s a tradtonal method of moton percepton, manly through the way of hand-handle, accordng to the acton of the real robot to put the correspondng acton and record (Klan et al.,1996). Ths method lacks the flexblty to generate blunt movements. ow the manstream method s to use the vsual percepton devce to capture the acton, so the generated acton s more realstc, whch makes the robot more expressve and frendly to people. Early research focused on the use of wearable vsual capture devces to capture human moton, such as the VICO optcal moton capture system. Wearable devces wth hgh precson, large amount of data, the captured data can be used drectly for robot moton smulaton research(sanna et al.,013; Shao and L, 013). However, the wearable devces are expensve, and the scene layout and preparaton are complcated. Users are nconvenent to use, whch s dffcult to be extended to practcal applcatons of human-computer nteracton. In recent years, non-wearable human body moton capture technology has matured, such as Mcrosoft's Knect somatosensory magng, Asus's XtonPRO, etc., they do not requre users to wear any devce can contnuously capture human moton, and low prces, s the human moton capture feld Drecton of development. However, due to the lmtaton of collecton prncple, moton obstructon and nterference of complcated envronment, the accuracy s low and the amount of captured data s also greatly reduced. Therefore, the captured raw data must be processed. In 015, Stone and Skubcused the mastermotormap (MMM) method to match the jont data of human jonts to the robot jonts, and used the Levenberg-Marquardt algorthm to optmze jont poston(stone and Skubc, 015). Lu normalzed the body to a vector that overcomes the problem of dfferent proportons of the robot and human body structure, and then uses the nverse knematcs method to mtate the acton(lu,016; Azary and Savaks, 013). In 014, Rosado et al. Kokmproved the accuracy of Knect's captured moton from both statc and pose calbraton(kok et al.,014). Statc calbraton was used to obtan the statc dstance between adjacent jonts and to correct jont poston n gesture calbraton. Thobb and other dynamc tme warpng (DTW) and weghted average method to acheve multple repettve movements of the tme calbraton, the fnal fuson to generate a comprehensve trajectory of the acton(han et al.,013). L Used Adaboost algorthm to construct the human body component detector, and screened the 10

2 canddate regons accordng to the conformty of components, foreground coverage and entropy, component relevance, bone densty and center degree, and obtaned the accurate poston of human body components(l et al.,008). The above methods are drectly on the dsplacement data constrants and optmzaton. The dsplacement s determned by the angle of each jont, the fnal drve robot to complete the acton s also the angle value, so the angle data s the real need to optmze the goal (Hu et al., 014). In ths paper, we frstly use the knematc data captured by Knect to compute the jont angle nformaton through the nverse knematcs equaton. Then, we propose an algorthm that combnes the automatc segmentaton of moton segments and the relevant vector machne (RVM) to mprove the perceved acton Accuracy. In addton, a varety of evaluaton methods are gven, and the results of dfferent algorthms are unformly evaluated and compared from the aspects of tme consstency, spatal consstency and smoothness.. THEORETICAL PREPARATIO.1 Knect technology Knect s a somatosensory perpherals product developed by Mcrosoft for the Xbox360. An nfrared laser projecton lens s used to project a set of nfrared laser dots to players. The other two CMOS cameras perform 3D scannng n X, Y and Z coordnates,through the bult-n patented algorthm, n order to dstngush the player, the background, and the player's acton ntentons, the precse spatal postonng performance for a wder range of games and nteroperablty. Knect can get three knds of nformaton at a tme, ncludng the normal camera color mage, 3D nfrared reflecton depth data, the sound sgnal. A total of 3 lenses on the Knect machne, s located n the mddle of the ordnary RGB color camera, the left and rght lens are respectvely nfrared emtter and nfrared CMOS camera formed by the 3D depth sensor, Knect manly by the 3D depth sensor to detect the player's acton. Central RGB color camera s manly used to dentfy the user dentty, the use of human face or body features to do recognton. In addton to ths, t's also possble to apply games on Augmented Realty as well as vdeo capabltes. And wth the base of the motor trackng mode, follow the target object automatcally rotate the lens poston, automatcally fnd the most sutable pcture of the central focus poston. Fgure 1.Knect system structure Processng technology, the depth of nformaton manly from the nfrared transmtter and nfrared camera recever, to determne the dstance of the target object. The 3D depth nformaton technology Mcrosoft uses comes from workng wth prme sense. Prme Sense offers moton detecton and detecton chp PS1080 and patented lght codng. But unlke tradtonal structured lght methods, t emts not only a set of perodcally changng two-dmensonal mages encodng, but also "volumetrc encodng" wth three-dmensonal depth data. Ths type of lght source, called a laser speckle, s a randomly dffracted speckle formed when a laser s rradated on a rough object or through a frosted glass. These speckles have a hgh degree of randomness and wll transform the pattern as the dstance vares, meanng that the speckle pattern at any two ponts n space s dfferent. The CMOS nfrared sensor collects each pont n the camera's feld of vew space accordng to the set 11

3 dstance reference plane, and perceves the surroundngs by means of the reflected black-and-whte spectrum: pure black represents nfnty and pure whte represents nfnty. The gray area between the black and whte corresponds to the physcal dstance from the object to the sensor, whch s then calculated by layerng the peaks and nterpolatng to form a depth mage representng the surroundng envronment. The sensor generates a depth of feld mage stream at 30 frames /s, completely reproducng the surroundngs n real tme. Fgure.Knect data processng. RVM prncple and trajectory regresson estmaton algorthm The correlaton vector machne s a sparse probablty model proposed by tppng n Bayesan framework. The gven sets of nput vector sets X x1, x,..., x and the target vector set H h1, h,..., h satsfy the followng mappng relatonshp: Where w w w w h y( x, w ), 1,,..., (1) 1,,..., s the set of weght vectors to be sought, y( x, w ) s the sum of the set of nput vectors and the set of weght vectors, { 1,,..., } are nose dsturbances, obeyng the Gaussan dstrbuton wth mean 0 and varance. Therefore, from (1) can be ntroduced: p h x h y x () ( ) ( ( ), ) In order to make the sample lnearly separable, the concept of kernel functon s ntroduced n RVM. Defne the kernel functon: Then yx ( ) can be wrtten as a weghted sum of ( x) : ( x ) K( x ; x ) (3) m m m m 0 (4) m 1 y( x ) w ( x ) w 1

4 Let [ ( x1), ( x),..., ( x )] T be an ( 1) dmensonal matrx.where ( x1) [1, K( x1, x ),..., (, )] T K x x, w [ w0, w1, w,..., w ] T are ( 1) 1 dmensonal augmented vector. y [ y ( x ), ( ),..., ( 1 y x y x )]T, then from (4) we can get: y w (5) Combnng Eqs. () and (5), the lkelhood functon of tranng set s: 1 w (6) / (, ) ( ) exp( h ) p h w Accordng to the prncple of RVM, a moton estmaton algorthm based on RVM s gven. The estmaton steps are as follows: In the moton segment, take the collected tme data as the nput vector set x (needs to be normalzed) of the RVM, and the correspondng angle value s the target vector set θ; Radal bass functon RBF ( ( x ) exp( x x ) s used as the kernel functon of RVM, and the m n m n parameter of RBF s optmzed by usng 50 fold cross-valdaton method, Mean square error (MSE). The RVM regresson model s establshed usng the optmal parameter * and the jont angle * at a gven tmng s estmated. Combned wth the automatc segmentng of the segments, the jont angle of the upper extremty can be obtaned. The RVM trajectory (19s ~ 9s) of the left arm wth four degrees of freedom. It can be seen that RVM regresson effectvely elmnates most of the fluctuatons and abrupt changes n the orgnal captured data, and the angular changes are consstent and stable. 3. MATHEMATICAL MODEL OF HUMA MOVEMET Tradtonal moton-sensng algorthms are ether not segmented or segmented manually. When the whole exercse has both strenuous exercse and slow exercse, no strenuous calculaton burden wll be ncurred. However, the manual method usually only recognzes a very obvous pause, and t s dffcult to dentfy the stagng pont of contnuous exercse. In ths secton, we use the characterstcs of jont angular velocty and angular acceleraton to propose a method of segmentng moton segments automatcally. Ths method can reduce the manual workload and mprove the computatonal effcency. Then use the correlaton vector machne to buld the regresson model and generate the angle trajectory. Assumng a total of Knect recorded a total of sets of upper lmb jont poston data, samplng tme t ( 1,,..., ). Usng nverse knematcs to calculate the jont angle tmng value, the angular velocty and angular acceleraton tmng values can be obtaned by dvdng the adjacent numercal dfference and the tme dfference. Accordng to the physologcal structure of the human jont, an angular velocty threshold T and an angular acceleraton threshold T & are set, and the angular velocty and the angular acceleraton are lower than the threshold, so as to form the canddate segmentaton pont set T C. Takng the left arm for example, set the angular velocty of each degree of freedom at t tme respectvely Ptch ( t) Roll ( t) Yaw ( t) Roll ( t).the LS LS LE LE angular acceleratons are & Ptch ( t) & Roll ( t) & Yaw ( t) & Roll ( t), respectvely. Then the set of segmentaton ponts C LS LS LE LE T can be determned accordng to the followng formula: Ptch Roll Yaw Roll T ={t ( t) ( t) ( t) ( t) all smaller thant and C LS LS LE LE Ptch Roll Yaw Roll & ( t) & ( t) & ( t) & ( t) all smaller thant &}, 1,,..., LS LS LE LE (7) 13

5 There are many moments n the T C s adjacent, where the frst two ponts of the adjacent moment are reserved. The range between them s defned as the transton segment, and the range between any two transton segments s defned as the moton segment. For each moton segment, a correlaton vector machne s used to generate the angle trajectory, whle the moton changes n the transton segment are relatvely small. The angles n each jont use the mean of the angles n the transton segment to mprove the computatonal effcency. The result of automatc segmentaton of the left shoulder ptch degree of freedom s shown n Fgure 3, t can be seen that the algorthm of ths method automatcally dvdes the acton of about 1 mn nto 16 segments. 4. EVALUATIO AD VERIFICATIO Fgure 3. Transton and moton epsodes Most humanod robots and human bengs do not have a 1: 1 sze rato. If you drectly compare the postons of the human body and the robotc jont, the coordnates need to be scaled. If the proportons of the lmbs of the robot are dfferent from those of real people and requre more complcated transformatons, the atttude of the robot devates from the real person even f the postons of the dstal jonts are matched. Because humanod robots mostly correspond to real people n the desgn of degrees of freedom, t s a more reasonable choce to use jont angles as the object of smlarty evaluaton. The smlarty evaluaton consders the smlarty between two scales of space and tme. The smlarty of spatal smlarty s the angle value at the same tme. Tme smlarty consders the contnuty of the acton, and the speed can reflect the change of the acton. Therefore, the angular velocty s chosen as the evaluaton object of the tme smlarty here. The data n Table 1 shows that the orgnal raw data of the untreated source and the real value of the spatal smlarty, but the worst tme smlarty. The temporal and spatal smlartes between trajectores and real values obtaned by ths method are comparable to those of SVM, and are more smlar to Kalman flterng methods. Based on the results at before, we can see that the comprehensve advantages of ths method are greater. Table 1 Comparson of spatal and temporal consstenceof angle trajectores among several methods method orgnal Kalmanflter SVM RVM S 91.06% 87.44% 89.1% 91.37% S U T S U 80.33% 83.1% 90.55% 9.97% 14

6 Fgure 4. RVM estmaton result and Knect data 5. COCLUSIOS Compared to wearable devces, Knect's data s less accurate and has less data. Ths paper presents an automatc segmentaton and RVM-based trajectory estmaton algorthm that mproves the accuracy of Knect vson-aware human moton data to enable good human moton percepton wth less expensve non-wearable devces. Tme consstency, space consstency and smoothness evaluaton results, as well as n the AO robot platform to acheve the acton to prove the effectveness of the method.the next step wll be to use the moton-sensng algorthm n ths paper to study the balance constrants, structural constrants, and moton predctons that arse when mtatng more and more complex motons. REFERECES AzaryS., & SavaksA. (013). Grassmannan sparse representatons and moton depth surfaces for 3d acton recognton, 13(4), Han J., Shao L., XuD.,ShottonJ. (013). Enhanced computer vson wth mcrosoft knect sensor: A revew, IEEE transactons on cybernetcs, 43(5), HuR. M., HeZ. D., & BaF. (014). The research of 3d human moton smulaton and vdeo analyss system mplemented n sports tranng. Advanced Materals Research, , Klan J., Haala., EnglchM. (1996). Capture and evaluaton of arborne laser scanner data, Internatonal Archves of Photogrammetry and Remote Sensng, 31, KokM., HolJ.D., SchönT.B. (014). An optmzaton-based approach to human body moton capture usng nertal sensors, IFAC Proceedngs Volumes, 47(3), LZ., DngZ., YuanF. (008). Close-range optcal measurement of arcraft's 3D atttude and accuracy evaluaton, Chnese Optcs Letters, 6(8), Lu, Y. (016). A Study on Vdeo of Moton Atttude for 3-D Human Computer Interacton, Internatonal Journal of Smulaton--Systems, Scence & Technology, 17(36). SannaA., LambertF., ParavatG.,Rocha F.D.(013). A knect-based nterface to anmate vrtual characters, Journal on Multmodal User Interfaces, 7(4), Stone E.E., Skubc M. (015). Fall detecton n homes of older adults usng the Mcrosoft Knect, IEEE journal of bomedcal and health nformatcs, 19(1), ShaoZ., & LY. F. (013). A new descrptor for multple 3d moton trajectores recognton