MOTION RECOVERY BASED ON FEATURE EXTRACTION FROM 2D IMAGES

Transcription

1 MOTION RECOVERY BASED ON FEATURE EXTRACTION FROM 2D IMAGES Jahu Zhao, Lg L 2 ad Kwoh Chee Keog 3,3 School of Computer Egeerg, Nayag Techologcal Uversty, Sgapore, School of Computg, Curt Uversty of Techology, Perth, Australa, 602 Abstract: Key words: Ths paper presets a method for moto recovery from moocular mages cotag huma motos. Image processg techques, such as spatal flter, lear predcto, cross correlato, least square matchg etc, are appled to extract feature pots from 2D huma fgures wth or wthout markers. A 3D skeleto huma model s adopted wth ecoded agular costrats. Eergy Fucto s defed to represet the resduals betwee extracted feature pots ad the correspodg pots resulted from projectg the huma model to the projecto plae. The a procedure for moto recovery s developed, whch makes t feasble to geerate realstc huma amatos Posture Recostructo, Huma Amato, Eergy Fucto. INTRODUCTION There are two basc methods classcal computer amato: kematcs ad dyamcs approaches. The dsadvatage of these methods s ther ablty to fltrate erroeous movemets. If real mages cotag huma motos are used to drve the vrtual huma body, more fathful motos ad varatos of dyamc scees ca be geerated the vrtual world. Ths uderstadg leads us to a source where great amout of moto formato ca be obtaed: the moocular mages cotag huma movemets. Ths approach ca be used may felds 2,3, e.g. vrtual realty, choreography, rehabltato, commucato, survellace systems, move producto, game dustry, mage codg, gat aalyss. However, due to the lack of formato the thrd dmeso ad the fact that the huma body s a extremely complex object, the problem of geeratg 3D huma moto from 2D mages take by sgle camera s qute dffcult. It s 075 K. Wojcechowsk et al. (eds.), Computer Vso ad Graphcs, Sprger. Prted the Netherlads.

2 076 mathematcally straghtforward to descrbe the process of projecto from a 3D scee to a 2D mage, but the verse process s typcally a ll-posed problem. Che ad Lee 4 preseted a method to determe 3D locatos of huma jots from a flm recordg walkg moto. I ths method, geometrc projecto theory, physologcal ad moto-specfc kowledge ad graph search theory are used. Aother approach s dvde-ad-coquer techque reported by Holt et al. for huma gat 5. Although the smplcty of ths approach s attractve, t s usatsfyg sce t does ot explot the fact that dfferet compoets do belog to the same model. The recostructo method proposed by Camllo J. Taylor 6 does ot assume that the mages are acqured wth a calbrated camera, but the user s requred to specfy whch ed of each segmet s closer to the observer. Barro ad Kakadars 7 estmated both the huma s athropometrcal measuremets ad pose from a sgle mage. Ther approach requres the user to mark the segmets whose oretato s almost parallel to the mage plae. The ovelty of our approach s that t s able to deal wth huma motos from 2D mages wthout camera calbrato ad user terferece. It provdes a alteratve way for huma amato by low cost moto capture whle t avods may lmtatos that come up wth curret moto trackg equpmets. 2. EXTRACTION OF FEATURE POINTS 2. Extracto from mage wth markers The markers wth dfferet colors are stuck to the tght clothes of a huma subject where the jots are located. Motos of the subject are recorded by a dgtal camcorder for 30 frames every secod. The vdeo sequece s composed of m moocular mages of JPEG format, ad each frame ca be represeted as a dscrete two-dmesoal fucto f j. Suppose there are markers o the huma fgure, ad each marker s represeted as M ( r, g, b), where r, g, ad b are color values of the marker ts Red, Gree, ad Blue plae respectvely. Gve a threshold value u, the, whether a pxel p s belog to the marker ca be determed by: p M ( M u p( x, M u) () The procedure for feature extracto from moocular mages wth markers s:

3 Moto Recovery Based o Feature Extracto from 2D Images 077 Step, 2D mage f j s used as put. Step 2, a 3 3 Low Pass Spatal Flter (LPSF) s used to reduce ose,.e., testy value of a pot s replaced by the average of all the pxels aroud t. Step 3, trackg of the markers s executed throughout f j by Equato (), ad the pxels belog to M ( r, g, b) are selected. Step 4, the trackg result may be rregular, or composed of several dscrete regos, thus the ma cotuous part s selected as the result, ad the other parts are dscarded. As llustrated Fgure, those blue crcle are ma part, whle those gree crcle are dscarded parts. Fgure. Processg of the tracked results. Step 5, to extract the feature pots, averagg method as follows s appled. x x, y y (2) where x ad y are averages of coordates of all pxels the tracked result. 2.2 Extracto from mage wthout markers Suppose there are m moocular frames a vdeo sequece wthout markers, ad feature pots huma body. The frame s represeted as f j, each feature pot s represeted as P, j (th feature pot ad jth frame), each template wdow wth the feature pot as ts ceter s w, j. Feature pots the frst frame are pcked maually, ad the procedure for feature extracto from the other frames s: Step, 2D mage f j s used as put. Step 2, a 3 3 LPSF s appled to reduce the ose. Step 3, Lear Predcto s used to predct P, j based o correspodg feature pots prevous frames as follows:

4 078 P, j, j P, j (3) P, j ( P, j P, j ), j m Step 4, Normalzed Cross Correlato (NCC) s utlzed to fd matches of the template mage w, j wth search mage s, j wth P, j as ts ceter by: c( r, t) x y s, j w, j 2 s, j x y x y ( x r, y t) w 2, j ( x r, y t) The posto where the maxmum value of c ( r, t) appears s selected as P, j. Step 5, Least Square Matchg method s appled to fd accurate P (, j x, from the tal pot P (, j x,, durg whch affe trasformatos (.e. rotato, shearg, scalg, ad traslatos) are cosdered as follows: a0 a x a y, b b x b y (5) xew 2 y ew 0 2 (4) 3. MOTION RECOVERY The employed 3D skeletal huma model cossts of 7 jots ad 6 segmets, ad the jots are Hp, Abdome, Chest, Neck, Head, Luparm, Ruparm, Llowarm, Rlowarm, Lhad, Rhad, Lthgh, Rthgh, Lsh, Rsh, Lfoot, Rfoot. Kematcs aalyss resolves ay moto to oe or more of sx possble compoets: rotato about ad traslato alog the three mutually perpedcular axes. Rotatoal rages of the jots 8 are utlzed as the geometrcal costrats of huma moto. Eergy Fucto (EF) s defed to express the devatos betwee the mage features ad the correspodg projecto features as EF Scale() _ agle Scale(2) _ legth Scale(3) _ posto (6) where _agle s devato of oretato, _legth s devato of legth, _posto s devato of posto, whle Scale are weghtg parameters.

5 Moto Recovery Based o Feature Extracto from 2D Images 079 The procedure for moto recovery s: Step. Take a seres of moocular mages as put; Step 2. Extract the feature pots; Step 3. Calculate tal projecto value of the 3D model for every body part; Step 4. Jot Hp s traslated the plae parallel wth the mage to place the projected pot of Hp to the accurate posto; Step 5. Rotate jot Hp based o Eq. (6), ad the descedet jots of Hp are Abdome, Lthgh, Lsh, Lfoot, Rthgh, Rsh, ad Rfoot; Step 6. Adjust the other jots by cosderg ts mmedate descedat such a order: Abdome, Lthgh (ad Rthgh), Lsh (ad Rsh), Chest, Neck, Luparm (ad Ruparm), Llowarm (ad Rlowarm); Step 7. Jot Hp s traslated for aother tme alog the le defed by posto of camera ad the extracted pot of Hp to make the projected posture have the same sze as the huma fgure 2D mage; Step 8. Rotate jot Hp by Eq. (6) aga wth referece to all the other jots of the huma model; Step 9. Adjust other jots by cosderg all ther descedat(s) the same order as Step 6; Step 0. Dsplay the recovered 3D postures. 4. EXPERIMENTAL RESULTS The adopted method for huma moto recostructo from moocular mages s tested by several vdeo sequeces of huma motos, as show Fgure 2 ad Fgure 3. There are 8 frames huma kckg sequece of Fgure 2, ad 3 of them ( st, 3 rd, 5 th ) are dsplayed; whle 3 of the 8 frames (2 d, 4 th, 6 th ) huma farewell sequece of Fgure 3 are llustrated. Fgures the st colum are 2D frames of the vdeo sequece; fgures the 2 d colum are the extracted feature pots (red&dot) ad the amated results (black&sold) from the same vewpot; fgures the 3 rd ad 4 th colum are results from sde ad top vews respectvely.

6 080 Fgure 2. Recovered moto from a kckg sequece. Fgure 3. Recovered moto from a farewell sequece. 5. CONCLUSION A approach for recostructo of huma postures ad motos from moocular mages s preseted. The advatage of ths method s that ether camera calbrato or user s terface s eeded. Expermets show that

7 Moto Recovery Based o Feature Extracto from 2D Images 08 recostructed results are ecouragg, whle some mprovemets are eeded. Future work cludes automatc ad accurate pckg of the occluded feature pots, further studyg of Eergy Fucto ad the bomechacal costrats, etc. REFERENCES. Yahya Ayd, Masayuk Nakajma, Database Guded Computer Amato of Huma Graspg usg Forward ad Iverse Kematcs, Computers & Graphcs, 23 (999), Page(s): D.M.Gavrla, The Vsual Aalyss of Huma Movemet: A Survey, Computer Vso ad Image Uderstadg, Vol. 73, No., Jauary 999, Page(s): Thomas B. Moeslud ad Erk Graum, A Survey of Computer Vso-Based Huma Moto Capture, Computer Vso ad Image Uderstadg 8, 200, Page(s): Ze Che ad Hs-Ja Lee, Kowledge_Guded Vsual Percepto of 3-D Huma Gat from a Sgle Image Sequece, Systems, Ma, ad Cyberetcs, IEEE Trasactos, Vol. 22, No. 2, March/Aprl 992, Robert J.Holt, Aru N.Netraval, Thomas S.Huag, Rchard J.Qa, Determg Artculated Moto from Perspectve Vews: A Decomposto Approach, IEEE Workshop o Moto of No-Rgd ad Artculated Objects, 994, Camllo J. Taylor, Recostructo of Artculated Objects from Pot Correspodeces a Sgle Ucalbrated Image, Computer Vso ad Image Uderstadg 80, 2000, Carlos Barro ad Ioas A. Kakadars, O the Improvemet of Athropometry ad Pose Estmato from a Sgle Ucalbrated Image, IEEE Workshop o Huma Moto, 2000, Jahu Zhao ad Lg L, Huma Moto Recostructo from Moocular Images Usg Geetc Algorthms, Joural of Computer Amato ad Vrtual Worlds, 2004 (5), Page(s):