Fish-Eye Camera Video Processing and Trajectory Estimation Using 3D Human Models

Transcription

1 Fish-Ee Camera Video rocessing and Trajecor Esimaion Using 3D Hman Models Konsanina Koari, Kosas Delibasis, Vassilis lagianakos, Ilias Maglogiannis To cie his version: Konsanina Koari, Kosas Delibasis, Vassilis lagianakos, Ilias Maglogiannis. Fish-Ee Camera Video rocessing and Trajecor Esimaion Using 3D Hman Models. Lazaros Iliadis; Ilias Maglogiannis; Harris apadopolos. 1h IFI Inernaional Conference on Arificial Inelligence Applicaions and Innovaions AIAI, Sep 14, Rhodes, Greece. Springer, IFI Advances in Informaion and Commnicaion Technolog, AICT-436, pp , 14, Arificial Inelligence Applicaions and Innovaions. <1.17/ _38>. <hal > HAL Id: hal hps://hal.inria.fr/hal Sbmied on 3 Nov 16 HAL is a mli-disciplinar open access archive for he deposi and disseminaion of scienific research docmens, wheher he are pblished or no. The docmens ma come from eaching and research insiions in France or abroad, or from pblic or privae research ceners. L archive overe plridisciplinaire HAL, es desinée a dépô e à la diffsion de docmens scienifiqes de nivea recherche, pbliés o non, émanan des éablissemens d enseignemen e de recherche français o érangers, des laboraoires pblics o privés. Disribed nder a Creaive Commons Aribion 4. Inernaional License

2 Fish-ee Camera Video rocessing and Trajecor Esimaion Using 3D Hman Models K. Koari 1, K.K. Delibasis 1,V. lagianakos 1, I. Maglogiannis 1 Universi of Thessal, Dep. of Comper Science and Biomedical Informaics, Lamia, Greece Universi of iraes, Dep. of Digial Ssems, iraes, Greece koarikonsanina@gmail.com, kdelibasis@ahoo.com, vpp@dib.h.gr; imaglo@nipi.gr Absrac.Video processing and analsis applicaions are par of Arificial Inelligence. Freqenl, silhoees in video frames lack deph informaion, especiall in case of a single camera. In his work, we ilize a hree-dimensional hman bod model, combined wih a calibraed fish-ee camera, o obain hree-dimensional 3D cles. More specificall, a generic 3D hman model in varios poses is derived from a novel mahemaical formalizaion of a wellknown class of geomeric primiives, namel he generalized clinders, which exhibi advanages over he exising parameric definiions. The se of he fishee camera allows he generaion of rendered silhoees, sing hese 3D models. Moreover, we presen a ver efficien algorihm for maching ha 3D model wih a real hman figre in order o recognize he posre of a moniored person. Firsl, he silhoee is segmened in each frame and he calclaion of he real hman posiion is calclaed. Sbseqenl, an opimizaion process adjss he parameers of he 3D hman model in an aemp o mach he pose posiion and orienaion relaivel o he camera of real hman. The experimenal resls are promising, since he pose, he rajecor and he orienaion of he hman can be accrael esimaed. Kewords: fish-ee camera video processing, hree-dimensional hman modelling, posre recogniion, minimizaion, generalized clinders, and ellipical inersecions. 1 Inrodcion The field of aomaed hman acivi recogniion ilizing fixed cameras of indoor environmens has gained significan ineres dring he las ears. I finds a varie of applicaions in diverse areas, sch as assisive environmens, smar homes, sppor for he elderl or he chronic ill, srveillance and secri, raffic conrol, indsrial processes, ec. This work focses on fish-ee camera video processing for pose esimaion of siing or sanding/walking hmans. Therefore, hman silhoee segmenaion of he video seqence is a prereqisie. Recognizing a hman paern is ofen possible via volme inersecion [1] or a voxel-based approach [,3]. Sereomer based models have also adfa, p. 1, 11. Springer-Verlag Berlin Heidelberg 11

3 been consrced hrogh calibraed camera pairs. Using rianglaion, he dephs of he poins are calclaed. This approach has been aken ino accon b länkers and Fa [4] and Hariaogl e al. in [5]. Sereo vision is also sed b Jojic e al. [6], wih he opional aid of projeced ligh paerns. The proposed algorihm is based on a parameric hree-dimensional 3D hman model wih limied degrees of freedom so ha i allows efficien maniplaion for sanding/walking and siing posres. Or aim is o esimae hman posiion, rajecor and sanding/siing sae, which wold be sefl owards hman behavior recogniion. The firs sep in applicaions dealing wih hman acivi recogniion from video is he foregrond segmenaion. Mos video segmenaion algorihms are based on backgrond sbracion. The backgrond has o be modelled, since i ma change de o a nmber of reasons, inclding: moion of backgrond objecs, changes in ligh condiions, or video compression arifacs. In his work, we emplo he forward and inverse camera model ha was proposed in [7]. We follow a op-down approach ha maches he model rendered hrogh he calibraed fish-ee camera, wih he segmened frame of he video. Then, an opimizaion algorihm is ilised o find he model parameers and deermine hman orienaion and pose. The res of he paper is srcred as follows: Secion discsses he echnical deails of he proposed algorihms, Secion 3 presens some iniial resls, while Secion 4 concldes he paper. roposed Mehodolog.1 Generalized Clinders For he generaion of he hman model, we ilized he concep of generalized clinders GC, as proposed in [8]. More specificall, le C 1 be a piecewise smooh crve defined in a Caresian coordinae ssem OXYZ, as: r = x,, z, [a,b] R 1 1 and C be a planar crve defined in an orhogonal local Caresian ssem OXY. Les now consider he srface S ha is generaed b moving he crve C along C 1, so ha is plane is perpendiclar o he angen vecor of C 1, and he origin of OXY belongs o r = r, [, π ] and C 1.If we express he planar crve C in polar coordinaes φ along he angen vecor of C1 inrodce a scale facor sand a roaion facor as fncion of posiion along C 1, hen he srface eqaion of he GC becomes: s r φ cos φ x, = x s x z r φ sin φ 1

4 sin cos, 1 r z s r x s φ φ φ φ = 3 sin, 1 r x s z z φ φ = 4 where, ] [, ], [, π b a, 1 z x = and x =.The complee proof is given in [8]. Fig. 1.Two examples of srfaces derived from eqaion 4 from [8]. 3D Model Consrcion In his work, a free rianglaed model of a sanding hman Fig. is ilised, defined b he Caresian coordinaes of approximael 7, verices [9]. Since we are ineresed in simlaing he rendering of he hman model hrogh he fish-ee camera in real ime, we discard he riangle informaion of he model and we rea i as a clod of poins.

5 Fig.. Trianglaedmodel of a sanding hman [9]. Therefore, we compe he inersecions of he model in a nmber of horizonal planes, in disance of wo cenimeres along he Z axis fee head direcion as shown in Fig.3 a. The same process is repeaed, along hands and legs see Fig. 3b a Fig. 3. Ellipical inersecions of orso and leg. b

6 Each inersecion is esimaed for approximaing an ellipse wih is semi-axes bsemi parallel o X and Y axis of coordinae ssem, as shown in Fig.4. a, semi Fig. 4. Ellipical inersecion of he hman orsowih hexy plane. The paramerical eqaion of GC 4 can be simplified b sing a piecewise sraigh line as crve C 1, each segmen of which is defined b vecor a, b, c and b assigning hese ellipses as planar closed crvec as follows: b r cos x = a acr sin a r cos b c r sin = b 5 a z = c b r sin a where semibsemi r = and a, b, and c are deermined b b cos a sin semi semi he direcion of he leading axis. In Fig.5 we see he resl of ellipical paerning of model inersecions, hrogh he inserion of ellipses o he GC Eq. 5. Noe ha a orsional inconsanc a he knees secion is being observed. This phenomenon has been explained in [8, secion 4] and does no affec he opimizaion process ha maches he 3D model o he segmened hman silhoee.

7 Fig. 5.3D sanding lef and siing hman righ. The esimaion of hman posre siing or sanding is based on he consrcion of he hman model. Having he 3D sanding hman model consrced as described above, is ransformaion o mach he siing posiion can be easil performed b changing he model parameer angles a wais and knees. The resl of ha ransformaion is shown in Fig. 5 righ, while Fig.6 depics boh models as he are ilised b he video-processing algorihm. Fig. 6. 3D Hman silhoees.3 Video rocessing Algorihm In his work, we analze videos capred b a fish-ee camera, fixed on he ceiling of a living environmen. The recorded videos have been foregrond segmened, while emp frames are being discarded. Then, he mask shown in Fig. 7a is applied o sppress nois segmened pixels oside he field of view. The iniial esimaion of he real hman posiion in he room coordinaes is accomplished b he recenl proposed algorihm in [7] based on he segmened frame pixels. For his prpose, we

8 emplo he calibraion of he acqiring fish-ee camera ha provides he spherical coordinaes θ, φ for each pixel of he crren frame, as well as he frame pixel ha corresponds o an real world poin x,,z, according o [7]: ji, M xz,, = 6 θϕ, = M i, j 7 1 Le HIi,j hold he vale of φ for pixel i,j, as obained b 7 and shown in Fig. 7b. Ths, for an pose of he 3D parameric model, we can obain he binar image- I M of he hman model, rendered b he fish-ee model sing 6. Fig.8illsraes image I M combined for varios sanding and siing models, as rendered b he calibraed fish-ee camera. Le I S be segmened binar frame, afer sing he binar mask in Fig.7a. The iniial esimaion of he person s posiion is obained b locaing he non-zero pixel i,j of I S ha holds he minimm vale of angle φ. The objecive fncion, which qanifies he mach beween he model and he segmened hman silhoee as a fncion of is real world posiion x, and is orienaion θ, is defined as he inersecion of he segmened silhoee I S and he rendered hman modeli M : f x I I 8,, θ = M S image domain where I M defined above is shown in red, I S shown in green and heir inersecion I M I S is shown in ellow. Fig. 9 presens graphicall he calclaion of he objecive fncion for one insance of each class siing and sanding. Sbseqenl, he simplex [1] mlidimensional nconsrained maximisaion algorihm is ilised o opimise he objecive fncion. The iniial posiion x, of he hman is approximael comped from he firs frame b finding he segmened hman silhoee pixel i,j wih maximm HI, as described in [7] and sed o iniialize he simplex mehod. Ths, he maximisaion algorihm compes he hman model parameers ha bes mach he segmened figre and rerns he coordinaes x and, as well as he orienaion θ of model. a Fig. 7. a Binar mask sed o exclde o of filed-of-view pixels in video frames b Visalizaion of he HI angle for each frame pixel as resled from he camera calibraion [7]. b

9 Fig. 8. Rendered sanding and siing hman model in varios angles and posiions in he room. a Fig. 9.Visalizaion of he calclaion of he objecive fncion see ex for deails for he sanding a and siing hman b. b 3 Experimenal Resls Classificaion resls of wo differen videos are presened in Table 1, considering he siing as posiive and he sanding as negaive sas. The grond rh was esablished b manall annoaing he video wih he hman pose, as siing or sanding. Table 1. ose classificaion resls T TN FN F Accrac Sensiivi Specifici Video 1 11/78 11/78 3/78 33/ Video 98/164 54/164 5/164 7/

10 The proposed model-based algorihm is able o esimae he rajecor of he hman silhoee, as well as is orienaion. Fig. 1 shows he esimaed posiions and orienaion of he hman silhoee for video 1, as recovered b he opimizaion described above. The person moves from lef o righ a he boom of he frame A o B, hen vice versa a he op of he frame and finall sis down a he poin designaed b E, where i roaes. Beween poins B and C he segmenaion fails emporaril, however, he model-based racking algorihm sccessfll recovers he silhoee s new posiion. The experimenal resls indicae ha he simplex opimizaion mehod is robs and efficien. I needs approximael 3 ieraions for each frame in order o converge. Each objecive fncion evalaion reqires approximael 3msec on an Inel i5 lapop wih 4 GB Ram sing he Malab environmen. Rnning ime approaches he 9 milliseconds per frame. D C E A B Fig. 1. Graphical represenaion of he resls for model based rajecor and orienaion esimaion for video 1. 4 Discssion and Conclsions In his paper, an algorihm for esimaing he rajecor of a hman silhoee in indoor videos acqired b an omni-direcional camera has been presened. The algorihm is based on a parameric 3D hman model and i recovers he model parameers ranslaion and orienaion b opimizing a siable defined objecive fncion. Iniial resls show ha he proposed algorihm can esimae he rajecor and orienaion and discriminae beween wo differen posres: siing and sanding. The proposed mehodolog ma improve he accrac of more complex acivi recogniion algorihms sall fond in ambien assised living environmens. This mehodolog

11 can be adoped for deecing higher level acivi evens and ndersand behavioral paerns. Acknowledgmen The ahors wold like o hank he Eropean Union Eropean Social Fnd ESF and Greek naional fnds for financiall spporing his work hrogh he Operaional rogram "Edcaion and Lifelong Learning" of he Naional Sraegic Reference Framework NSRF - Research Fnding rogram: \Thalis \ Inerdisciplinar Research in Affecive Comping for Biological Acivi Recogniion in Assisive Environmens. References 1. Boino A., Larenini A.: A silhoee-based echniqe for he reconsrcion of hman movemen. In: Comper Vision and Image Undersanding CVIU, 831, pp German K.M. Cheng, Baker S., Kanade T.: Shape-from silhoee of ariclaed objecs and is se for hman bod kinemaics esimaion and moion capre. In: roceedings of he Conference on Comper Vision and aern Recogniion CVR 3, vol. 1, pp , Madison, WI Mikic I., Trivedi M., Hner E., Cosman.: Hman bod model acqisiion and racking sing voxel daa, Inernaional Jornal of Comper Vision 533, länkers R., Fa.: Tracking and modeling people in video seqences, Comper Vision and Image Undersanding CVIU 813, Hariaogl I., Harwood D., Davis L. S., W4s: A real ime ssem deecing and racking people in 1/D In: roceedingsof he Eropean Conference on Comper Vision ECCV 98,Lecre Noes in Comper Science, vol. 146, , Freibrg, German Jojic N., G J., Shen H., S. Hang T.S: 3-Dreconsrcion of mlipar, selfocclding objecs. In: roceedingsof he Asian Conference on Comper Vision ACCV 98, , HongKong, China Delibasis K.K., Godas T., lagianakos V.., Maglogiannis I.: Fishee Camera Modeling for Hman Segmenaion Refinemen in Indoor Videos. In: ETRA 13, Ma , Island of Rhodes, Greece Coprigh 13 ACM /13/5. 8. K.K. Delibasis K., Kechriniois A., Maglogiannis I.: A novel ool for segmening 3D medical images based on generalized clinders and acive srfaces, Comper Mehods and rograms in Biomedicine, 111, hp:// 1. Nelder J.A. and Mead R.: A simplex mehod for fncion minimizaion, Comper Jornal, 7, pp