A Novel Approach for Monocular 3D Object Tracking in Cluttered Environment

Transcription

1 Inernaional Journal of Compuaional Inelligence Research ISSN Volume 13, Number 5 (2017), pp Research India Publicaions hp:// A Novel Approach for Monocular 3D Objec Tracking in Cluered Environmen Navnee S. Ghedia Research scholar, Gujara Technological Universiy, Gujara, India. Dr. C.H. Vihalani Professor and Head of EC Dep., Governmen Engineering College, Rajko, India. Dr. Ashish Kohari Associae Professor and Head of EC Dep., Amiya Insiue of Technology and Science, Rajko, Gujara, India. Absrac Machine percepion is an essenial feaure for an auonomous sysem. For he compuer vision researcher percepion of scene is an imporan aspec. Smar surveillance sysem can be able o sense he environmens and undersand i in a smarly. Locaion and behavior of objecs in a space is helpful in deecion and racking of i in dynamic scenes. Deecion and Tracking of objecs is really a difficul ask if i is o be esimaed in 3D. This paper presens novel and robus approach for 3D objec deecion and racking using monocular scene. Our saisical approach akes geomeric informaion of he 3D scene. So our proposed algorihm is capable o rack rigid objecs in 3D using Monocular camera and i can also handle non saic background and parial occlusions. The performance evaluaion will shows he significan amoun of improvemens, robusness and he efficiency of our proposed algorihm. Keywords: Compuer vision, Tracking, Monocular scene, Geomeric informaion, MAP-EM.

2 852 Navnee S. Ghedia, Dr. C.H. Vihalani & Dr. Ashish Kohari I. INTRODUCTION In he area of compuer vision and machine percepion he saic camera and he gray or color images are he only sources and racking objecs from monocular scene is an criical research due o is numerous applicaion like surveillance, auonomous driving assisance, Spor analysis, Augmened Realiy, ec, [11]. Non saic or dynamic, cluer background and parial occlusions and he esimaion of 3D coordinaes in 2D image plane will add he navigaional complexiy. Our proposed algorihm shows novel probabilisic 3D objec deecion and racking approach. Our mehod is capable o srongly rack a differen number of arges in 3D coordinae in dynamic and complex scenes and in presence of large amoun of variaions in visual appearance. In spie of monocular video sequence and oher consrains, proposed algorihm gives reasonable dynamics and gives unlikely false posiives. Generally we can reduce false posiives and on he same way we can decrease false negaives or we can increase he recall by means of adoping he muli view insead of monocular view. The use of muliview is essenial for he precise racking purpose agains he muliple and fully occlusions. Muliview racking is used o reduce he unseen Areas and give 3D informaion regarding he objecs and he scene. Figure.1 3D objec deecion in monocular scenes Our Paper presens offline racking of muliple objecs in monocular video scenes using geomeric informaion of 3d coordinaes in 2d image plane. We are also using modified Gaussian mixure model as a background subracion and color model as a feaure ype. The mixure model gives robusness o handle dynamic scenes.

3 A Novel Approach for Monocular 3D Objec Tracking in Cluered Environmen 853 Monocular Video Sequences (Capure Frame) Preprocessing (Morphological Analysis) Background Modeling, Background Iniializaion and Mainenance Calculae 3D coordinaes and shape Saisical Approach (MAP-EM) No Is Objec Moving Yes Oupu Image (background and 3D bounding Box of moving objec) Track wih Binary Mask (Track Moving Objecs) Figure 2. Block diagram of 3D objec deecion and racking Figure 2 shows general approach of he 3D moving objec deecion and racking. Objec posiion, objec shape, and he acions carried ou by he objec is an essenial in scene invesigaion. The robus 3D racking depends on he robusness of he foreground voxel deecion. Under he assumpion ha our deecion model is linear and he sysem noise and poseriori disribuions are Gaussian in naure, among all racking approaches predicion filer approach gives beer accuracy [3]. Generally, in a video sequences saic background is available in a video. For he ypical applicaions like spors evens, auonomous driving assisance having a dynamic background, background frames are no available so for he Iniializaion subsequen frames are required o generae he background. In he nex sep preprocessing such as morphology, image resizing and he edge deecion is required. Gaussian mixure model is required for he background subracion and model parameers is iniialized by PDF s and mainain he model parameers by MAP-EM approach. For he 3D objec deecion and racking calculae 3D shape and

4 854 Navnee S. Ghedia, Dr. C.H. Vihalani & Dr. Ashish Kohari coordinaes in 2D image plane. Finally he segmened voxel is required o rack using he recursive approach. Among all he racking approaches Kalman will give beer racking accuracy. II. RELATED WORK We presen background of he proposed algorihm and recen advances of 3D objec racking using monocular video sequence. There is pleny of relaed work on single saic camera moving objec deecion and racking algorihms are available, roughly mos of are suffered from he racking difficulies in parial and fully occlusions. Zhao e.al, [4] proposed a monocular 3D racking mehod. They have used 3D shape model of objec in 2D image plane o help in segmenaion and handling he fully occlusions. Complex objec silhouees can be racked in 3D using kalman filering. Appearance and geomeric informaion are he wo wide approaches for he 3D foreground deecion informaion. In appearance approach, visual hulls are creaed by a silhouee mehod and every model are rained on heir shapes of every moving objec voxel caegorizaion in o objecs [5]. The caegorized objecs can be racked using kalman filering [6] or mean shif [7]. For non similar or discriminaive appearance such algorihms provides excellen racking accuracy and recall while i canno performed well agains he similar objec appearance. he level se mehod [9], human body model fiing [8], are he differen approaches available o handle similar objec appearance and hese approaches can able o handle dynamic scene as well as occlusions. Hoiem e al. [10] and Ess e al. [11] presened a unifying approach o locae objec in 3D space using segmenaion and objec deecions. Kelly e al. [12] proposed a 3D siuaion model using he voxel characerisic. Humans were modeled as a se of hese voxels o decide he camera-handoff difficuly. i is simple o design a rajecory wih hen excellen qualiy of he filering by dynamic programming which rajecories are expeced one afer anoher [13]. For he deecion and racking in 3D space using monocular video sequence, exrac foreground voxels and is locaion is evaluaed by analyzing or considering deecion responses from more han one muliview in a common 3D space [3]. Peer Carre.a.[14] proposed a novel approach for he 3D objec deecion using geomeric primiives because in occupancy map i produced fixed locaion specific paerns. Mean shif is responsible for deermining objecs and is posiions.in comparison o 3D, 2D algorihm gives low robusness of he foreground deecion informaion in crowded scenes due o he absence of aliude. Our objecive is o develop a saisical probabilisic scene model ha can deduce he locaion of moving objecs in 3D coordinaes in monocular video sequences. III PROPOSED METHOD Muliple objec deecion in 3d space under he dynamic background is required vigorous moion segmenaion and precise racking. To rack 3D objecs in monocular sequences our proposed algorihm works on geomery based mehod. In our proposed approach objec appearances are discriminaive. The objec silhouee can be deeced using he radiional modified GMM approach. To represen 3D objec shape, semi parameric PDFs and for he objec parameer MAP-EM are used.

5 A Novel Approach for Monocular 3D Objec Tracking in Cluered Environmen 855 Inpu Video Sequnces /Video frames (Monocular Video) Iniial Frames Subsequen Frames Exrac Background Image Analysis Noise Removal Background Esimaion using Mixure Model (GMM) Se Learning Parameer, Esimae Model parameer and mixure componens Exrac Foreground region Nodes and edges Updae/ Mainain Mixure Parameer,, Calculae 3D coordinaes X i Assign labels L Find Newly Tracked objec Calculae 3D shape (SP-PDF) i Classify Foreground Voxels using MAP - EM Nex Frame Esep : Parameric Esimaion M sep : Sae Esimaion Tracked Objec (Kalman) Sae and parameer Updaion Figure 3. Proposed Algorihm Our proposed algorihm based on modified GMM approach. The background model can be esimaed and iniialize he parameers using MLE esimaion and mainain he background model parameers using EM algorihm. Once foreground voxel are esimaed calculae 3D coordinaes and 3D shape using semi parameric PDFs.. The

6 856 Navnee S. Ghedia, Dr. C.H. Vihalani & Dr. Ashish Kohari foreground voxels can be classified using MAP-EM algorihm. Finally he segmened voxels can be racked using he 3D kalman filer. Foreground Voxel Classificaion: We classify foreground voxels of objecs as he assumpion of he muual disribuion p( x, l ). Foreground voxels of objecs a ime, A {1,2,..., n}. Where, x =3D coordinaes of moving objec voxel,l A). n= number of objecs. The join and condiional disribuions can be shown as, p( x, l) p( l) p( x l) Where, pl () =prior, is used as a mixing coefficien in EM frame work. p( x l ) =likelihood funcion. Prior for he EM algorihm can be expressed as, p() l prior l ( l 1 0 l 1, l A) Le, l = moving objec 3D posiion l a. p( x l) is he probabiliy of x given is gives, p( x l) : p( x ) Where, px ( l ) = semi parameric PDF of 3D shape of a objec. l For he MAP esimaion of he in MAP-EM is assume he posiions of objec a l are close o ( 1), he prior of l can be defined as a mulivariae Gaussian disribuions, -1 p( ) : (, ) l l l where =covariance MAP-EM: Le represens he 3D coordinaes for he moving objec voxels. For he said daase, calculae he Maximum a poseriori for mixure model. Model parameers can be iniialized as, { l, } Calculae E M seps. 1 from he curren approximaion E sep: We use o find he poserior disribuion. p( l x, ) ( x ) Of he laen variables M sep: We esimae l l la using he repeiive compuaion and 1 by maximizing he sum of he expecaion of he log likelihood Q( ) and he logarihm of he prior p( ) wih as follows, 1 R arg max ( ) Where, ( R ) Q( ) ln p( )

7 A Novel Approach for Monocular 3D Objec Tracking in Cluered Environmen 857 Q( ) is known as Q-funcion. Where, ln p( ) ln p ( l ) la We can maximize i by, 1 Q p arg max ( ) ln ( ) Where, ( Q ) log p( l ) p( l x, ) dx l( x) According o Bayes heorem, p( l x, ) p( l) p( x l, ) and ln p ( x l, ) {ln p ( l ) ln p ( x l )} px ( ) xx Model parameers are updaed as, Prior can be updaed by. l l u The 3D posiion of objec can be updaed by, l lu Objec Tracking: Muliple objec racking requires high precision. Proposed algorihm can handle dynamic environmen and occlusions. For he robus racking approach objec moion informaion and objec feaure examinaion are required. Generally for he predicive and saic environmen racking can be done along wih he deecion. While in presence of noise or dynamic background racking can be done afer deecion or moion segmenaion. Fas moving and non-rigid objec may increase he racking complexiy. We can jus rack moving objecs by applying cerain consrains on objec moion, appearance and silhouee. Prior knowledge and number of objecs informaion make i easy o rack moving objecs. Various objec racking approaches are available o mee he differen challenges like objec represenaion, feaure, moion, appearance, silhouee and environmen [3]. Kalman filering: Tracking and daa predicion can be achieved using he kalman filering. I can gives an analyical approach for he visual moion esimaion and racking. I is he numerical approach which uses successive inpus and saisical equaions o minimize he measuremen and process noise.

8 858 Navnee S. Ghedia, Dr. C.H. Vihalani & Dr. Ashish Kohari Prioir Knowledge Of Sae Predicion Sep Based On Physical Model Updae he Nex Time sep Esimae of error K K 1 P X 1 & k 1 k Esimae nex sep p X / k/ k& k k Updae sep (Compare predicion o measuremens) Oupu Esimae Of Sae Measuremens Figure 4. Block diagram of Kalman filer The Kalman filer is used o esimae pas, presen and fuure sae under unprediced sae. The kalman filering recursively esimaes he process sae. I aains feedback as filer measuremens. All he filer equaions are generally caegorized ino ime and measuremen updaes. By using ime equaions, kalman filer esimaes nex ime seps wih he help of curren sae and he error sae. The complee recursive approach is beauifully managed by measuremen equaion. The rime and measuremen equaions can also be considered as predicion and correcion seps for he kalman filering [19]. Time Updae Equaions Measuremen Updae Equaions xˆ Axˆ Bu k k1 k1 P AP A Q k T k1 T T K P H ( HP H R) k k k xˆ xˆ K z Hxˆ k k k k k P ( I K H) P k k k 1 IV. EXPERIMENT RESULT To evaluae our proposed approach, we rely he sandard CLEAR muliple-objec racking (MOT) performance meric. The muli objec racking accuracy is calculaed wih he help of differen frame errors like error raios, false negaives (FNs), false posiives (FPs), and ideniy swiches (IDSs). The robusness of he sysem can be verified from he MOTA values. The MODA score uilize missed couns and false alarms couns [15]. We are evaluaing our algorihm on couple of sandard challenging video daases. (APIDIS [16], ICG- Lab-6 [17] and PETS 2009 [18 ]). Our proposed algorihm is considered for 3D objec deecion and racking for monocular sequences. In monocular sequences he ground ruh is available in 2D and we are going o rack objec in 3D wihou he objec aliude.

9 A Novel Approach for Monocular 3D Objec Tracking in Cluered Environmen 859 Figure 5: 3D racked model APIDIS Figure 5 shows he ypical racking resul. The sequence is suffered wih he cluer background, occlusions and similar appearance. Our proposed approached will aain precise and exac racking resuls, even wih he consrains like shadowing effecs, heavy reflecions and he random movemen of he moving objecs. Our proposed algorihm fails o deec and rack objecs which are fully occluded. In addiion o his, fig. 6 shows an addiional racking resul of PETS daa se validae our proposed approach. In boh he sequences we have generaed 3D deecion maps using one of he muliviews available from he daases. Under he various consrains and challenges our proposed approach shows significan improvemens in racking and deecion resul. Figure 6 3D racked Model PETS 2009

10 860 Navnee S. Ghedia, Dr. C.H. Vihalani & Dr. Ashish Kohari Table 1: Muliple Objec deecion accuracy a a olerance or disance hreshold of 0.5m Sequence [3 ] [ 2] [14] Proposed APIDIS PETS LEAF MUCH PETS [3] [2] [14] PROPOSED 0.8 MODA APIDIS PETS 2009 LEAF 2 MUCH PETS 2006 Sequences Figure 7. Muliple Objec Deecion Accuracy for he differen Daases. Table 1 exhibis comparaive analysis of he sandard daases. For he monocular 3D objec deecion proposed algorihm shows significan improvemen in erms of he recall for he APIDIS sequences. While for he oher sequences i sill exhibis comparaive improvemens for aypical olerance of 0.5m. Our proposed algorihm reduces he amoun of false posiives and ulimaely i leads o beer deecion resuls. Fig. 7 indicaes he comparaive analysis of he Muliple Objec deecion Accuracy. MODA MOTA F MODA 1 T n p Fp F MOTA 1 n m fp mme g

11 A Novel Approach for Monocular 3D Objec Tracking in Cluered Environmen 861 Table 2: Comparaive Evaluaion. Sequence Mehod MOTA TP FP FN IDS APIDIS Vol [3] POM [2] Proposed LEAF 2 Vol [3] POM [2] Proposed MUCH Vol [3] POM [2] Proposed APIDIS [16], ICG- Lab-6 [17] Muliple Objec Tracking Accuracy MOTA, he oal number of rue posiives (TP), false posiives (FP), false negaives (misses, FN), and ideniy swiches (IDS). 1.0 POM [2] VOL.MASS [3] PROPOSED 0.8 MOT accuracy APIDIS LAEF 2 MUCH sequences Figure 8. Muliple Objec Tracking Accuracy for he sandard daases. Table 2 shows he comparaive analysis of he MOTA and errors like False Posiives, Negaives and mismach error and rue posiives. Higher Muliple Objec Tracking Accuracy exhibis robusness of he algorihm, generally i is calculaed from he raio of false posiives, negaives and mismach or ideniy swiches. Our proposed mehod shows significan improvemen in APIDIS sequences. While in case of he oher sequences i shows comparaive improvemen. Our proposed algorihm gives beer

12 862 Navnee S. Ghedia, Dr. C.H. Vihalani & Dr. Ashish Kohari precision a he cos of lower recall as false negaives increases bu overall i will increases he racking accuracy compares o oher approaches. Figure 9 shows he comparaive evaluaion for he sandard video sequence PETS Analyze he evaluaion a a consan FPPI rae of 0.1, decor achieves missrae of Dynamic and cluer backgorund inroduces false deecion. We can ge lower missrae or improvemen in issrae a he cos of false posiives. Our proposed algorihm shows cosiderable improvemen in missrae and o he decion and racking accuracy PROPOSED [2] [3] miss rae false posiive per image Figure 9. Comparaive anaysis for PETS 2009 V. CONCLUSION We have presened an algorihm for racking muliple moving objecs in monocular 3D scene geomery. There are numerous possible approaches are available, one is o use objec appearance o differeniae he objecs. Such approach canno disinguish objecs wih similar appearance. Our proposed mehod used geomeric informaion regarding 3D scene srucure. We have proposed a probabilisically MAP-EM model o classify foreground voxels. In performance evaluaion merics, our proposed mehod shows significan improvemens in deecion and racking accuracies. Our approach also shows he comparaive improvemens in false deecion. For he fuure aspecs we can plan o expand our proposed model ha can handle exensive occlusions, shadowing effecs and heavy reflecions.

13 A Novel Approach for Monocular 3D Objec Tracking in Cluered Environmen 863 REFERENCES [1] Taiki Sekii, Robus, Real-Time 3D Tracking of Muliple Objecs wih Similar Appearances, In CVPR [2] J. Berclaz, F. Fleure, E. T ureken, and P. Fua. Muliple objec racking using k-shores pahs opimizaion. PAMI,33(9): , [3] H. Possegger, S. Sernig, T. Mauhner, P. M. Roh, and H. Bischof. Robus real-ime racking of muliple objecs by volumeric mass densiies. In CVPR, [4] Zhao, T. and Nevaia, T Tracking Muliple Humans in Complex Siuaions, IEEE PAMI, [5] L. Guan, J. S. Franco, and M. Pollefeys. Probabilisic muliview dynamic scene reconsrucion and occlusion reasoning from silhouee cues. IJCV, 90(3): , [6] M. C. Liem and D. M. Gavrila. Join muli-person deecion and racking from overlapping cameras. CVIU, 128:36 50, [7] A. Tyagi, M. Keck, J. Davis, and G. Poamianos. Kernel based 3D racking. In IEEE Inernaional Workshop on Visual Surveillance, [8] X. Luo, B. Berendsen, R. T. Tan, and R. C. Velkamp. Human pose esimaion for muliple persons based on volume reconsrucion. In ICPR, 2010 [9] Y. Iwashia, R. Kurazume, K. Hara, and T. Hasegawa. Robus moion capure sysem agains arge occlusion using fas level se mehod. In ICRA, [10] Hoiem, D., Efros, A.A., Heber, M.: Puing objecs in perspecive. IJCV 80 (2008) [11] Ess, A., Leibe, B., Schindler, K., Van Gool, L.: Robus muli-person racking from a mobile plaform. PAMI 31 (2009) [12] P. Kelly, A. Kakere, D. Kuramura, S. Moezzi, S. Chaerjee, and R. Jain, An Archiecure for Muliple Perspecive Ineracive Video, Proc. Third ACM In l Conf. Mulimedia, [13] F. Fleure, J. Berclaz, R. Lengagne, and P. Fua, Muli-Camera People Tracking Wih a Probabilisic Occupancy Map, IEEE Transacions on Paern Analysis and Machine Inelligence, vol. 30, no. 2, pp , February [14] P. Carr, Y. Sheikh and I. Mahews, Monocular Objec Deecion Using 3D Geomeric Primiives, ECCV [15] K. Bernardin and R. Siefelhagen. Evaluaing muliple objec racking performance: The CLEAR MOT merics. In EURASIP JIVP, [16] 1hp://

14 864 Navnee S. Ghedia, Dr. C.H. Vihalani & Dr. Ashish Kohari [17] hp://lrs.icg.ugraz.a/download#lab6 [18] PETS 2009 Daase [19] Greg Welch and Gary Bishop. An inroducion o he kalman filer, 1995 & [20] PETS 2006 Daase