MODEL BASED TECHNIQUE FOR VEHICLE TRACKING IN TRAFFIC VIDEO USING SPATIAL LOCAL FEATURES

Transcription

1 MODEL BASED TECHNIQUE FOR VEHICLE TRACKING IN TRAFFIC VIDEO USING SPATIAL LOCAL FEATURES Arun Kumar H. D. 1 and Prabhakar C. J. 2 1 Deparmen of Compuer Science, Kuvempu Universiy, Shimoga, India ABSTRACT In his paper, we proposed a novel mehod for visible vehicle racking in raffic video sequence using model based sraegy combined wih spaial local feaures. Our racking algorihm consiss of wo componens: vehicle deecion and vehicle racking. In he deecion sep, we subrac he background and obained candidae foreground objecs represened as foreground mask. Afer obaining foreground mask of candidae objecs, vehicles are deeced using Co-HOG descripor. In he racking sep, vehicle model is consruced based on shape and exure feaures exraced from vehicle regions using Co-HOG and CS- LBP mehod. Afer consrucing he vehicle model, for he curren frame, vehicle feaures are exraced from each vehicle region and hen vehicle model is updaed. Finally, vehicles are racked based on he similariy measure beween curren frame vehicles and vehicle models. The proposed algorihm is evaluaed based on precision, recall and VTA merics obained on GRAM-RTM daase and i-lids daase. The experimenal resuls demonsrae ha our mehod achieves good accuracy. KEYWORDS Traffic vehicle racking, Vehicle model, Spaial local feaures, CS-LBP, Co-HOG, Shape feaures, Texure feaures, 1. INTRODUCTION Real-ime vehicle racking in raffic video is an essenial componen of an inelligen video raffic surveillance sysem. Accurae and real-ime vehicle racking will grealy improve he performance of vehicle classificaion, road vehicle densiy esimaion, vehicle aciviy analysis and high-level abnormal evens analysis like lane crossing, sudden and long ime vehicle sop. The aim of a vehicle racker is o generae he rajecory of he vehicle over ime by locaing is posiion in every frame of raffic video. Developmen of a robus racking mehod for vehicles is challenging because of: complex vehicle appearances like pose and scale variaions, occlusion (he vehicle may be occluded by he background or oher moving vehicles), and complex vehicle moion. The feaures-based vehicle racking algorihms (Perez, P. e al., 2002; Avidan, S., 2007; Ross, D. e al., 2008; Grabner, H. e al., 2006; Wang, S. e al., 2011) are mos promising and racking is performed based on racking of feaures such as disinguishable poins or lines on he vehicle. Selecing he righ feaures plays an imporan role in order o increase he accuracy of feauresbased racking algorihms (Beymer, e al., 1997). In general, he desirable propery of a visual feaure is is uniqueness so ha he vehicle can be easily disinguished in he feaure space. For example, color, exure, inensiy, and pixel-based feaures are he spaial appearance feaures widely used o rack he vehicle. Su, X. e al. (2007) have proposed rule based muliple objecs racking sysem for raffic surveillance using a collaboraive background exracion algorihm. DOI: /mlaij

2 Jung, Y.K. e al. (2001) have proposed feaures-based vehicle racking sysem, which exracs corner feaures of he vehicle and racks he feaures using a linear Kalman filer. Babaei, P. e al. (2010) have proposed he racking sysem which is based on a combinaion of a emporal difference and correlaion maching in defined raffic zones. The sysem effecively combines simple domain knowledge abou vehicle classes wih ime domain saisical measures o recognize arge vehicles in he presence of parial occlusions. Gao, e al. (2008) have proposed paricle filering based racking mehod. A moving vehicle is deeced by redundan discree wavele ransforms mehod (RDWT), and he key poins are obained by scale-invarian feaure ransform (SIFT). The maching of key poins in he follow-up frames is obained by he SIFT mehod and are used as he firs paricles o improve he racking performance. Dahlkamp, e al. (2004) proposed Edge-Elemen Associaion (EEA) and Marginalized Conour (MCo) approaches for 3D model-based vehicle racking in raffic scenes. Based on usage of global and local feaures, feaures-based racking algorihms can be furher classified ino wo caegories: Global mehods(ha, e al., 2011), and Local mehods (Grabner, H. e al., 2006; Yu, Q. e al., 2008; Tran, S. e al., 2007; He, W. e al., 2009; Wang, S. e al., 2011). The global mehods work in many pracical applicaions, bu have several basic limiaions. Firs, i is very difficul o capure he small changes in illuminaion variaion and difficul o represen he local deails like scale and shape variaions. Second, global represenaions are no robus for parial occlusion. Once he vehicles are occluded, he whole feaure vecor of vehicle represenaion is affeced. Third, global represenaions are hard o updae. Hence, global mehods are no efficien for vehicle racking in raffic video. Recenly, local mehods have opened a promising direcion o solve hese problems by represening a vehicle as a se of local pars or sparse local feaures. Par-based rackers generally use ses of conneced or visual local properies. The pars used for vehicle represenaion are updaed during racking by removing he old pars ha exhibi signs of drifing and adding new ones for easy accommodaion of appearance changes. In order o solve he problems associaed wih he global feaures-based algorihms, researchers have developed model-based racking algorihms for vehicle racking (Liu, X. e al., 2011; Cehovin, L. e al., 2013; Kwon, J. e al., 2009). In he model-based echnique, here are wo key componens: vehicle represenaion and dynamics. Vehicle represenaion ries o model he vehicle as correcly as possible so ha he racking algorihm can correcly describe he complex vehicle appearance. The vehicle dynamics model represens how he vehicle appearance evolves over ime o be able o handle appearance variaions. These wo problems are usually coupled ogeher. The vehicle represenaion should be designed o simply updae he model based on appearance variaions, while he vehicle dynamics should be able o ake advanage of he characerisics of vehicle represenaion for model updae. In his paper, we proposed vehicle racking in raffic video using model-based sraegy combined wih spaial local feaures. We consruc a vehicle model which capures he variaion in vehicle scale, vehicle pose, and complex vehicles occlusion based on spaial local feaures such as shape and exure feaures exraced using Co-HOG and CS-LBP operaor respecively. Afer consrucing he vehicle model for he curren frame, he vehicle feaures are exraced from each foreground mask of vehicle region and hen vehicle model is updaed. Finally, he vehicles are racked based on he similariy measure beween curren frame vehicles and vehicle model. 2. RELATED WORK Tracking is used o measure vehicle pahs in video sequences. The racking generally follows wo seps: in he firs sep, feaures for he vehicle regions are generaed in every video frame, and in 2

3 he second sep, a daa associaion sep has o provide correspondences beween he regions of consecuive frames based on he feaures and dynamic model. The vehicle racking is mainly used in wo ypes of raffic videos such as highway and urban raffic scenes videos. Vehicles racking on highways are easier han in urban raffic as here are few ypes of objecs (one moorized vehicle of various sizes), lile change in he orienaion of he vehicles and few known enry and exi poins. Cameras are also usually locaed much higher han in urban scenes, which reduce he vehicle occlusions. Tracking vehicles on highways are more challenging when he raffic is slower because he iner-vehicle space is significanly reduced, increasing he occlusion beween vehicles. In urban areas, raffic includes pedesrians, moorcycles, and vehicles, and more complicaed rajecories, wih vehicles urning a inersecions, sopping and parking, and many more enry and exi poins in he scene. Differen compuer vision mehods have hus been developed for hese wo applicaions. Rad, R. (2005) has proposed real-ime racking of muliple vehicles on he highway. They used Kalman filer and background subracion echniques. They exrac he conour of he vehicle using morphological operaions, and he algorihm has hree phases, deecion of pixels on moving vehicle, deecion of a shape of ineres in frame sequences and finally deerminaion of relaion among objecs in frame sequences. Ma, C. e al. (2016) have proposed fusion based hashing mehod for visual objec racking. Nguyen, P.V. e al. (2008) have proposed Muli-modal Paricle Filer (MPF) for racking vehicles. The aim of his mehod is o build some mos basic funcions of a moorcycle surveillance sysem using MPF based on he color observaion model. Babaei, P. e al. (2013) have mehod which addresses synchronizing he cameras for racking vehicles simulaneously in overlapping fields of view. Arrospide, J. e al. (2008) have proposed muli objec feaure racking sraegy. I racks specially seleced poins of he image based on compuaion of sparse opical flow. The racking sraegy includes a cenral oulier rejecion sage, ha ensures robusness of he racker based on probabilisic echniques, and a kalman filering sage o smooh ou he rajecories. Niknejad, H. e al. (2011) have proposed an embedded real ime mehod for deecion and racking of muli objecs including vehicles, pedesrians, moorbikes and bicycles in urban environmen. The feaures of differen objecs are learned as a deformable objec model hrough he combinaion of a laen suppor vecor machine (LSVM) and hisograms of oriened gradiens(hog). Laser deph daa have been used as a priori o generae objecs hypohesis regions and HOG feaure pyramid level is used o reduce he deecion ime. Deeced objecs are racked hrough a paricle filer which fuses he observaions from laser map and sequenial images. Messelodi, S. e al. (2005) have proposed he sysem ha uses a combinaion of segmenaion and moion informaion o localize and rack moving vehicles on he urban road plane, uilizing a robus background updaing, and a feaure-based racking mehod. Srigel, E. e al. (2013) have proposed vehicle deecion and racking a inersecions by fusing muliple camera views. Using his fusion map, he pose, widh and heigh of he vehicles can be deermined. Afer ha, he deeced vehicles are racked by a Gaussian-Mixure approximaion of he Probabiliy Hypohesis Densiy filer. Zheng, Y. e al. (2012) have proposed model based vehicle localizaion and racking for urban raffic surveillance using image gradien maching. The maching beween he 3D model projecion and 2D image daa is a key echnique for model based localizaion, recogniion and racking problems. Lee, K. H. e al. (2015) have proposed a model based 3D consrained muliple kernel racking. This approach regards each pach of he 3D vehicle model as a kernel and racks he kernels under cerain consrains faciliaed by he 3D geomery of he vehicle model. A kernel densiy esimaor is designed o well fi he 3D vehicle model during racking. Kim,G. e al. (2011) proposed vehicle racking based on Kalman filer. They deec cars based in Adaboos and he vehicles are racked using Kalman filer. Barh, A. e al. (2010) have proposed real-ime muli-filer approach for vehicle racking a inersecions. Boh moion and deph informaion is combined o esimae he pose and moion parameers of an oncoming vehicle, including he yaw rae, by means of kalman filering. 3

4 Niknejad, H. T. e al. (2012) have proposed muli-vehicle deecion and racking using vehicle mouned monocular camera. The feaures of vehicles are learned by Laen Suppor Vecor Machine(LSVM) and Hisograms of Oriened Gradiens (HOG). The deecion algorihm combines boh global and local feaures of he vehicle as a deformable objec model. Deeced vehicles are racked hrough a paricle filer. Sivaraman, S. e al., (2011) have proposed sereomonocular fusion approach o on-road localizaion and racking of vehicles. Uilizing a calibraed sereo-vision rig, he proposed approach combines monocular deecion wih sereo-vision for on road vehicle localizaion and racking for driver assisance. The sysem iniially acquires synchronized monocular frames and calculaes deph maps from he sereo rig. The sysem hen deecs vehicles in he image plane using an acive learning-based monocular vision approach. Using he image coordinaes of deeced vehicles, he sysem hen localizes he vehicles in realworld coordinaes using he calculaed deph map. The vehicles are racked boh in he image plane, and in real-world coordinaes. 3. PROPOSED WORK Our approach for racking of vehicles in raffic video based on model-based sraegy involves wo seps. In he firs sep, background is subraced and vehicles are deeced in frame. We subrac he background and obained candidae foreground objecs represened as foreground mask using our previous work (Arun Kumar, H.D. e al., 2015). The background subracion reduces compuaion ime and removes complex background. Afer obaining foreground mask of candidae objecs in frame, vehicles are deeced using Co-occurrence Hisograms of Oriened Gradien (Co-HOG) descripor. In he second sep, we consruc a vehicle model for each vehicle in frame based on shape and exure feaures exraced from he foreground mask of vehicle regions using Co-HOG and CS-LBP operaor. The vehicle model capures he variaion in vehicle scale, vehicle pose, and complex vehicles occlusion. Afer consrucing he vehicle model for he curren frame, he vehicle feaures are exraced from each deeced vehicle image and hen vehicle model is updaed. Finally, he vehicles are racked based on he similariy measure beween curren frame vehicles and vehicle model. Vehicle posiion is locaed by inegraing all maching in he vehicle model. Since, feaures may appear and disappear due o viewpoin changes and occlusions, our dynamic model is designed o be able o add a new feaure model and remove expired ones adapively and dynamically. The Hungarian algorihm is used o link deecions for racking. The flow diagram of proposed vehicle racking approach is shown in Figure 1. 4

5 3.1 Formulaion of proposed approach Figure 1. The flow diagram of our approach Traffic vehicle racking can generally be formulaed as a muli-variable esimaion problem. i Given he video sequence { V1, V2,..., V m } as inpu, we use S o indicae he sae of he h h i vehicle in he frame. We use S = ( S1,, S2,,..., SM ) o indicae he saes of all he h M vehicles in he frame, O { O }, i 1, 2,..., S = { S, S,..., S } o indicae he sequenial saes of he i,1: i,1 i,2 i, = = as all he deecions, h i vehicle from he firs frame o h he frame, and S1: = { S1, S2,..., S} o indicae all he sequenial saes of all he vehicles from h he firs frame o he frame. S = arg max P( S O ), (1) 1: 1: 1: The sae of he vehicle in he curren frame only depends on he sae of he vehicle in previous frame 1. When we process frame, only he racking in frame 1 and he image in he h frame I are involved in he calculaion. P( S O ) = P( S O, S ) = P( S O, I, S ) P( O I, S ) do, (2) 1: where P( O I, S 1) indicaes how realisic he deecion in frame is, P( S O, I, S 1) describes how well he deecions in frame maches racking s in frame 1. 5

6 3.2 Background Subracion and Vehicles Deecion In his secion, we inroduce he mehod for background subracion and vehicles deecion. The formulaion for background subracion (foreground deecion) and vehicles deecion process is P( O I, S ) defined in equaion (2), which is described as: 1 Pfg Pde if he caegory of objec is moving vehicle, P( O I, S 1) = (3) P oherwise. bg We subrac he background and obained he foreground mask of candidae moving objecs using our approach proposed in he previous work (Arun Kumar, H.D. e al. 2015), which reduces he compuaion ime and removes he complex background. Our previous approach has verified o be an efficien and effecive background subracion mehod. Our approach for background subracion uses modified SXCS-LBP exure descripor for finding foreground mask of candidae objecs, 1 if O is in foreground regions by background subracion, Pfg ( O I, S 1) = 0 oherwise. (4) Afer background subracion, each pixel is labelled as foreground or background using 0 or 1, 0 indicaes background and 1 indicaes foreground. Once he background subracion process is compleed, we obain foreground mask for each candidae objec. Once foreground mask of each objec is obained, i is necessary o deec vehicles among candidae objecs in he frame. The deecion of vehicles is described as P de. In order o deec vehicles, we uilize he approach proposed by Tomoki Waanabe, e al. (2009), in which Cooccurrence Hisograms of Oriened Gradien (Co-HOG) feaures are exraced o represen moving vehicles. P 1 if O is obained by he deecion, ( O I, S ) = (5) 0 oherwise. de Vehicle Tracking Afer deecing he vehicles in frame, we rack he moving vehicles in raffic video. There are wo subsecions. Firs, we iniialize he racking, in he second sep, we calculae he similariy beween racking and observaion, and hen observaion could be linked o being a racking Iniializaion of racking In order o iniialize racking, vehicle model is consruced for each deeced vehicle a ime. The appearance of he vehicle under racking may change over ime due o changes of vehicle scale, vehicle pose, complex vehicle occlusion, and he appearance variaion would lead o losing rack. Hence, vehicle model capures hese changes occurred in vehicle while i is moving. In he saring sage of he vehicle racking, he arge vehicle model in he firs frame is only iniialized and all he remaining vehicle models are empy. Generally, he vehicle model is updaed incremenally. Whenever a larger appearance variaion is deeced, he updaed arge vehicle models are sored in he empy models. 6

7 For M vehicles a ime, we consruc M vehicle models represened by a shape and exure feaures obained using Co-HOG and CS-LBP mehod respecively. In he following subsecions, we presen procedure involved in exracing he Co-HOG and CS-LBP feaures from each deeced vehicle image. Co-occurrence Hisogram of Oriened Gradiens (Co-HOG) Co-occurrence Hisogram of Oriened Gradiens (Co-HOG) (Tomoki Waanabe, e al., 2009) descripor is an exension of he original HOG shape descripor ha capures he spaial informaion of neighboring pixels. Insead of couning he occurrence of he gradien orienaion of a single pixel, gradien orienaions of wo or more neighboring pixels are considered. For each pixel in an image block, he gradien orienaions of he pixel pair formed by is neighbor and iself are examined. The Co-HOG has wo imporan meris. One is he robusness agains illuminaion variaion because gradien orienaions are compued from he local inensiy difference. The oher meri is he robusness agains deformaions because sligh shifs deformaions make small hisogram value changes. The co-occurrence marix expresses he disribuion of gradien orienaions a a given offse over an image. The combinaions of neighbor gradien orienaions can express shapes in deail. Mahemaically, a co-occurrence marix K is defined a an each block N Χ M of an image I, parameerized by an offse (x, y) as: K x, y ( i, j) 1 if I( p, q) = i and I( p + x, q + y) = j, (6) 0 oherwise. N M = p= 1 q= 1 We describe he process of Co-HOG calculaion as follows. Iniially, we compue gradien orienaions from an image by I y θ = arcan, I x (7) where arcan( ) reurns he inverse angen of he elemens in degrees. I y and I x are verical and horizonal gradien respecively calculaed by Gaussian filer. We label each pixel wih one of eigh discree orienaions. In our approach, all 0 0 o orienaions are spli up ino eigh orienaions per Then, we compue co-occurrence marices using Eq. (6). We used 31 offses, including a zero offse. In mos of oher applicaions, he auhors proceed by dividing an image ino a number of blocks and from each block exrac co-occurrence marices. We divide he vehicle image pach ino non overlapping blocks of size N Χ M, he co-occurrence marices are compued for each block. Finally, he componens of all he co-occurrence marices are concaenaed ino a vecor. We divide he vehicle image ino 2 Χ 4 (he accuracy of our approach is considerably beer han for oher number of blocks. Hence, in all he experimens, we divide he vehicle image ino 2 Χ 4 blocks) blocks and Co-HOGs a each block are compued. Figure 2 gives an illusraion of Co- HOG feaure descripor exracion process from a given vehicle image. 7

8 Figure 2. Illusraion of Co-HOG feaure descripor exracion process The dimension of Co-HOG feaure vecor for he given vehicle image is 15,424 when we divide he vehicle image ino 2 Χ 4 blocks. From one small region or block, Co-HOG obains 31 cooccurrence marices. A co-occurrence marix has 64 componens. Thus, Co-HOG obains ( ) (2 4) = 15,424 componens for each vehicle image. Cener Symmeric Local Binary Paern (CS-LBP) Heikkila, e al. (2002) have proposed a novel ineres region descripor called as Cener Symmeric Local Binary Paern (CS-LBP) descripor which is an exension of LBP exure operaor. The CS-LBP descripor has several advanages such as olerance o illuminaion changes, robusness on fla image areas, and compuaional efficiency. The CS-LBP compares he gray values of pairs of pixels in he cener-symmeric direcion. For 8 neighbors, LBP produces 256 differen binary paerns, whereas for CS-LBP produces only 16 binary paerns, Figure 3 gives an illusraion of CS-LBP feaure descripor compuaion process, and CS-LBP is mahemaically defined as follows: where g p and p p+ 2 p 1 2 p CS LBPR, P = S p 2, p g g p= 0 p+ (8) 2 g correspond o gray values of he cener-symmery pair of pixels and he funcion S( x) is hreshold funcion defined as follows: 1 x 0 S( x) = (9) 0 oherwise. Figure 3. LBP and CS-LBP feaures for neighborhood of 8 pixels 8

9 The concaenaed Co-HOG and CS-LBP feaure vecor of each vehicle represens vehicle model for each vehicle and concaenaed feaure vecor (fv) of each vehicle (O i ) is defined as N { hv j} j= 1 where N is he oal number of feaure vecor bins and fv( O ) =, (10) i hv j is he value of h j bin. For each rajecory, he vehicle model is updaed frame by frame. The appearance in he curren frame is he mos imporan one, so we updae as follows: ( α ) fv = k 1 1 fv + + k α fvc, (11) h where fvk is he feaure vecor afer updaing in k frame,α is a consan which is se o 0.8 and fv c is he feaure vecor calculaed in he curren frame. Each vehicle is racked based on minimum disance beween feaure vecor of he vehicle in he curren frame and is associaed vehicle model. The Hellinger disance is used o compare he feaure vecors N 1 d fv, fv = 1 hv hv, (12) ( ) 1 2 q,1 q,2 2 fv 1 1, fv q 2N = where fv1 and fv 2 are wo feaure vecors, and 1 fv = hv k = k N q, k, 1,2 (13) N q = Similariy beween racking and observaion We simplify P( S O, I, S 1) is probabiliy for a racking and a deecion, where ( ) P( S O, I, S 1) = Pa, (14) ( ( ), ) 2 1 d fv D fv Pa = exp, (15) 2 2πσ 2σ a a fv D is he feaure vecor of deeced vehicle in frame, fv is he feaure vecor of racking afer updaing in frame, d (.) is he funcion o calculae Hellinger disance, and σ a is a given hreshold. 4. EXPERIMENTAL RESULTS The performance evaluaion of he proposed mehod for vehicle-racking is a frame-by-frame evaluaion process. We carry ou experimens on challenging wo raffic surveillance video daases such as GRAM-Road Traffic Monioring (GRAM-RTM) (Guerrero-Gómez-Olmedo, e al., 2013) and i-lids. In order o measure he performance of our approach for vehicle-racking, we used evaluaion merics such as precision, recall, and Vehicle-Tracking Accuracy (VTA ) 9

10 (Smih, K. e al., 2005). The precision measures how much of he esimaes (ε is racker oupus are referred o as esimaes) cover he ground ruh (GT ) vehicle and can ake values beween 0 (no overlap) and 1 (full overlap). I is possible o have high precision wih poor qualiy racking as depiced in Figure 4(a). The recall measures how much of hegt is covered by hen ε and can ake values beween 0 (no overlap) and 1 (full overlap). I is possible o have a high recall ye have poor qualiy racking (Figure 4(b)). Vehicle-Tracking Accuracy (VTA ) is oal posiion error for maching vehicle hypohesis pair over all frames, averaged by he oal number of maches. The precision ( v i, j ), recall ( ρ i, j ), and VTA are defined as follows: v ε GT = (16) i j i, j, ε i ρ ε GT = (17) i j i, j, GTj ( M + FP + MM ) ( GT ) ( ) VTA = 1 = 1 M + FP + MM, (18) where GT j is he ground ruh for racking arge vehicles and indexed by j, ε i is he racker oupu are referred o as esimaes and indexed by i, M is he number of missed vehicles a ime. FP is he number of false posiives ha correspond o deec vehicles and ha do no overlap any real vehicles in he scene. MM is he number of mismaches a ime. number of vehicles a ime. M, FP and MM represen he corresponding raio. GTj is he oal Figure 4. a) precision ( v ) b) recall ( ρ ) c) boh precision and recall should have high values There are four basic ypes of errors ha our sysem can make. The firs ype of error may occur when a vehicle exiss, bu he sysem does no recognize i (False Negaive: A ground ruh objec exiss ha is no associaed wih an esimae). The second ype of error occurs when he sysem may indicae he presence of a vehicle which does no exis (False Posiive: An esimae exiss ha is no associaed wih a ground ruh objec). The hird ype of error occurs when one vehicle is racked by muliple esimaes (Muliple Trackers: Two are more esimaes are associaed wih he same ground ruh). The las ype of error occurs when muliple vehicles are racked by one esimae (Muliple Objecs: Two or more ground ruh objecs are associaed wih he same esimae). These errors are depiced in Figure 5. 10

11 Figure 5. Types of configuraion errors ε s (1, 2,3, 4) aemp o rack four GT s ( a, b, c, d ) 4.1. Experimens on GRAM-RTM daase In he firs se of experimens, we evaluaed he performance of our approach quaniaively using GRAM Road-Traffic Monioring (GRAM-RTM) daase. The GRAM Road-Traffic Monioring daase consiss of hree raffic video sequences and hese video sequences were recorded under differen condiions and wih differen plaforms. The firs video sequence is M-30 video (7529 frames), was recorded on a sunny day and resoluion for each frame is 800 Χ 480. The second video sequence is M-30-HD (9390 frames), was recorded in he same locaion, bu cloudy day and resoluion for each frame is 1200 Χ 720. The hird video sequence Urban1 (2345 frames), was recorded a a busy urban inersecion and he resoluion of each frame is 600 Χ 360. The ground ruhs of hese hree video sequences were manually obained. We compared he performance of our approach wih CS-LBP alone (Texure descripor) and Co-HOG alone (Shape Descripor). Table 1 shows he comparaive sudy of our approach (CS-LBP + Co-HOG) wih CS-LBP alone and CO-HOG alone. All of he values represen he average of he considered crierion obained for he whole hree M-30, M-30-HD, and Urban1 video sequences. I is observed ha our approach achieves he highes precision and highes recall compared o CS-LBP alone and CO- HOG alone for M-30, M-30-HD, and Urban1 video sequences. The VTA of our approach is 89%, 88% and 81% for he hree challenging video sequences, which means ha he racking of almos all of he vehicles is deeced. The low FP level of our approach (03%, 02%, and 06%) shows ha almos all of he deeced racking s correspond o a real vehicle in he scene. 03% of he vehicles are oally missing from he M-30 video sequence, while 04% are missing from he M-30-HD video sequence, and 06% are missing from he Urban1 video sequence. The increase in FP for Urban1 video sequence is mainly due o some pedesrians locaed a he scene of he ROI. Our approach decreases he mismach rae (03%, 02% and 06% for respecive video sequences) compared o shape and a exure descripor alone. The combinaion of CS-LBP and Co-HOG improves he VTA, precision, recall and reduce false deecion, mismach and missed vehicles. Figure 6 shows he qualiaive performance of our approach for all hree GRAM Road-Traffic Monioring daase video sequences. The red color windows in he sample video frames describe racked vehicles. 11

12 Table 1. The resuls comparison of our approach wih CS-LBP alone and Co-HOG alone based on precision, recall and VTA for M-30, M-30-HD, and Urban1 video sequences Experimens on i-lids daase In he second se of experimens, we evaluaed he performance of our approach quaniaively using i-lids daase. The i-lids daase consiss of seven raffic video sequences, among seven video sequences, we seleced AVSS PV Easy video sequence which includes scenes of vehicle urning, illuminaion changes, and vehicles moving from far o near. The resoluion for each frame is 720 Χ 576. The ground ruhs of his video sequence were manually obained. Table 2 shows he precision, recall, and VTA based quaniaive comparison of our approach resul wih CS-LBP alone and Co-HOG alone for he AVSS PV easy video sequence. The proposed approach (CS-LBP + Co-HOG) has achieved highes precision, recall and VTA compared o CS- LBP alone and Co-HOG alone. The FP rae of our approach is very high because AVSS PV easy video sequence conains some pedesrians, moorcycles, and vehicles moving from far o near. Figure 7 shows he racking resuls of our approach for sample frames of AVSS PV easy i-lids daase. The firs row of Figure 7 shows sample original video frames where vehicles pose are changing because of curved lane. The second row shows racking resul represened using red color window. I is observed ha our approach accuraely rack he vehicles even hough vehicle pose changes. This is because, in each frame, he vehicle model is updaed efficienly in order o capure he variaion in pose of he vehicles. 12

13 Figure 6. Vehicle racking resul of our approach for all hree video sequences of GRAM Road-Traffic Monioring daase a) M-30 b) M-30-HD, and c) Urban1(firs row is exclusion area shown using red color). Table 2. The resuls comparison of our approach wih CS-LBP alone and Co-HOG alone based on precision, recall and VTA for AVSS PV Easy video sequence. 13

14 Figure 7. Vehicle racking resul of our approach for AVSS PV easy video sequence 4.3. Visual comparison wih exising mehod We compared racking resuls of our approach obained on i-lids daase visually wih he racking resuls of SIFT-based Mean Shif algorihm proposed by Liang e al. (2014). The firs row of Figure 8 shows sample original video frames, such as Frame #26, Frame #55, Frame #75, and Frame #85. The second and hird row shows he resuls of Mean Shif mehod and our proposed approach resuls for he sample frames. The racking window of Mean shif mehod deviaes a he Frame #55, Frame #75 and Frame #85. This is because he color hisogram of he candidae emplae is changed when he moving vehicle is urning lef, and he illuminaion is affeced by he shadow of he building. For Mean Shif algorihm, when he illuminaion and shape of he vehicle changes, he number of mached poins grealy increases and he performance of he mehod decreases, which records false racking rae. Our approach resuls presened in he hird row demonsrae ha he moving vehicles are racked more accuraely for he Frame #26, Frame #55, Frame #75, and Frame #85. The increase in racking rae of our approach is due o he fac ha adapaion of CS-LBP descripor, which is illuminaion invarian and exracs accurae vehicles exure feaures, and he Co-HOG descripor gives accurae shape feaures even hough he vehicle changes is pose. Hence, he combinaion of shape and exure descripors increases he vehicle racking resul. 14

15 Figure 8. Vehicle racking resul for AVSS PV Easy video sequences for Mean Shif (red), and proposed mehod (yellow) a) Frame #26, b) Frame #55, c) Frame #75, and d) Frame # CONCLUSION In his paper, we proposed a model-based vehicle racking echnique using spaial local feaures such as shape and exure feaures. The shape descripor such as Co-HOG is used for he represenaion of vehicle shape and CS-LBP exure descripor is used for represenaion of vehicle exure. The vehicle model is consruced which capures he variaion in illuminaion, vehicle scale, vehicle pose and complex vehicles occlusion. The evaluaion process conduced on wo popular daases such as GRAM-RTM and i-lids demonsrae ha our approach achieves highes vehicle racking accuracy. The visual comparison wih exising mehod shows ha our approach yields accurae racking even vehicle pose and illuminaion changes. The drawback of our approach is ha when he pedesrians and moorcycles are presen in he video sequence, our approach racks hese objecs as vehicles and i reduces he racking accuracy. REFERENCES [1] Pérez, P., Hue, C., Vermaak, J., & Gangne, M. (2002), Color-based probabilisic racking, In Compuer vision ECCV 2002 (pp ). Springer Berlin Heidelberg. [2] Avidan, S. (2007), Ensemble racking, Paern Analysis and Machine Inelligence, IEEE Transacions on, 29(2), [3] Ross, D. A., Lim, J., Lin, R. S., & Yang, M. H. (2008), Incremenal learning for robus visual racking, Inernaional Journal of Compuer Vision, 77(1-3), [4] Grabner, H., & Bischof, H. (2006, June), On-line boosing and vision, In Compuer Vision and Paern Recogniion, 2006 IEEE Compuer Sociey Conference on (Vol. 1, pp ). IEEE. [5] Grabner, H., Grabner, M., & Bischof, H. (2006, Sepember), Real-Time Tracking via On-line Boosing, In BMVC (Vol. 1, No. 5, p. 6). [6] Wang, S., Lu, H., Yang, F., & Yang, M. H. (2011, November), Superpixel racking, In Compuer Vision (ICCV), 2011 IEEE Inernaional Conference on(pp ). IEEE. 15

16 [7] Beymer, D., McLauchlan, P., Coifman, B., & Malik, J. (1997, June), A real-ime compuer vision sysem for measuring raffic parameers, In Compuer Vision and Paern Recogniion, Proceedings., 1997 IEEE Compuer Sociey Conference on (pp ). [8] Su, X., Khoshgofaar, T. M., Zhu, X., & Folleco, A. (2007), Rule-based muliple objec racking for raffic surveillance using collaboraive background exracion, In Advances in Visual Compuing (pp ). Springer Berlin Heidelberg. [9] Jung, Y. K., & Ho, Y. S. (2001), A feaure-based vehicle racking sysem in congesed raffic video sequences, In Advances in Mulimedia Informaion Processing PCM 2001 (pp ). Springer Berlin Heidelberg. [10] Babaei, P. (2010, December), Vehicles racking and classificaion using raffic zones in a hybrid scheme for inersecion raffic managemen by smar cameras, In Signal and Image Processing (ICSIP), 2010 Inernaional Conference on (pp ). IEEE. [11] Gao, T., Liu, Z. G., Gao, W. C., & Zhang, J. (2008), Moving vehicle racking based on SIFT acive paricle choosing, In Advances in Neuro-Informaion Processing (pp ). Springer Berlin Heidelberg. [12] Dahlkamp, H., Pece, A. E., Olik, A., & Nagel, H. H. (2004), Differenial analysis of wo modelbased vehicle racking 0020 approaches, In Paern Recogniion (pp ). Springer Berlin Heidelberg. [13] Ha, S. W., & Moon, Y. H. (2011), Muliple objec racking using SIFT feaures and locaion maching, Inernaional Journal of Smar Home, 5(4), [14] Yu, Q., Dinh, T. B., & Medioni, G. (2008), Online racking and reacquisiion using co-rained generaive and discriminaive rackers, In Compuer Vision ECCV 2008 (pp ). Springer Berlin Heidelberg. [15] Tran, S., & Davis, L. (2007, Ocober), Robus Objec Tracking wih Regional Affine Invarian Feaures, In Compuer Vision, ICCV IEEE 11h Inernaional Conference on (pp. 1-8). IEEE. [16] He, W., Yamashia, T., Lu, H., & Lao, S. (2009, Sepember), Surf racking, In Compuer Vision, 2009 IEEE 12h Inernaional Conference on (pp ). IEEE. [17] Liu, X., Lin, L., Yan, S., Jin, H., & Jiang, W. (2011), Adapive objec racking by learning hybrid emplae online, Circuis and Sysems for Video Technology, IEEE Transacions on, 21(11), [18] Cehovin, L., Krisan, M., & Leonardis, A. (2013), Robus visual racking using an adapive coupled-layer visual model, Paern Analysis and Machine Inelligence, IEEE Transacions on, 35(4), [19] Kwon, J., & Lee, K. M. (2009, June), Tracking of a non-rigid objec via pach-based dynamic appearance modeling and adapive basin hopping Mone Carlo sampling, In Compuer Vision and Paern Recogniion, CVPR IEEE Conference on (pp ). IEEE. [20] Rad, R., & Jamzad, M. (2005), Real ime classificaion and racking of muliple vehicles in highways, Paern Recogniion Leers, 26(10), [21] Ma, C., Liu, C., Peng, F., & Liu, J. (2016), Muli-feaure Hashing Tracking, Paern Recogniion Leers, 69, [22] Nguyen, P. V., & Le, H. B. (2008), A muli-modal paricle filer based moorcycle racking sysem, In PRICAI 2008: Trends in Arificial Inelligence (pp ). Springer Berlin Heidelberg. [23] Babaei, P., Fahy, M., & Berangi, R. (2013), Consisen Vehicles Tracking By Using A Cooperaive Disribued Video Surveillance Sysem, Inernaional Research Journal of Applied and Basic Sciences, 4(10), [24] Arróspide, J., Salgado, L., Nieo, M., & Jaureguizar, F. (2008, Ocober), On-board robus vehicle deecion and racking using adapive qualiy evaluaion, In h IEEE Inernaional Conference on Image Processing. IEEE. [25] Niknejad, H. T., Takahashi, K., Mia, S., & McAlleser, D. (2011, June), Embedded muli-sensors objecs deecion and racking for urban auonomous driving, In Inelligen Vehicles Symposium (IV), 2011 IEEE (pp ). IEEE. [26] Messelodi, S., Modena, C. M., & Zanin, M. (2005), A compuer vision sysem for he deecion and classificaion of vehicles a urban road inersecions, Paern analysis and applicaions, 8(1-2),

17 [27] Srigel, E., Meissner, D., & Diemayer, K. (2013, June), Vehicle deecion and racking a inersecions by fusing muliple camera views, In Inelligen Vehicles Symposium (IV), 2013 IEEE (pp ). IEEE [28] Zheng, Y., & Peng, S. (2012, Sepember), Model based vehicle localizaion for urban raffic surveillance using image gradien based maching, In Inelligen Transporaion Sysems (ITSC), h Inernaional IEEE Conference on (pp ). IEEE. [29] Lee, K. H., Hwang, J. N., & Chen, S. I. (2015), Model-Based Vehicle Localizaion Based on 3-D Consrained Muliple-Kernel Tracking, Circuis and Sysems for Video Technology, IEEE Transacions on, 25(1), [30] Kim, G., Kim, H., Park, J., & Yu, Y. (2011), Vehicle racking based on kalman filer in unnel, In Informaion Securiy and Assurance (pp ). Springer Berlin Heidelberg. [31] Barh, A., & Franke, U. (2010, Sepember), Tracking oncoming and urning vehicles a inersecions, In Inelligen Transporaion Sysems (ITSC), h Inernaional IEEE Conference on (pp ). IEEE. [32] Niknejad, H. T., Takeuchi, A., Mia, S., & McAlleser, D. (2012), On-road mulivehicle racking using deformable objec model and paricle filer wih improved likelihood esimaion, Inelligen Transporaion Sysems, IEEE Transacions on, 13(2), [33] Sivaraman, S., & Trivedi, M. M. (2011, Ocober), Combining monocular and sereo-vision for real-ime vehicle ranging and racking on mulilane highways In h Inernaional IEEE Conference on Inelligen Transporaion Sysems (ITSC) (pp ). IEEE. [34] Arun Kumar, H. D., Prabhakar, C. J. (2015), Moving Vehicles Deecion in Traffic Video Using Modified SXCS-LBP Texure Descripor, Inernaional Journal of Compuer Vision and Image Processing (IJCVIP), Vol 5, No. 2, pp [35] Waanabe, T., Io, S., & Yokoi, K. (2009), Co-occurrence hisograms of oriened gradiens for pedesrian deecion, In Advances in Image and Video Technology (pp ). Springer Berlin Heidelberg. [36] Heikkilä, M., & Pieikäinen, M. (2002), A exure-based mehod for modeling he background and deecing moving objecs, Paern Analysis and Machine Inelligence, IEEE Transacions on, 28(4), [37] Guerrero-Gómez-Olmedo, R., López-Sasre, R. J., Maldonado-Bascón, S., & Fernández-Caballero, A. (2013, June), Vehicle racking by simulaneous deecion and viewpoin esimaion, In Inernaional Work-Conference on he Inerplay Beween Naural and Arificial Compuaion (pp ). Springer Berlin Heidelberg. [38] Smih, K., Gaica-Perez, D., Odobez, J. M., & Ba, S. (2005, June), Evaluaing muli-objec racking, In 2005 IEEE Compuer Sociey Conference on Compuer Vision and Paern Recogniion (CVPR'05)-Workshops (pp ). IEEE. [39] i-lids: hp:// Auhors Arun Kumar H. D. Received M.Sc. degree in compuer Science from Kuvempu Universiy, Karnaaka, India in 2009, He is pursuing Ph.D. degree in Kuvempu Universiy, Karnaaka, India. His research ineress are image and video processing, Compuer Vision and Machine Vision Prabhakar C.J. received Ph.D. degree in Compuer Science in he year 2009 from Gulbarga Universiy, Gulbarga, Karnaaka, India. He is currenly working as Assisan Professor in he deparmen of Compuer Science, Kuvempu Universiy, Karnaaka, India. His research ineress are compuer vision, Image and video processing. 17