Dynamic Camera Assignment and Handoff

Transcription

1 12 Dynamc Camera Assgnment and Handoff Br Bhanu and Ymng L 12.1 Introducton Techncal Approach Motvaton and Problem Formulaton Game Theoretc Framework Computaton of Utltes Barganng among Cameras Expermental Results Data Crtera for Camera Assgnment and Handoff Evaluaton Measurements Analyss for Experments wth Dfferent Crtera Convergence of Results for Barganng Comparson wth Another Related Approach Summary Bblographcal and Hstorcal Remarks References K10681_C012.ndd 337 8/13/ :48:36 AM

2 338 Intellgent Vdeo Survellance: Systems and Technology 12.1 INTRODUCTION Due to the broad coverage of an envronment and the possblty of coordnaton among dfferent cameras, vdeo sensor networks have attracted much nterest n recent years. Although the feld-of-vew (FOV) of a sngle camera s lmted and the cameras may have overlappng or nonoverlappng FOVs, seamless trackng of movng objects can be acheved by explotng the handoff capablty of multple cameras. Camera handoff s the process of fndng the next best camera to see the target object when t s leavng the FOV of the current camera that s beng used to track t. Ths wll provde a better stuaton assessment of the envronment under survellance. Multple cameras enable us to have dfferent vews of the same object at the same tme, such that we can choose one or some of them to montor a gven envronment. Ths can help to solve the occluson problem to some extent, as long as the FOVs of the cameras have some overlaps. However, snce multple cameras may be nvolved over long physcal dstances, we have to deal wth the handoff problem as well. It s evdent that the manual camera handoff wll become unmanageable when the number of cameras s large. Therefore, we need to develop survellance systems that can automatcally carry out the camera assgnment and handoff task. In ths chapter, we provde a new perspectve to the camera handoff problem that s based on game theory. The mert of our approach s that t s ndependent of the topology of how the cameras are placed. When multple cameras are used for trackng and where multple cameras can see the same object, the algorthm can automatcally provde an optmal as well as stable soluton of the camera assgnment quckly. Snce game theoretc approach allows dealng wth multple crtera optmzaton, we are able to choose the best camera based on multple crtera that are selected a pror. The detaled camera calbraton or 3D scene understandng s not needed n our approach. Our approach dffers from the earler tradtonal approaches. We propose a game theoretc approach for camera assgnment and handoff usng the vehcle-target model (Arslan et al. 2007). We model camera assgnment and handoff as a multplayer game and allow for both coordnaton and confl cts among these players. Multple crtera, whch are used to evaluate the trackng performance, are used n the K10681_C012.ndd 338 8/13/ :48:37 AM

3 Dynamc Camera Assgnment and Handoff 339 utlty functons for the objects beng traced. The equlbrum of the game provdes the soluton of the camera assgnment. The followng sectons are devoted to descrbng the proposed game-theory-based method. Secton 12.2 formulates the camera assgnment and handoff problem and then constructs the utltes and barganng steps. The mplementaton of ths approach and some expermental results are also shown n ths secton. The conclusons are gven n Secton Some related works n ths area are revewed n Secton TECHNICAL APPROACH Motvaton and Problem Formulaton Game theory can be used for analyzng the nteractons as well as confl cts among multple agents. Analogously, n a vdeo sensor network, collaboratons as well as compettons among cameras exsts smultaneously. Ths concept enlghtens us to vew the camera assgnment problem n a game theoretc manner. The nteractve process s called a game, whle all the partcpants of the game are called players, who strve to maxmze ther utltes. In our problem for each person to be tracked, there exsts a multplayer game, wth the avalable cameras beng the players. If there are multple persons n the system, ths becomes a multple of multplayer game beng played smultaneously. Vehcle-target assgnment (Arslan et al. 2007) s a multplayer game that ams to allocate a set of vehcles to a group of targets and acheve an optmal assgnment. Vewng the persons beng tracked as vehcles whle the cameras as targets, we can adopt the vehcle-target assgnment model to choose the best camera for each person. In the followng, we propose a game-theory-based approach that s well suted to the task at hand Game Theoretc Framework Game theory nvolves utlty, whch refers to the amount of welfare an agent derves n a game (Myerson 1991, L and Bhanu 2008). We are concerned wth three dfferent utltes: K10681_C012.ndd 339 8/13/ :48:37 AM

4 340 Intellgent Vdeo Survellance: Systems and Technology 1. Global utlty: The overall degree of satsfacton for trackng performance. 2. Camera utlty: How well a camera s trackng the persons assgned to t based on the user-suppled crtera. 3. Person utlty: How well the person s satsfed wth beng tracked by the current selected camera. Our objectve s to maxmze the global utlty as well as to make sure that the best camera tracks each person. Movng objects are detected n multple vdeo streams. Ther propertes, such as the sze of the mnmum boundng rectangle and other regon propertes (color, shape, vew, etc.) are computed. Varous utltes (camera utlty, person utlty, predcted person utlty, and global utlty) are computed based on the user-suppled crtera and barganng process among avalable cameras are executed based on the predcton of person utltes (the so-called predcted person utlty) n each step. The results obtaned from the strategy executon are n turn used for updatng the camera utltes and person utltes untl the strateges converge. Fnally, those cameras wth the hghest converged probabltes wll be used for trackng and ths assgnment of persons to the best cameras leads to the soluton of the handoff problem n multple vdeo streams. A set of symbols are used n the dscusson for our approach and ther descrptons are gven n Table Computaton of Utltes We frst defne the followng propertes of our system: 1. A person P can be n the FOV of more than one camera. The avalable cameras for P belong to the set A. C 0 s assumed to be a vrtual (null) camera. 2. A person can only be assgned to one camera. The assgned camera for P s named as a. 3. Each camera can be used for trackng multple persons. For some person P, when we change ts camera assgnment from a to a whle assgnments for other persons reman the same, f K10681_C012.ndd 340 8/13/ :48:37 AM

5 Dynamc Camera Assgnment and Handoff 341 TABLE 12.1 Notatons of Symbols Used n the Chapter Symbols Notatons P Person C j Camera j N P Total number of persons n the entre network at a gven tme N C Total number of cameras n the entre network at a gven tme A The set of cameras that can see person, A = {a 1, a 2,, a n } C n C Number of cameras that can see object, number of elements n A n P Number of persons currently assgned to camera C j a The assgned best camera for person a The assgnment of cameras for the persons excludng person a Assgnment of cameras for all persons, a = (a, a ) UC ( a) j Camera utlty for camera j UP ( a) Person utlty for person U g (a) Global utlty 1 l UP ( k) Predcted person utlty for person at step k, UP( k) = [ UP( k),, U P nc T l, UP ( k)], where U P s the predcted person utlty for P f camera a l s used p (k) Probablty of person s assgnment at step k, pk ( ) = [ pk ( ) = [ p 1 ( k), l n, ( ),, C l p k p ( k)], where pk ( ) s the probablty for camera a l to track person P U ( a, a ) < U ( a, a ) U ( a, a ) < U ( a, a ) (12.1) P P g g the person utlty U P s sad to be algned wth the global utlty U g, where a stands for the assgnments for persons other than P, that s, a = ( a 1,, a 1, a+ 1,, an P ). So, we can also express a as a = (a, a ). We defne the global utlty as Ug( a) = UC ( a ) (12.2) j Cj C where UC j ( a ) s the camera utlty and defned to be the utlty generated by all the engagements of persons wth a partcular camera. Now, we defne the person utlty as U ( a) = U ( a, a ) U ( C, a ) = U ( a, a ) U ( C, a ) (12.3) P g g 0 Cj Cj 0 K10681_C012.ndd 341 8/13/ :48:37 AM

6 342 Intellgent Vdeo Survellance: Systems and Technology The person utlty UP j ( a ) can be vewed as a margnal contrbuton of to the global utlty, that s, the dfference of the utlty ganed by assgnng camera to track person compared wth s tracked by a vrtual camera (no camera). To calculate (12.3), we have to construct a scheme to calculate the camera utlty UC j ( a ). We assume that there are N Crt crtera to evaluate the qualty of a camera used for trackng an object. Thus, the camera utlty can be bult as Cj n N P Crt = U ( a, a ) Cr (12.4) s= 1 l= 1 where n P s the number of persons that are currently assgned to camera for trackng. Pluggng (12.4) nto (12.3) we can obtan P Crt U ( a, a ) = Crt Cr (12.5) Pj sl n N P Crt s= 1 l= 1 s= 1 l= 1 s P n N where s P means that we exclude person from the those who are beng tracked by camera C j. One thng to be notced here s that when desgnng the crtera, we have to normalze them Barganng among Cameras As stated prevously, our goal s to optmze each person s utlty as well as the global utlty. Competton among cameras fnally leads to the Nash equlbrum. Unfortunately, ths Nash equlbrum may not be unque. Some of them are not stable solutons, whch are not desred. To solve ths problem, a barganng mechansm among cameras s ntroduced, to make them fnally come to a compromse and generate a stable soluton. When barganng, the assgnment n the kth step s made accordng to a set of probabltes pk p k pk p k 1 l n ( ) = [ ( ),, ( ),, C ( )] K10681_C012.ndd 342 8/13/ :48:38 AM

7 Dynamc Camera Assgnment and Handoff 343 where n C s the number of cameras that can see the person and n 1 C l p ( k ) = 1, wth each 0 p l ( k) 1, l = 1,, nc. We can generalze p (k) to be pk p k pk p k 1 l NC ( ) = [ ( ),, ( ),, ( )] by assgnng a zero probablty for those cameras that cannot see the person, meanng that those cameras wll not be assgned accordng to ther probablty. Thus, we can construct an N P N C probrablty matrx 1 NC p1 ( k) p1 ( k) 1 NC pn ( k) p ( ) P N k P At each barganng step, we wll assgn a person to the camera that has the hghest probablty. Snce n most cases a person has no nformaton of the assgnment before t s made, we ntroduce the concept of predcted person utlty UP ( k ): Before we decde the fnal assgnment profle, we predct the person utlty usng the prevous person s utlty nformaton n the barganng steps. As shown n (12.5), person utlty depends on the camera utlty, so, we predct the person utlty for every possble camera that may be assgned to track t. Each element n UP ( k ) s calculated by (12.6) U l P l 1 l l UP( k) + ( UP( ak ( ))) UP( k), a( ) = k A l ( k+ 1) p ( k) = U l P ( k), otherwse (12.6) l wth the ntal state U P (1) to be assgned arbtrarly as long as t s j wthn the reasonable range for UP ( k ), for l = 1,, n C. Once these predcted person utltes are calculated, t can be proved that the equlbrum for the strateges les n the probablty dstrbuton that maxmzes ts perturbed predcted utlty (Arslan et al. 2007), T Pk ( ) U ( k) +τ H(( pk ) ) (12.7) P K10681_C012.ndd 343 8/13/ :48:38 AM

8 344 Intellgent Vdeo Survellance: Systems and Technology where T Hpk ( ( )) = pk ( ) log( pk ( )) (12.8) s the entropy functon and τ s a postve parameter belongng to [0,1] that controls the extent of randomzaton. The larger the τ s, the faster the barganng process converges; the smaller the τ s, the more accurate result we can get. So, there s a trade-off when selectng the value of τ and we select τ to be 0.5 n our experments. The soluton of (12.7) s proved (Arslan et al. 2007) to be l p ( k) = e e l (1/ τ) UP ( k) l nc (1/ τ) UP ( k) (1/ τ) UP ( k) + + e (12.9) After several steps of calculaton, the result of p tends to converges (refer to Fgure 12.8). Thus, we fnally get the stable soluton, whch s proved to be at least suboptmal (Arslan et al. 2007). Ths overall algorthm s summarzed n Algorthm as follows: Algorthm: Game Theoretc Camera Assgnment and Handoff Input: Multple vdeo streams. Output: A probablty matrx accordng to whch camera assgnments are made. Algorthm descrpton: At a gven tme, perform moton detecton and get the selected propertes for each person that s to be tracked. For each person and each camera, decde whch cameras can see a gven person P. For those that can see the person P, ntalzed the predcted person utlty vector U P (1). Repeat 1. Derve the Crt sl for each avalable camera. 2. Compute the camera utltes UC ( a ) by (4). j 3. Compute the person utltes UP ( a ) by (5). 4. Compute the predcted person utltes UP ( k ) by (6). 5. Derve the strategy by P (k) usng (9). Untl the strateges for assgnment converge. Do the camera assgnment and handoff based on the converged strateges. K10681_C012.ndd 344 8/13/ :48:39 AM

9 Dynamc Camera Assgnment and Handoff Expermental Results Data In ths secton, we test the proposed approach for both a sngle person and two persons who are walkng through three Axs 215 PTZ cameras. The experments are carred out wth no camera calbraton needed. The cameras are placed arbtrarly. An llustraton for the camera confguraton for the experment s shown n Fgure To fully test whether the proposed approach can help to select the best camera based on the user-suppled crtera, some of the FOVs of these cameras are allowed to ntersect ntentonally whle some of them are nonoverlappng. Ths s mportant for trackng varous people n a camera network. A person observer selects the walkng person manually when he frst enters the FOV of a camera and detected by background subtracton around the edges when he leaves and re-enters the FOV (we suppose that there are no doors n the FOVs). The trackng s done by usng the Contnuous Adaptve Mean-Shft Algorthm proposed by Bradsk Camera 3 Camera 2 Camera 1 Fgure 12.1 Camera confguraton n our experments. K10681_C012.ndd 345 8/13/ :48:39 AM

10 346 Intellgent Vdeo Survellance: Systems and Technology n Bradsk (1998). Dfferent persons are dentfed by calculatng the correlaton of the hue hstograms of the pxels nsde ther boundng boxes usng the OpenCV functon CompareHst ( garage.com/wk/cvreference). In our experment, we compare the colors of the upper bodes (around 1/2 sze of the boundng box) frst, when the colors of the upper bodes are smlar, then, we contnue to compare the colors of the lower body and dentfy persons by the color combnaton of the upper body and the lower body. For the sake of prvacy, we perform only smulatons for large numbers of cameras and persons nstead of dong the experments wth real data Crtera for Camera Assgnment and Handoff A number of crtera, ncludng human bometrcs, can be used for camera assgnment and handoff. For easer comparson between the computed results and the ntutve judgment, four crtera are used for a camera selecton: 1. The sze of the tracked person. It s measured by the rato of the number of pxels nsde the boundng box of the person to that of the sze of the mage plane. Here, we assume that nether a too large nor a too small object s convenent for observaton. Assume that λ s the threshold for best observaton, that s, when r = λ ths crteron reaches ts peak value, where r = # of pxels nsde the boundary box. # of pxels nsde the mage plane Crt 1 1 λ r, when r < λ = 1 r 1, when r 1 λ λ (12.10) 2. The poston of the person n the FOV of a camera. It s measured by the Eucldean dstance that a person s away from the center of the mage plane K10681_C012.ndd 346 8/13/ :48:40 AM

11 Dynamc Camera Assgnment and Handoff 347 Crt 2 = ( x x ) + ( y y ) xc + yc c c (12.11) where (x, y) s the current poston of the person and (x c, y c ) s the center of the mage plane. 3. The vew of the person, as measured by the rato of the number of pxels on the detected face to that of the whole boundng box, whch s smlar to crteron 1. We assume that the threshold for best frontal vew s,.e. when R = ξ the vew of the person s the # of pxels on the face best, where R =. # of pxels on the entre body Crt 3 1 ξ r, when R < ξ = 1 R 1, when R 1 ξ ξ (12.12) 4. Combnaton of crtera 1, 2, and 3, whch s called the combned crteron s gven by the followng equaton, Crt = 3 4 m m m= 1 where w m are the weghts for dfferent crtera. w Crt (12.13) It s to be notced that all these crtera are normalzed for calculatng the correspondng camera utltes. In our experments, we gve value to the parameters emprcally. λ= (1/15), ξ= (1/6), w1= 0.2, w 2= 0.1, and w 3 = 0.7. Ths s because we prefer a frontal vew of the person whenever t s avalable, whle a contnuous trackng s the bottom lne when the frontal vew s not detected. K10681_C012.ndd 347 8/13/ :48:41 AM

12 348 Intellgent Vdeo Survellance: Systems and Technology Evaluaton Measurements In our experments, the bottom lne s to track walkng persons seamlessly whenever they appear n the FOV of any of the cameras. In the case where more than one camera can see the persons, those ones that can see the persons face are always the most preferable. Based on ths goal, f we defne the error n our experments as ether falng to track a person or falng to get the frontal-vew of the person whenever t s avalable. The performances for usng crteron 1, crteron 2, or crteron 3 alone n a two-person experment are 25.56%, 10.00%, and 30.00% respectvely. We can notce that based on our error defnton, sngle crteron ncurs a hgh error rate. Hence, we ntroduce the combned crteron to overcome ths error rate Analyss for Experments wth Dfferent Crtera A sngle-person case Fgure 12.2 gves the camera handoffs based on the combned crteron n a sngle-person experment. The camera wth a dark boundng box s the one to be chosen. As shown n Fgure 12.2, a frontal-vew person (whenever t s avalable) s selected for most of the frames. Sometmes, the frontal vew s not selected because the face s not detected. In some frames, such as n Fgure 12.2b, although the frontal vew s avalable, the person s too close to the edge of the mage or the sze of the person s far from the good sze threshold. In ths case, the system wll choose some other avalable camera. All the handoffs and nterestng events are lsted n Table 12.2, where we use E denotng that the person s enterng the FOV of a camera whle L denotng that the person s leavng the FOV of a camera. A mples that the camera can see the object and, thus, t s avalable for trackng, whle N mples that there s no object n the FOV of a camera. The last column n Table 12.2, Used, gves the camera that s selected to track the person. There are altogether 600 frames. It shows that camera handoff s carred out correctly especally when the person s enterng or leavng the FOV of some cameras. K10681_C012.ndd 348 8/13/ :48:41 AM

13 Dynamc Camera Assgnment and Handoff 349 (a) (b) (c) (d) (e) (f) Camera 1 Camera 2 (g) Camera 3 Fgure 12.2 (See color nsert followng page xxx.) Camera assgnments and handoffs n the 1 person 3 cameras case. The camera n whch the boundng box s the dark color s selected to track the person. K10681_C012.ndd 349 8/13/ :48:41 AM

14 350 Intellgent Vdeo Survellance: Systems and Technology TABLE 12.2 Handoffs among 3 Cameras durng 20 s for the 1 Person 3 Cameras Case Frame Camera 1 Camera 2 Camera 3 Used 1 A A N 1 69 A A E 1 88 A A A A A A L A A E A A A L A A N L A N E A E A A A A A A A L A A E A A A A L A A N END Note: A, avalable; E, enterng; L, leavng; N, not avalable. AQ1 A more detaled dscusson for choosng dfferent crtera s analyzed n the two person case as dscussed n the followng text. A two-person case Dfferent experments are carred out to compare the results for usng the three dfferent sngle crtera mentoned prevously wth the combned crteron. The weghts we use to combne the three crtera, n our experments, are 0.2, 0.1, and 0. 7 respectvely, as stated prevously. To make t convenent for a comparson, we show the trackng results of other cameras as well, no matter whether they are selected for trackng or not. Smlar to the sngle person experment, the cameras, for whch the boundng boxes are drawn n dark color, are selected for trackng whle the ones n the lght color are not as good as the dark ones. A comparson for usng crteron 1, crteron 2, and crteron 3 respectvely at two tme nstants s shown n Fgure We can K10681_C012.ndd 350 8/13/ :48:42 AM

15 Dynamc Camera Assgnment and Handoff 351 (a) (d) (b) (e) (c) (f) Fgure 12.3 (See color nsert followng page xxx.) A comparson for usng dfferent crtera. The left column and the rght column are for two tme nstants respectvely. The frst row through the thrd row are usng crteron 1 to crteron 3, respectvely. observe that n Fgure 12.3a through c uses crteron 1 through 3 n tme nstant 1 whle Fgure 12.3d through f uses crteron 1 through 3 at tme nstant 2. It can be notced from Fgure 12.3d that the problem for usng crteron 1 only s that when the objects are gettng K10681_C012.ndd 351 8/13/ :48:42 AM

16 352 Intellgent Vdeo Survellance: Systems and Technology close to the cameras, the sze of the boundng box wll ncrease to a non- desrable sze for observaton any more. Meanwhle, there are often some cases that when the person s enterng the scene, ts sze s not small but only part of the body s shown, whch should not be preferred as well f some other cameras can gve a full vew of the body. Thus, we ntroduce crteron 2, consderng the relatve poston of the objects n the FOVs of the cameras. The closer the centrod of the person s to the center of the FOV, the hgher the camera utlty s generated. We can observe that when applyng crteron 2 n Fgure 12.3e, the camera wth the object near the center s chosen and we can, thus, obtan a hgher resoluton of the person compared wth the results for usng crteron 1 n Fgure 12.3d. However, the problem for usng crteron 1 or crteron 2 only s that n many frames we reject the cameras that can see a person s face, whch s of general nterest. Ths case s shown n Fgure 12.3a, b, and d. To solve ths problem, we come up wth crteron 3. So, when applyng crteron 3, we can obtan a more desrable camera wth a frontal vew of the person n Fgure 12.3c and f. Whereas crteron 3 can successfully select a camera wth a frontal-vew person, t may fal to track a person when no face can be detected. For nstance, as shown n Fgure 12.3f, although the person s n the FOV of some camera, t s lost based on crteron 3. So, f nally, we come up wth a weghted combnaton of these three crtera. As stated prevously, we use 0.2, 0.1, and 0.7 as the weghts for these three crtera respectvely so that, n most cases, the system wll choose the camera that can see a person s face. For those frames where there s a person wth no face detected, the combnaton crteron can also provde a best camera based on crtera 1 and 2 and, thus, realze a contnuous trackng. All the camera handoffs, when applyng the combned crteron, are shown n Fgure 12.4a through. The error rate (as def ned n Secton ) n ths case s 5.56%. Ths combned crteron provdes camera assgnments and handoffs wth a mnmum error rate among the four crtera def ned n Secton Camera utltes, person utltes, and the correspondng assgnment probabltes for the usng the combned crteron s shown n Fgure 12.5, where Probablty[][j] stands for the probablty that Camera j s assgned to track Person. K10681_C012.ndd 352 8/13/ :48:43 AM

17 Dynamc Camera Assgnment and Handoff 353 (a) (b) (c) (d) (e) (f) (g) (h) () Fgure 12.4 (See color nsert followng page xxx.) All camera handoffs when applyng the combned crteron for 2 persons and 3 cameras case. In an n-camera n-person case, we can expect that the number of teraton of the proposed approach wll go up much slower than that of the exhaustve approach. So, the computatonal tme-savng advantage of the proposed approach wll be more obvous as the stuaton of the task becomes more complcated, as shown n our smulaton result n Fgure K10681_C012.ndd 353 8/13/ :48:43 AM

18 354 Intellgent Vdeo Survellance: Systems and Technology 1.5 Camera utltes Camera utlty 1 Camera utltes Camera utlty 2 Camera utlty Frames person utlty Person utlty 1 Person utlty 0.5 Person utlty Frames probabltes Probabltes 0.5 Probablty(1)(1) Probablty(1)(2) Probablty(1)(3) Probablty(2)(1) Probablty(2)(2) 0 Probablty(2)(3) Frames Fgure 12.5 (See color nsert followng page xxx.) Utltes and assgnment probabltes for each processed frame when usng the combned crteron. Probablty[][j] stands for the probablty that Camera j s selected to track Person Convergence of Results for Barganng In our experments, n most cases, the probabltes for makng the assgnment profle converges (wth ε < 0.05, where ε s the dfference between the two successve results) wthn 5 teraton. So, we use 5 K10681_C012.ndd 354 8/13/ :48:45 AM

19 Dynamc Camera Assgnment and Handoff 355 Number of teratons Proposed approach Exhaustve approach (a) Number of persons (b) Number of cameras Number of teratons Proposed approach Exhaustve approach Fgure 12.6 Smulaton result for more cameras and persons. (a) The result for a f xed number of cameras. (b) The result for a f xed number of persons. 5 Number of teraton Frames Fgure 12.7 Number of teraton for the barganng mechansm n each. as the teraton threshold when barganng. Thus, for those cases that wll not converge wthn 5 teratons, there may be an assgnment error based on the unconverged probabltes, as dscussed n Secton In Fgure 12.7, we plot the number of teraton wth respect to every processed frame. It turns out that the average teraton number for the case n our experment s As the numbers of persons and cameras ncrease, ths barganng system wll save a lot of computatonal cost to get the optmal camera assgnments. A typcal convergence for one of the assgnment probabltes n a barganng among cameras s gven n Fgure Comparson wth Another Related Approach In ths secton, we wll compare our approach wth another camera assgnment approach by Jo and Han n (2006). The authors perform K10681_C012.ndd 355 8/13/ :48:46 AM

20 356 Intellgent Vdeo Survellance: Systems and Technology 0.55 Prob (1)(2) n each barganng step Number of teraton Fgure 12.8 A typcal convergence n the barganng number of teraton Prob. (From Javed, O., Khan, Rasheed, Z., and Shah, M. Camera handoff: Trackng n multple uncalbrated statonary cameras. IEEE Workshop on Human Moton, Austn, TX, pp , 212, Wth permsson.) AQ2 camera handoffs by calculatng the COR (the co-occurrence to occurrence rato). We wll call ths the COR approach. In Jo and Han (2006), the mean probablty that a movng object s detected at a locaton x n the FOV of a camera s called an occurrence at x. The mean probablty that movng objects are smultaneously detected at x n the FOV of one camera and x n the FOV of another camera s called a co-occurrence of x and x. The COR approach decdes whether two ponts are n correspondence wth each other by calculatng the COR. If the COR s hgher than some predefned threshold, then the two ponts are decded to be n correspondence wth each other. When one pont s gettng close to the edge of the FOV of one camera, the system wll handoff to another camera that has ts correspondng pont. However, the COR approach n Jo and Han (2006) has been appled to two cameras only. We generalze ths approach to the cases wth more cameras by comparng the accumulated COR n the FOVs of multple cameras. We randomly select 100 ponts on the detected person, tran the system for 10 frames to construct the correspondence for these 100 ponts, calculate the cumulatve CORs n the FOVs of dfferent cameras, and select the one wth the hghest value for handoff. Experments have been done to compare the COR approach wth our approach for the 1 person 3 cameras case and the 2 persons 3 cameras case. K10681_C012.ndd 356 8/13/ :48:46 AM

21 Dynamc Camera Assgnment and Handoff 357 The 1 person 3 cameras case The handoff process by usng the COR approach and the correspondng frames by usng our approach (may not be the handoff frames) are shown n Fgure In Fgure 12.9f through h, the COR approach (a) (b) (c) (d) (e) (f) (g) (h) Fgure 12.9 (See color nsert followng page xxx.) Two camera handoffs by usng the co-occurrence to occurrence rato (COR) approach and the comparson wth our approach. The left column are the results by our approach and the rght column are the results by the COR approach. K10681_C012.ndd 357 8/13/ :48:46 AM

22 358 Intellgent Vdeo Survellance: Systems and Technology swtches to camera 1, whle our proposed approach stcks to camera 2 to get the frontal vew of the person. The COR approach needs a tme perod to construct the correspondence between vews of dfferent. As stated earler, we let ths perod to be 10 frames. As a result, there s some tme delay for the handoff. For nstance, n Fgure 12.9a through d, our approach has already swtched to camera 3 n Fgure 12.9a as long as the sze of the person s unacceptable, but the COR approach does ths n Fgure 12.9d. The 2 person 3 cameras case In Fgure 12.10, we show the comparson of results n ths case. We can notce that the COR approach produces more errors. Whenever there s occluson n one of the FOVs, there s a hgh probablty that the COR approach wll provde the wrong ponts n another FOV and ths causes loss of track for some persons. Examples are Fgure 12.10b, d, f. By the comparson, we can notce that the COR approach can only swtch the camera to another one when the person s about to leave the FOV, but cannot select the best camera based on other crtera. So, the number of handoffs by our approach s larger than that of the COR approach (see Table 12.3). If we use the def nton of error as stated n the Secton , the error rates for these two cases are compared n Table SUMMARY In ths paper, we proposed a new prncpled approach based on game theory for camera assgnment and handoff problem. The approach s ndependent of the spatal and geometrcal relatonshps among the cameras. It s robust wth respect to multple crtera for trackng are consdered. The key mert of the proposed approach s that we use the game theory framework wth barganng mechansm for camera assgnment n a vdeo network so that we can obtan a stable soluton wth a small number of teratons. Ths makes our approach computatonally more effcent and robust wth respect to other exstng approaches, K10681_C012.ndd 358 8/13/ :48:48 AM

23 Dynamc Camera Assgnment and Handoff 359 (a) (b) (c) (d) (e) (f) Fgure (See color nsert followng page xxx.) Some camera handoff errors by the co-occurrence to occurrence rato (COR) approach and handoffs n a sngle-person case. In (b), snce there s no dentfcaton mechansm, the COR approach msmatches the two persons and loses track one of the two persons. In (d) the system fals to calculate the co-occurrence to occurrence rato and, thus, loses the person n the lght color although t s avalable n camera 2. In (f), the COR approach msmatches the person and loses to track one of the persons. such as Jo and Han (2006). Future work wll allow communcatons among cameras, whch wll make the computatonal framework and computatonal resources decentralzed and dstrbuted. K10681_C012.ndd 359 8/13/ :48:48 AM

24 360 Intellgent Vdeo Survellance: Systems and Technology TABLE 12.3 Comparson of Error Rates for the Co-Occurrence to Occurrence Rato (COR) Approach and Our Proposed Approach Case 1 (1 Person 3 Cameras) Number of Handoffs Error Rate (%) Case 2 (2 Person 3 Cameras) Number of Handoffs Error Rate (%) The COR approach Our approach BIBLIOGRAPHICAL AND HISTORICAL REMARKS There have been many papers dscussng approaches for camera assgnments n a vdeo network. Javed et al. (2000) focus on fndng out the lmts of overlappng FOVs of multple cameras. Park et al. (2006) create dstrbuted look-up tables accordng to how well the cameras can mage a specfc locaton. Jo and Han (2006) construct a handoff functon by computng the rato of co-occurrence to occurrence for several pars of ponts n the FOVs of two correspondng cameras. Ths knd of approaches rely on obtanng the spatal topology of the camera network and calculatng the geometrcal relatonshps among cameras, whch tends to be qute complcated when the topology becomes complex and t s dffcult to learn the topology based on the random traffc patterns. Statstcs-based methods (Kettnaker and Zabh 1999, Chang and Gong 2001, Kang et al. 2003, Javed et al. 2005, Song and Roy-Chowdhury 2007) provde an optmal soluton wth respect to object trajectores, whle other factors, such as orentaton, shape, and face, whch are also very mportant for trackng, are not consdered. Also, many researchers have used calbrated cameras, an example s Ca and Aggarwal (1999). REFERENCES Arslan, G., Marden, J.R., and Shamma, J.S Autonomous vehcle-target assgnment: A game-theoretcal formulaton. ASME Journal of Dynamc Systems, Measurement, and Control, specal ssue on Analyss and Control of Mult-Agent Dynamc Systems, 129, K10681_C012.ndd 360 8/13/ :48:50 AM

25 Dynamc Camera Assgnment and Handoff 361 Bradsk, G.R Computer vson face trackng for use n a perceptual user nterface. Intel Technology Journal, Q2, Ca, Q. and Aggarwal, J.K Human moton n structured envronments usng a dstrbuted camera system. IEEE Transacton on Pattern Analyss and Machne Intellgence, 21(11), Chang, T. and Gong, S Bayesan modalty fuson for trackng multple people wth a mult-camera system. In European Workshop on Advanced Vdeo- Based Survellance Systems, Vancouver, Canada. Javed, O., Khan, Rasheed, Z. and Shah, M Camera handoff: Trackng n multple uncalbrated statonary cameras. In IEEE Workshop on Human Moton, Austn, TX, pp , 212. Javed, O., Shafque, K., and Shah, M Appearance modelng for trackng n multple non-overlappng cameras. In Internatonal Conference on Computer Vson and Pattern Recognton, San Dego, CA, Vol. 2, pp Jo, Y. and Han, J A new approach to camera handoff wthout camera calbraton for the general scene wth non-planar ground. In ACM Internatonal Workshop on Vdeo Survellance and Sensor Networks, Santa Barbara, CA, pp Kang, J., Cohen, I., and Medon, G Contnuous trackng wthn and across camera streams. In Internatonal Conference on Computer Vson and Pattern Recognton, Madson, WI, Vol. 1, pp Kettnaker, V. and Zabh, R Bayesan mult-camera survellance. In Internatonal Conference on Computer Vson and Pattern Recognton, Fort Collns, CO, p L, Y. and Bhanu, B Utlty-based dynamc camera assgnment and handoff n a vdeo network. In Internatonal Conference on Dstrbuted Smart Cameras, Stanford, CA. Myerson, R.B Game Theory-Analyss of Conflct. Harvard Unversty Press, Cambrdge, MA. Park, J., Bhat, P.C., and Kak, A.C A look-up table based approach for solvng the camera selecton problem n large camera networks. In IEEE Workshop on Dstrbuted Smart Cameras, Boulder, CO. Song, B. and Roy-Chowdhury, A Stochastc adaptve trackng n a camera network. In Internatonal Conference on Computer Vson, Ro de Janero, Brazl, pp AQ3 AQ4 K10681_C012.ndd 361 8/13/ :48:50 AM

26 AUTHOR QUERY [AQ1] Please provde the sgnfcance of the usage of bold n Table [AQ2] Please check whether the nserted source lne s correct. [AQ3] Please provde the ntals for the author Khan. [AQ4] Please check f the updated locaton s correct n Refs. Jo. Y and Park, J. et al K10681_C012.ndd 362 8/13/ :48:50 AM