2 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE (a) Ambgutes from person re dentfcaton n solaton (b) Assocatng groups of people may reduce ambgutes n mat

Transcription

1 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE 1 Assocatng Groups of People We-Sh Zheng jason@dcs.qmul.ac.uk Shaogang Gong sgg@dcs.qmul.ac.uk Tao Xang txang@dcs.qmul.ac.uk School of EECS, Queen Mary Unversty of London, London E1 4NS, UK Abstract In a crowded publc space, people often walk n groups, ether wth people they know or strangers. Assocatng a group of people over space and tme can assst understandng ndvdual s behavours as t provdes vtal vsual context for matchng ndvduals wthn the group. Seemngly an easer task compared wth person matchng gven more and rcher vsual content, ths problem s n fact very challengng because a group of people can be hghly non-rgd wth changng relatve poston of people wthn the group and severe self-occlusons. In ths paper, for the frst tme, the problem of matchng/assocatng groups of people over large space and tme captured n multple non-overlappng camera vews s addressed. Specfcally, a novel people group representaton and a group matchng algorthm are proposed. The former addresses changes n the relatve postons of people n a group and the latter deals wth varatons n llumnaton and vewpont across camera vews. In addton, we demonstrate a notable enhancement on ndvdual person matchng by utlsng the group descrpton as vsual context. Our methods are valdated usng the 08 -LIDS Multple-Camera Trackng Scenaro (MCTS) dataset on multple camera vews from a busy arport arrval hall. 1 Introducton Object recognton has always been mportant for computer vson. In recent years, the focus of object recognton has shfted from recognsng objects captured n solaton aganst clean background under well-controlled lghtng condtons to a more challengng but also more useful problem of recognsng objects subject to occluson aganst cluttered background wth drastc vew angle and llumnaton changes. In partcular, the problem of person re-dentfcaton or trackng (from dsjont vews) has receved ncreasng nterest [6, 8, 9, 10, 13, 18], whch ams to match a person observed n dfferent non-overlappng locatons over dfferent camera vews. In ths paper, we consder a new problem, albet closely related to the above, of assocatng groups of people over dfferent camera vews. In a crowded publc space, people often walk n groups, ether wth people they know or strangers. To be able to assocate the same group of people over dfferent camera vews at dfferent locatons can brng about two sgnfcant benefts: (1) Matchng a group of people over large space and tme can be extremely useful n understandng and nferrng longerterm assocaton and more holstc behavour of a group of people n publc space. (2) It c 09. The copyrght of ths document resdes wth ts authors. It may be dstrbuted unchanged freely n prnt or electronc forms. BMVC 09 do: /c.23.23

2 2 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE (a) Ambgutes from person re dentfcaton n solaton (b) Assocatng groups of people may reduce ambgutes n matchng (c) Dffcult examples of assocatng groups of people Fgure 1: Advantages from and challenges n assocatng groups of people vs. person redentfcaton n solaton. can provde vtal vsual context for assstng the match of ndvduals as the appearance of a person often undergoes drastc change across camera vews caused by lghtng and vew angle varatons. Most sgnfcantly, people appearng n publc space are prone to occlusons by others near by. These vewng condtons make person re-dentfcaton an extremely hard problem. On the other hand, groups of people are less affected by occluson whch can provde a rcher context and reduce ambguty n dscrmnatng an ndvdual aganst others. Ths s llustrated by examples shown n Fg. 1 (a) where each of the sx groups of people conssts of one or two people n dark clothng. Based on appearance alone, t s dffcult f not mpossble to dstngush them n solaton. However, when they are consdered n context by assocatng groups of people they appear together, t becomes much clearer that all canddates hghlghted by red boxes are dfferent people. Fg. 1 (b) shows examples of cases where matchng groups of people together seems to be easer than matchng ndvduals n solaton due to the changes n the appearance of people n dfferent vews caused by occluson or change of body posture. We consder that the group context s more robust aganst these changes and more consstent over dfferent vews. However, assocatng groups of people ntroduces new challenges: (1) Compared to an ndvdual, the appearance of a group of people s hghly non-rgd and the relatve postons of the members can change sgnfcantly and often. (2) Although occlusons by other objects s less an ssue, self-occluson caused by people wthn the group remans a problem whch can cause changes n group appearance. (3) Dfferent from a relatvely stable shape of every uprght person whch has smlar aspect rato, the aspect rato of the shapes of dfferent groups of people can be very dfferent. Some dffcult examples are shown n Fg. 1 (c). Due to these challenges, exstng representaton descrptors and matchng methods for person re-dentfcaton are not sutable for solvng the group assocaton problem. In ths paper, a novel people group representaton s proposed based on two new rato-occurrence descrptors ntroduced here. Gven ths group representaton, a group matchng algorthm s formulated to acheve robustness aganst both changes n relatve postons of people wthn a group and varatons n llumnaton and vew angle across dfferent cameras. In addton, a new person re-dentfcaton method s ntroduced by utlsng assocated group of people as vsual context to mprove the matchng of ndvduals across camera vews. To the best of our knowledge, there has been no prevous attempt at addressng the problem of matchng/assocatng groups of people over multple camera vews. There are related work reported n the lterature on crowd detecton and analyss [1, 2, 11, ] and group actvty recognton [7, 17]. However, these are not concerned wth group assocaton over space and tme ether wthn the same camera vews or across dfferent vews. The proposed

3 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE 3 model s valdated usng 08 -LIDS Multple-Camera Trackng Scenaro (MCTS) dataset captured by multple camera vews from a busy arport arrval hall [14]. 2 Modellng Group Assocaton 2.1 Group Representaton Gven a gallery set and a probe set of mages of dfferent groups of people, we am to fnd the best matched group template regstered n the gallery for any probe group mage. Smlar to [16, 18], we frst assgn a label to each pxel of a gven group mage I. The label can be a smple colour or a vsual word ndex of colour and gradent nformaton together. Due to the change of camera vew and varyng postons and motons of a group of people, we consder that ntegraton of local rotatonal nvarant features and color densty nformaton s better for constructng vsual words for ndexng. In partcular, we extract SIFT features [12] (a 128-dmensonal vector) for each RGB channel at each pxel wth a surroundng support regon (12 12 n our experment), and obtan an average RGB colour vector of that pxel over a support regon (3 3 n our experment), where the colour vector s normalzed to [0,1] 3. The SIFT vector and colour vector are then concatenated for each pxel for representaton, whch we call the SIFT+RGB feature. The SIFT+RGB features are quantzed nto n clusters by K-means and a code book A of n vsual words w 1,...,w n s bult. Fnally, an appearance label mage s bult by assgnng a vsual word ndex to the correspondng SIFT+RGB feature at each pxel of the group mage. In order to remove background nformaton, background subtracton s frst performed. Then, only features extracted for foreground pxels are used to construct vsual words for group mage representaton. To represent the dstrbuton of vsual words of any mage, a sngle hstogram of vsual words, whch we call the holstc hstogram, can be used [3]. However, ths representaton loses all spatal dstrbuton nformaton of the vsual words. One way to allevate ths problem s to dvde the mage nto grd blocks and concatenate the hstograms of blocks one by one, for nstance smlar to [4]. However, ths stll cannot cope wth a common case (examples n Fg. 1 (c)) n group mages when people swap ther postons. Moreover, correspondng mage grd postons between two group mages are not always guaranteed to represent foreground regons, therefore such a hard-wred grd block based representaton s not always vald. In addton, t s noted that whlst global spatal relatonshps between people wthn a group can be hghly unstable, local spatal relatonshps between small patches wthn a local regon may be stable, e.g. wthn the boundng box of a person. In vew of these characterstcs of group mages, we propose to represent a group usng two descrptors: a center rectangular rng rato-occurrence descrptor whch ams to descrbe the rato nformaton of vsual words wthn and between dfferent rectangular rng regons, and a block based rato-occurrence descrptor for explorng more specfc local spatal nformaton between vsual words that could be stable. These two descrptors are fnally combned for group representaton. Center Rectangular Rng Rato-Occurrence Descrptor (CRRRO): Rectangular rng regons are consdered to be approxmately rotatonal nvarant and effcent ntegral computaton of vsual words hstogram s also avalable [16]. To that end, we defne a holstc rectangular rng structure expandng from the center of a group mage. The l rectangular rngs dvde a group mage nto l non-overlapped regons P 1,,P l from nsde to outsde. Every rectangular rng s 0.5 N/l and 0.5 M/l thck along the vertcal and horzontal drectons respectvely (see Fg. 2 (a) wth l = 3), where the group mage s of sze M N. Such

4 4 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE (a) CRRRO Descrptor (b) BRO Descrptor N P 1 P 2 P 3 SB 5 SB 4 SB 1 SB 0 SB SB 3 2 β 2 β 1 M Fgure 2: Partton of a group mage by two descrptors. Left: the Center Rectangular Rng Rato-Occurrence Descrptor (β 1 = M/2l,β 2 = N/2l,l = 3); Rght: the Block based Rato- Occurrence Descrptor (γ = 1), where whte lnes are to show the grds of the mage. Fgure 3: An llustraton of a group of people aganst dark background. a parttonng of a group mage s especally useful for descrbng a par of people because the dstrbuton of consttuent patches of each person n each rng s lkely to be more stable aganst changes n relatve postons between the two people over dfferent vewponts or scalng (see llustratons n Fg. 3). After a partton of any mage for representaton, a common way for constructng a codebook s to concatenate the hstogram of vsual words from each rng. However, ths gnores any spatal relatonshps exsted between vsual words from dfferent rng-zones of a partton. We consder retanng such spatal relatonshps to be mportant and ntroduce a noton of ntra- and nter- rato-occurrence maps as follows. For each rng-regon P, a hstogram h s bult, where h (a) ndcates the frequency (occurrence) of vsual word w a. Then for P, an ntra rato-occurrence map H s defned as h (a) H (a,b) = h (a) + h (b) + ε, (1) where ε s a very small postve value n order to avod 0/0. H (a,b) then represents the rato-occurrence between words w a and w b wthn the regon. In order to capture any spatal relatonshp between vsual words wthn and outsde regon P, we further defne another two rato occurrence maps for rng-regon P. Defne: g = 1 j=1 h j, s = l h j, j=+1 where g represents the dstrbuton of vsual words enclosed by the rectangular rng P and s represents the dstrbuton of vsual words outsde P, where we defne g 1 = 0 and s l = 0. Then two nter rato-occurrence maps S and G are formulated as follows: G (a,b) = g (a) g (a) + h (b) + ε, S s (a) (a,b) = s (a) + h (b) + ε. (2) Therefore, for each rng-regon P, we construct a trplet representaton T r = {H,S,G }, and a group mage s represented by a set {T r} l =1. We shall demonstrate n our experment that ths group mage representaton usng a set of trplet ntra- and nter-rato occurrence maps gves better performance for assocatng groups of people than that of usng a conventonal concatenaton based representaton. Block based Rato-Occurrence Descrptor (BRO): The descrptor desgned above stll cannot cope well wth large non-center-rotatonal changes n people s postons wthn a group. It also does not utlse any local structure nformaton that may be more stable or

5 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE 5 consstent across dfferent vews of the same group, e.g. certan parts of a person can be vsually more consstent than others. As we do not make any assumptons on people n a group beng well segmented due to self-occluson, we revst a group mage to explore patch (partal) nformaton approxmately by dvdng t nto ω 1 ω 2 grd blocks B 1,B 2,,B ω1 ω 2, and only the foreground blocks (defned as the block wth more than percent pxels are foreground) are consdered. Due to the approxmate partton of a group mage and the low resoluton of each patch or potental llumnaton change and occluson, we extract rather smple (therefore probably more robust) spatal relatonshps between vsual words n each foreground block by further dvdng the block nto small block regons usng L-shaped partton [18] wth a modfcaton that the most nner four block regons are merged (see Fg. 2 (b)). Ths s because those block regons are always small and may not contan suffcent nformaton. As a result, we obtan 4γ + 1 block regons wthn each block B denoted by SB 0,,SB 4γ for some postve nteger γ. For assocatng groups of people over dfferent vews, we frst note that not all blocks B appear at the same postons n the group mages. For example, a par of people may swap ther postons resultng n the blocks correspondng to those foreground pxels change ther postons n dfferent mages. Also, there may be other vsually smlar blocks n the same group mage. Hence, descrbng local matches only based on features wthn block B could not be dstnct enough. To reduce ths ambguty, for representng each block B, we further nclude a complementary mage regon SB 4γ+1, whch s the mage porton outsde block B (see Fg. 2 (b) wth γ = 1). Therefore, for each block B, we partton the group mage nto SB 0,SB 1,,SB 4γ and SB 4γ+1. We demonstrate n our experment that ncludng such complementary regon SB 4γ+1 would sgnfcantly enhance matchng performance. Lke the Center Rectangular Rng Rato-Occurrence Descrptor, for each block B, we learn an ntra rato-occurrence map H j between vsual words n each block regon SB j. Smlarly, we explore an nter rato-occurrence map O j between dfferent block regons SB j. Snce the sze of each block regon n block B would always be relatvely much smaller than the complementary regon SB 4γ+1, the rato nformaton between them wll be senstve to nose. Consequently we consder two smplfed nter rato-occurrence maps O j between block B and ts complementary regon SB 4γ+1 formulated as follows: t (a) O 1 (a,b) = t (a) + z (b) + ε, O 2 (a,b) = z (a) z (a) + t (b) + ε, (3) where z and t are the hstograms of vsual words of block B and mage regon SB 4γ+1, {O j } 2 j=1, and a group respectvely. Then, each block B s represented by T b = {H j }4γ+1 j=0 mage s represented by a set {T b }m =1 where m s the amount of foreground blocks B. These two proposed descrptors, CRRRO and BRO are specally desgned for assocatng mages of groups of people. Due to hghly unstable postons of people wthn a group and lkely partal occlusons among them, they explore the nter-person spatal relatonal nformaton n a group and the lkely local patch (partal) nformaton for each person respectvely. 2.2 Group Image Matchng We match two group mages I 1 and I 2 by combnng the dstance metrcs of the two proposed descrptors as follows: ( ) ( ) d(i 1,I 2 ) = d r {T r(i 1 )} l =1,{T r (I 2 )} l =1 + α d b {T b (I 1)} m 1 =1,{T b (I 2)} m 2 =1, α 0, (4)

6 6 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE where {T r(i 1 )} l =1 ndcates the center rectangular rng rato-occurrence descrptor for group mage I 1 whlst {T b (I 1)} m 1 =1 s for the block based descrptor. For d r, the L 1 norm metrc s used to measure the dstance between each correspondng rato-occurrence map and d r s obtaned by averagng these dstances. For d b, snce the spatal relatonshp between patches s not stable n dfferent mages of the same group and also not all the patches n one group mage can be matched wth those n another, t s napproprate to drectly measure the dstance between the correspondng patches (blocks) of two group mages. To address ths problem, we assume that for each par of group mages, there exsts at most k pars of matched local patches between two mages. We then defne d b as a top k-match metrc where k s a postve nteger as follows: ) d b ({T b (I 1)} m 1 =1,{T b (I 2)} m { } 2 =1 = mn k 1 AC BD 1, C,D A R q m 1,B R q m 2,C R m 1 k,d R m 2 k, (5) where the th ( th ) column of matrx A (B) s the vector representaton of T b (I 1) (T b (I 2)), each column c j (d j ) of C (D) s an ndcator vector n whch only one entry s 1 and the others are zeros, and the columns of C (D) are orthogonal. Note that m 1 and m 2, the amount of foreground blocks n two group mages, may be unequal. Generally, drectly solvng Eq. (5) s hard. Notng that mn C,D { AC BD 1 } k j=1 mn c j,d j { Ac j Bd j 1 } where {c j } and {d j } are sets of orthogonal ndcator vectors, we therefore approxmate the k-match metrc value as follows: the most matched patches a 1 and b 1 are frst found by fndng the smallest L 1 dstance between columns of A and B. We then remove a 1 and b 1 from A and B respectvely and fnd the next most matched par. Ths procedure repeats untl the top k matched patches are found. 3 Group as Contextual Cue for Person Re-dentfcaton We wsh to explore group nformaton for reducng the ambguty n person re-dentfcaton f a person would appear n the same group. Suppose a set of L pared samples {(I p,i g)} L =1 are gven, where I g s the correspondng group mage of the th person mage I p. We ntroduce a group-contextual-descrptor smlar n sprt to the center rectangular rng descrptor ntroduced above, wth a mnor modfcaton that we expand the rectangular rng structure surroundng each person. Ths makes the group context person specfc,.e. two people n the same group would have dfferent context. Note that, only context features at foreground pxels are extracted. As a result, the most nner rectangular regon P 1 s the boundng box of a person, and for other outer rngs, they are max{m x M 1,x M 1 }/(l 1) and max{n y N 1,y N 1 }/(l 1) thck along the horzontal and vertcal drectons, where (x 1,y 1 ) s the center of regon P 1, M and N are wdth and heght of the group mage, and M 1 and N 1 are wdth and heght of P 1. In partcular, when l = 2, the rectangular rng structure would dvde a group mage nto two parts: a person-centred boundng box and a surroundng complementary mage regon. There can be many ways to ntegrate group nformaton for person re-dentfcaton. In ths paper, we smply combne the dstance metrc d p of some person descrptor such as the colour hstogram and the dstance metrc d r of the correspondng group context descrptor between two people. More specfcally, denote the person descrptors of person mage I 1 p and I 2 p as P 1 and P 2 respectvely and denote ther correspondng group context descrptors as T 1 and T 2 respectvely. Then the dstance between two people s computed as: d(i 1 p,i 2 p) = d p (P 1,P 2 ) + β d r (T 1,T 2 ), β 0. (6)

7 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE 7 4 Experments We conducted extensve experments usng the 08 -LIDS Multple-Camera Trackng Scenaro (MCTS) dataset to evaluate the feasblty and performance of the proposed methods for assocatng groups of people n a crowded publc space. Dataset & Parameter Settngs: The -LIDS MCTS dataset was captured at an arport arrval hall n the busy tmes under a mult-camera CCTV network. We extracted mage frames captured from two non-overlappng camera vews. In total, 64 groups were extracted and 274 group mages were cropped. Most of the groups have 4 mages, ether from dfferent camera vews or from the same camera but captured at dfferent locatons at dfferent tmes. These group mages are of dfferent szes. Automatc detecton of group mage should be requred n practce. In ths paper, we take the frst step on assocaton of groups of people and focus on the evaluaton of group descrptors. Sldng wndow based technque could be used n practce for group mage detecton based on the proposed group descrptors. From the group mages, we extracted 476 person mages for 119 pedestrans, most of whch are wth 4 mages. All person mages were normalzed to pxels. Dfferent from other person datasets [6, 8, 18], these person mages were captured by non-overlappng cameras, and many of them underwent large llumnaton change and were subject to occluson. For code book learnng, addtonal 80 mages (of sze 6 480) were randomly selected wth no overlap wth the dataset descrbed above. As descrbed n Secton 2, the SIFT+RGB features were extracted at each pxel of an mage. In our experments, a code book wth vsual words (clusters) was bult usng K-means. Unless otherwse stated, our descrptors are set as follows. For the center rectangular rng rato-occurrence (CRRRO) descrptor, we set l = 3. For the block based rato-occurrence (BRO) descrptor, each mage was dvded nto 5 5 blocks, γ was set to 1, and the top 10-match score was computed. The default combnaton weght α n Eq. (4) was set to CMC Curve Synthetc dsambguaton rate (%) SDR Curve Holstc Color Hstogram Holstc Vsual Word Hstogram Concatenated Hstogram (RGB) Concatenated Hstogram (SIFT) CRRRO BRO Number of targets Fgure 4: Compare the CMC and SDR curves for assocatng groups of people usng the proposed CRRRO-BRO descrptor wth those from other commonly used descrptors. Evaluaton of Group Assocaton: We randomly selected one mage from each group to buld the gallery set and the other group mages formed the probe set. For each group mage n the probe set, we measured ts smlarty wth each template mage n the gallery. The s-nearest correct match for each group mage was obtaned. Ths procedure was repeated 10 tmes and the average cumulatve match characterstc (CMC) curve [18] and the synthetc dsambguaton rate (SDR) curve [8] were used to measure the performance, where the top matchng rates are shown for CMC curve and the SDR curve s able to gve an overvew of the whole CMC curve from the reacquston pont of vew [8].

8 8 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE Probe Image Rank 1 Rank 2 Rank 3 Rank 4 Rank 5 Probe Image Rank 1 Rank 2 Rank 3 Rank 4 Rank 5 Fgure 5: Examples of assocatng groups of people usng our model. Correct matches are hghlghted by red boxes. The performance of the combned Center Rectangular Rng Rato-Occurrence and Block based Rato-Occurrence (CRRRO-BRO) descrptor approach (Eq. (4)) s shown n Fg. 4. We compare our model wth two commonly used descrptors, colour hstogram and vsual word hstogram of SIFT features (extracted at each colour channel) [3], whch represent the dstrbutons of colour or vsual words of each group mage holstcally. We also apply these two descrptors to the desgned center rectangular rng structure by concatenatng the colour or vsual word hstogram of each rectangular rng. For the colour hstogram, we selected the number of colour bns from {8,16,32,64,128} and found 16 was the best one. In order to make the compared descrptors scale nvarant, the hstograms used n the compared methods were normalzed [5]. For measurement, the Ch-square dstance χ 2 [5] was used. Results n Fg. 4 show the proposed CRRRO-BRO descrptor gves the best performance. It always keeps a notable margn to the CMC curve of the second best method, wth 44.62% aganst 36.14% and 77.29% aganst 69.57% for rank 5 and matchng respectvely. Compared to the exstng holstc representatons and the concatenaton of local hstograms representatons, the proposed descrptor benefts from explorng the rato nformaton between vsual words wthn and outsde each local regon. Moreover, Fg. 6 (b) shows that ether usng the proposed center based or block based descrptor can stll acheve an overall mprovement as compared to the concatenated hstogram of vsual words usng SIFT+RGB features (descrbed n Secton 2) denoted by "Concatenated Hstogram (Center, SIFT+RGB)" and "Concatenated Hstogram (Block, SIFT+RGB, k = 10)" n the fgure, respectvely. Ths suggests the rato maps can provde more nformaton for matchng. Fnally, Fg. 5 shows some examples of assocatng groups of people usng the proposed model (Eq. (4) wth α = 0.8). It demonstrates that our model s capable of establshng correct matchng when there are large varatons n people s appearances and ther relatve postons n a group caused by some very challengng vewng condtons ncludng sgnfcantly dfferent vew angles and severe occlusons. Evaluaton of the Proposed Descrptors: To gve more nsght on how the proposed descrptors perform n dfferent aspects, we show n Fg. 6 (a) comparatve results between the combnaton CRRRO-BRO (Eq. (4)) and the ndvdual CRRRO and BRO descrptors usng the metrcs d r and d b as descrbed n Secton 2.2. It shows that the combnaton of the center rng based and local block based descrptors utlses complementary nformaton and mproves the performance of each ndvdual descrptor. Fg. 6 (b) evaluates the effects of usng rato map nformaton as dscussed n the last paragraph. Fg. 6 (c) shows that by explorng the nter rato-occurrence between regons on the top of the ntra one, an overall

9 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE 9 (a) Combnaton of proposed descrptors 75 CRRRO BRO CRRRO BRO, α = 0.8 (b) Rato vs. Non rato Occurrence CRRRO Concatenated Hstogram (Center,SIFT+RGB) BRO(k=10) Concatenated Hstogram (Block,SIFT+RGB, k=10) 10 (c) Wth vs. Wthout Inter Rato Occurrence CRRRO, wth nter nformaton CRRRO, wthout nter nformaton BRO(k=10), wth nter nformaton BRO(k=10), wthout nter nformaton 10 Fgure 6: Evaluaton of the proposed descrptors. (a) Colour hstogram Wthout Group Context Wth Group context 10 (b) Shape and appearance context Wthout Group Context Wth Group context (d) Wth vs. Wthout Complementary Regon for BRO 10 (c) Appearance context model k=1, wthout complementary regon k=10, wthout complementary regon k=1, wth complementary regon k=10, wth complementary regon 5 Wthout Group Context Wth Group context Fgure 7: Improvng person re-dentfcaton usng group context. better performance s obtaned as compared wth a model wthout utlsng such nformaton. For the block based rato-occurrence descrptor, Fg. 6 (d) ndcates that ncludng the complementary regon wth respect to each block B can reduce the ambguty durng matchng. Person Re-Identfcaton wthout/wth Group Context: To show the effect of group context for mprovng person re-dentfcaton, we mplemented three methods for person redentfcaton. One s to use colour hstogram as representaton of a person mage and the other two are the appearance context model [18] and the shape and appearance context model [18]. For colour hstogram, the number of colour bns was 16. For the appearance context model and the shape and appearance context model, we learned the correspondng code books from addtonal person mages extracted from the mages used for code book learnng for the proposed group descrptors. We employed the same code book szes as suggested by [18], used 8 quantzng orentatons for the HOG n [18] and L-shaped regons for the plane partton n [18]. For group context, as descrbed n Secton 3, a tworectangular-rng structure s expanded from the center of the boundng box of each person. For evaluaton, one mage for each person was randomly selected as the gallery template and the others were as probe mages. Ths procedure was repeated 10 tmes and the average performances of these technques wthout and wth group context are shown n Fg. 7, where χ 2 dstance s used for the colour hstogram model and L 1 norm dstance s for the other two person descrptors. It s evdent that ncludng group context notably mproves the matchng rate regardless of the choce of dfferent person re-dentfcaton technques. Over 10% mprovement was always acheved for the colour hstogram model and the shape and appearance context model, whlst about 8-9% mprovement was obtaned over the appearance context model. Note that the performance we obtaned for the shape and appearance context model are not as hgh as that reported n [18]. Ths s because the person mages from the -LIDS MCTS dataset are much more challengng snce they were captured from non-overlappng multple camera vews subject to sgnfcant occluson, large varatons n both vew angle and llumnaton. 5 Concluson In ths paper, for the frst tme, we have formulated the problem of assocatng groups of people over multple non-overlappng camera vews and proposed a center rectangular rng and

10 10 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE block based rato-occurrence descrptors for effectve representaton of mages of groups of people n crowded publc spaces, and a top k-match model for matchng possble local patches of two group mages. We further demonstrated the advantages ganed from utlsng group context nformaton n mprovng person re-dentfcaton under very challengng vewng condtons usng the 08 -LIDS Multple Camera Trackng Scenaro dataset. Our ongong work s to mprove the descrptors and matchng algorthms n order to cope wth severe varatons n the relatve postons of people n groups. Acknowledgement Ths research was partally funded by the EU FP7 project SAMURAI, grant no References [1] O. Arandjelovć. Crowd detecton from stll mages. In BMVC, 08. [2] G. J. Brostow and R. Cpolla. Unsupervsed bayesan detecton of ndependent moton n crowds. In CVPR, 06. [3] G. Csurka, C. Dance, L. Fan, J. Wllamowsk, and C. Bray. Vsual categorzaton wth bags of keyponts. In ECCV Internatonal Workshop on Statstcal Learnng n Computer Vson, 04. [4] N. Dalal and B. Trggs. Hstograms of orented gradents for human detecton. In CVPR, 05. [5] C. Fowlkes, S. Belonge, F. Chung, and J. Malk. Spectral groupng usng the nystrom method. PAMI, 26(2):214 2, 04. [6] N. Ghessar, T. B. Sebastan, P. H. Tu, J. Rttscher, and R. Hartley. Person redentfcaton usng spatotemporal appearance. In CVPR, 06. [7] S. Gong and T. Xang. Recognton of group actvtes usng dynamc probablstc networks. In ICCV, 03. [8] D. Gray and H. Tao. Vewpont nvarant pedestran recognton wth an ensemble of localzed features. In ECCV, 08. [9] W. Hu, M. Hu, X. Zhou, J. Lou, T. Tan, and S. Maybank. Prncpal axs-based correspondence between multple cameras for people trackng. PAMI, 28(4): , 06. [10] O. Javed, Z. Rasheed, K. Shafque, and M. Shah. Trackng across multple cameras wth dsjont vews. In ICCV, 03. [11] D. Kong, D. Gray, and H. Tao. Countng pedestrans n crowds usng vewpont nvarant tranng. In BMVC, 05. [12] D. Lowe. Dstnctve mage features from scale-nvarant keyponts. IJCV, 2():91 110, 04.

11 ZHENG et al.: ASSOCIATING GROUPS OF PEOPLE 11 [13] C. Madden, E. Cheng, and M. Pccard. Trackng people across dsjont camera vews by an llumnaton-tolerant appearance representaton. Mach. Vson Appl., 18(3): , 07. [14] UK Home Offce. -LIDS Multple Camera Trackng Scenaro Defnton. 08. [] V. Rabaud and S. Belonge. Countng crowded movng objects. In CVPR, 06. [16] S. Savarese, J. Wnn, and A. Crmns. Dscrmnatve object class models of appearance and shape by correlatons. In CVPR, 06. [17] S. Saxena, F. Brémond, M. Thonnat, and R. Ma. Crowd behavor recognton for vdeo survellance. In 10th Internatonal Conference on Advanced Concepts for Intellgent Vson Systems, 08. [18] X. Wang, G. Doretto, T. Sebastan, J. Rttscher, and P. Tu. Shape and appearance context modelng. In ICCV, 07.