arxiv: v1 [cs.cv] 23 Apr 2015

Transcription

1 Person Re-dentfcaton wth Correspondence Structure Learnng Yang Shen 1, Weyao Ln 1, Junch Yan 1, Mnglang Xu 2, Janxn Wu 3 and Jngdong Wang 4 1 Shangha Jao Tong Unversty, Chna 2 Zhengzhou Unversty, Chna 3 Nanng Unversty, Chna 4 Mcrosoft Research Asa arxv: v1 [cs.cv] 23 Apr 2015 Abstract Ths paper addresses the problem of handlng spatal msalgnments due to camera-vew changes or human-pose varatons n person re-dentfcaton. We frst ntroduce a boostng-based approach to learn a correspondence structure whch ndcates the patch-wse matchng probabltes between mages from a target camera par. The learned correspondence structure can not only capture the spatal correspondence pattern between cameras but also handle the vewpont or human-pose varaton n ndvdual mages. We further ntroduce a global-based matchng process. It ntegrates a global matchng constrant over the learned correspondence structure to exclude cross-vew msalgnments durng the mage patch matchng process, hence achevng a more relable matchng score between mages. Expermental results on varous datasets demonstrate the effectveness of our approach. 1. Introducton Person re-dentfcaton (Re-ID) s of ncreasng mportance n vsual survellance. The goal of person Re-ID s to dentfy a specfc person ndcated by a probe mage from a set of gallery mages captured from cross-vew cameras (.e., cameras that are non-overlappng and dfferent from the probe mage s camera). 1 It remans challengng due to the large appearance changes n dfferent camera vews and the nterferences from background or obect occluson. One maor challenge for person Re-ID s the uncontrolled spatal msalgnment between mages due to cameravew changes or human-pose varatons. For example, n Fg. 1a, the green patch located n the lower part n camera A s mage corresponds to patches from the upper part n camera B s mage. However, most exstng works [25, 11, 12, 7, 8, 9, 22, 19] focus on handlng the overall appearance varatons between mages, whle the spatal msalgnment among mages local patches s not addressed. Although some patch-based methods [17, 15, 27] 1 In ths paper, an mage refers to the pxel regon of one person whch s cropped from a larger mage of a camera vew (cf. Fg. 1) [6]. (a) (b) (c) Fgure 1. (a) and (b): Two examples of usng a correspondence structure to handle spatal msalgnments between mages from a camera par. Images are obtaned from the same camera par: A and B. The colored squares represent sample patches n each mage whle the lnes between mages ndcate the matchng probablty between patches (lne wdth s proportonal to the probablty values). (c): The correspondence structure matrx ncludng all patch matchng probabltes between A and B (the matrx s down-sampled for a clearer llustraton). (Best vewed n color) address the spatal msalgnment problem by decomposng mages nto patches and performng an onlne patch-level matchng, ther performances are often restraned by the onlne matchng process whch s easly affected by the msmatched patches due to smlar appearance or occluson. In ths paper, we argue that due to the stable settng of most cameras (e.g., fxed camera angle or locaton), each camera has a stable constrant on the spatal confguraton of ts captured mages. For example, mages n Fgures 1a and 1b are obtaned from the same camera par: A and B. Due to the constrant from camera angle dfference, body parts n camera A s mages are located at lower places than those n camera B, mplyng a lower-to-upper correspondence pattern between them. Meanwhle, constrants from camera locatons can also be observed. Camera A (whch montors an ext regon) ncludes more sde-vew mages, whle camera B (montorng a road) shows more front or back-vew mages. Ths further results n a hgh probablty of sde-to-front/back correspondence pattern. Based on ths ntuton, we propose to learn a correspondence structure (.e., a matrx ncludng all patch-wse matchng probabltes between a camera par, as Fg. 1c) to encode the spatal correspondence pattern constraned by a 1

2 camera par, and utlze t to gude the patch matchng and matchng score calculaton processes between mages. Wth ths correspondence structure, spatal msalgnments can be sutably handled and patch matchng results are less nterfered by the confuson from appearance or occluson. In order for the correspondence structure to model human-pose varatons or local vewpont changes nsde a camera vew, the correspondence structure for each patch s descrbed by a one-to-many graph whose weghts ndcate the matchng probabltes between patches, as n Fg. 1. Besdes, a global constrant s also ntegrated durng the patch matchng process, so as to acheve a more relable matchng score between mages. Note that our approach s not lmted to person re-dentfcaton wth fxed camera settngs. Instead, t can also be appled to capture the camera-and-person confguraton and cross-vew correspondence for unfxed cameras, as demonstrated n the expermental results. In summary, our contrbutons to person Re-ID are three folds. Frst, we ntroduce a correspondence structure to encode cross-vew correspondence pattern between cameras, and develop a global-based matchng process by combnng a global constrant wth the correspondence structure to exclude spatal msalgnments between mages. These two components n fact establsh a novel framework for addressng the person Re-ID problem. Second, under ths framework, we propose a boostng-based approach to learn a sutable correspondence structure between a camera par. The learned correspondence structure can not only capture the spatal correspondence pattern between cameras but also handle the vewpont or human-pose varaton n ndvdual mages. Thrd, ths paper releases a new and challengng benchmark ROAD DATASET for person Re-ID. The rest of ths paper s organzed as follows. Sec. 2 revews related works. Sec. 3 descrbes the framework of the proposed approach. Sectons 4 to 5 descrbe the detals of our proposed global-based matchng process and boostngbased learnng approach, respectvely. Sec. 6 shows the expermental results and Sec. 7 concludes the paper. 2. Related Works Many person re-dentfcaton methods have been proposed. Most of them focus on developng sutable feature representatons about humans appearance [25, 11, 12, 7, 14], or fndng proper metrcs to measure the crossvew appearance smlarty between mages [8, 9, 22, 19]. Snce these works do not effectvely model the spatal msalgnment among local patches nsde mages, ther performances are often lmted due to the nterferences from vewpont changes and human-pose varatons. In order to address the spatal msalgnment problem, some patch-based methods are proposed [23, 17, 3, 15, 27, 26, 5, 20] whch decompose mages nto patches and perform an onlne patch-level matchng to exclude patch-wse Fgure 2. Framework of the proposed approach. msalgnments. In [23, 3], a human body n an mage s frst parsed nto semantc parts (e.g., head and torso). And then, smlarty matchng s performed between the correspondng semantc parts. Snce these methods are hghly dependent on the accuracy of body parser, they have lmtatons n scenaros where the body parser does not work relably. In [17], Orefe et al. dvde mages nto patches accordng to appearance consstences and utlze the Earth Movers Dstance (EMD) to measure the overall smlarty among the extracted patches. However, snce the spatal correlaton among patches are gnored durng smlarty calculaton, the method s easly affected by the msmatched patches wth smlar appearance. Although Ma et al. [15] ntroduce a body pror constrant to avod msmatchng between dstant patches, the problem s stll not well addressed, especally for the msmatchng between closely located patches. To reduce the effect of patch-wse msmatchng, some salency-based approaches [27, 26] are recently proposed, whch estmate the salency dstrbuton relatonshp between mages and utlze t to control the patch-wse matchng process. Although these methods consder the correspondence constrant between patches, our approach dffers from them n: (1) our approach focuses on constructng a correspondence structure where patch-wse matchng parameters are ontly decded by both matched patches. Comparatvely, the matchng weghts n the salency-based approach [26] s only controlled by patches n the probe-mage (probe patch). (2) Our approach models patch-wse correspondence by a one-to-many graph such that each probe patch wll trgger multple matches durng the patch matchng process. In contrast, the salency-based approaches only select one best-matched patch for each probe patch. (3) Our approach ntroduces a global constrant to control the patch-wse matchng result whle the patch matchng result n salency-based approaches s locally decded by choosng the best-matched one wthn a neghborhood set. 3. Overvew The framework of our approach s shown n Fg. 2. Durng the tranng process, whch s detaled n Secton 5, we

3 present a boostng-based process to learn the correspondence structure between the target camera par. Durng the predcton stage, whch s detaled n Secton 4 gven a probe mage and a set of gallery mages, we use the correspondence structure to evaluate the patch correlatons between the probe mage and each gallery mage, and fnd the optmal one-to-one mappng between patches, and accordngly the matchng score. The Re-ID result s acheved by rankng gallery mages accordng to ther matchng scores. 4. Person Re-Identfcaton wth Correspondence Structure In ths secton, we ntroduce the concept of correspondence structure, show the scheme of computng the patch correlaton usng the correspondence structure, and fnally present the patch-wse mappng method to compute the matchng score between the probe mage and the gallery mage Correspondence structure The correspondence structure, Θ A,B, encodes the spatal correspondence dstrbuton between a par of cameras, A and B. In our problem, we adopt a dscrete dstrbuton, whch s a set of patch-wse matchng probabltes, Θ A,B = {P (x A, B)}N A =1, where N A s the number of patches of an mage n camera A. P (x A, B) = {P (x A, xb 1 ), P (x A, xb 2 ),..., P (x A, xb N B )} descrbes the correspondence dstrbuton n an mage from camera B for the th patch x A of an mage captured from camera A, where N B s the number of patches of an mage n B. An llustraton of the correspondence dstrbuton s shown on the top-rght of Fg. 1c. The defnton of the matchng probabltes n the correspondence structure only depends on a camera par and are ndependent to the specfc mages. In the correspondence structure, t s possble that one patch n camera A s hghly correlated to multple patches n camera B, so as to handle human-pose varatons and local vewpont changes n a camera vew Patch correlaton Gven a probe mage U n camera A and a gallery mage V n camera B, the patch-wse correlaton between U and V, C(x U, xv ), s computed from both the correspondence structure between cameras A and B and the vsual features and wrtten as: C(x U, x V ) = λ Tc (P (x U, x V )) log Φ(f x U ; x U, x V ). (1) Here x U and x V are th and th patch n mages U and V ; f x U and f x V are the feature vectors for x U and x V. P (xu, xv ) = P (xa, xb ) s the correspondence structure of cameras A and B. λ Tc (P (x U, xv ) = 1 P (x U, xv ) > T c, and 0 otherwse, and T c = 0.05 s a threshold. Φ(f x U ; x U, xv ) s the correspondencestructure-controlled smlarty between x U and x V, Φ(f x U ; x U, x V ) = Φ z (f x U )P (x U, x V ), (2) where Φ z (f x U ) s the appearance smlarty between x U and x V. The correspondence structure P (x U, xv ) n Equatons 1 and 2, s used to adust the appearance smlarty Φ z (f x U ) such that a more relable patch-wse correlaton strength can be acheved. The thresholdng term λ Tc (P (x U, xv )) s ntroduced to exclude the patch-wse correlaton wth a low correspondence probablty, whch effectvely reduces the nterferences from msmatched patches wth smlar appearance. The patch-wse appearance smlarty Φ z (f x U ) n Eq. 2 can be acheved by many off-the-shelf methods [27, 26, 2]. In ths paper, we extract Dense SIFT and Dense Color Hstogram [27] from each patch and utlze the KISSME dstance metrc [9] to compute Φ z (Φ z (f x U )) (note that we tran dfferent KISSME metrcs for patch-pars at dfferent locatons) Patch-wse mappng Wth C(x U, xv ), the algnment-enhanced correlaton strength, we can fnd a best-matched patch n mage V for each patch n U and heren calculate the fnal mage matchng score. However, locally fndng the largest C(x U, xv ) may stll create msmatches among patch pars wth hgh matchng probabltes. For example, Fg. 3a shows an mage par U and V contanng dfferent people. When locally searchng for the largest C(x U, xv ), the yellow patch n U wll be msmatched to the bold-green patch n V snce they have both large appearance smlarty and hgh matchng probablty. Ths msmatch unsutably ncreases the matchng score between U and V. To address ths problem, we ntroduce a global one-toone mappng constrant and solve the resultng lnear assgnment task [10] to fnd the best matchng: Ω U,V = arg max Ω U,V {x U,xV } Ω U,V C(x U, x V ) (3) s.t. x U x U s, x V x V t {x U, x V }, {x U s, x V t } Ω U,V where Ω U,V s the set of the best patch matchng result between mages U and V. {x U, xv } and {xu s, x V t } are two matched patch pars n Ω. Accordng to Eq. 3, we want to fnd the best patch matchng result Ω U,V that maxmzes the

4 (a) (b) Fgure 3. Patch matchng result (a) by locally fndng the largest correlaton strength C(x U, x V ) for each patch and (b) by usng a global constrant. The red dashed lnes ndcate the fnal patch matchng results and the colored sold lnes are the matchng probabltes n the correspondence structure. (Best vewed n color) total mage matchng score ψ U,V = {x U,xV } Ω U,V C(x U, x V ), (4) gven that each patch n U can only be matched to one patch n V and vce versa. Eq. 3 can be solved by the Hungary method [10]. Fg. 3b shows an example of the patch matchng result by Eq. 3. From Fg. 3b, t s clear that by the ncluson of a global constrant, local msmatches can be effectvely reduced and a more relable mage matchng score can be acheved. Based on the above process, we can calculate the mage matchng scores ψ between a probe mage and all gallery mages n a cross-vew camera, and rank the gallery mages accordngly to acheve the fnal Re-ID result [15]. 5. Correspondence Structure Learnng 5.1. Obectve functon Gven a set of probe mages {U α } from camera A and ther correspondng cross-vew mages {V β } from camera B n the tranng set, we learn the optmal correspondence structure Θ A,B between cameras A and B so that the correct match mage s ranked before the ncorrect match mages n terms of the matchng scores. The formulaton s gve as below, mn R(V α ; ψ Uα,V Θ α (Θ A,B ), Ψ Uα,V β α (Θ A,B )), A,B U α where V α s the correct match gallery mage of the probe mage U α. ψ Uα,V α (Θ A,B ) (as computed from Eq. 4) s the matchng score between U α and V α and Ψ Uα,V β α (Θ A,B ) s the set of matchng scores of all ncorrect match mages. R(V α ; ψ Uα,V α (Θ A,B ), Ψ Uα,V β α (Θ A,B )) s the rank of V α among all the gallery mages accordng to the matchng scores. Intutvely, the penalty s the smallest f the rank (5) s 1,.e., the matchng score of V α s the greatest. The optmzaton s not easy as the matchng score calculaton (Eq. 4) s complcated. We present an approxmate soluton, a boostng-based process, to solve ths problem Boostng-based learnng The boostng-based approach utlzes a progressve way to fnd the best correspondence structure wth the help of bnary mappng structures. A bnary mappng structure s smlar to the correspondence structure except that t smply utlzes 0 or 1 nstead of matchng probabltes to ndcate the connectvty or lnkage between patches, cf. Fg. 4a. It can be vewed as a smplfed verson of the correspondence structure whch ncludes rough nformaton about the crossvew correspondence pattern. Snce bnary mappng structures only nclude smple connectvty nformaton among patches, ther optmal solutons are tractable for ndvdual probe mages. Therefore, by searchng for the optmal bnary mappng structures for dfferent probe mages and utlzng them to progressvely update the correspondence structure, sutable crossvew correspondence patterns can be acheved. The entre boostng-based learnng process can be descrbe by the followng steps as well as Algorthm 1. Fndng the optmal bnary mappng structure. For each tranng probe mage U α, we frst create multple canddate bnary mappng structures under dfferent search ranges by adacency-constraned search [27], and then fnd the optmal bnary mappng structure M α such that the rank order of U α s correct match mage V α s mnmzed under M α. Note that we fnd one optmal bnary mappng structure for each probe mage such that the obtaned bnary mappng structures can nclude local cross-vew correspondence nformaton n dfferent tranng samples. Correspondence Structure Intalzaton. In ths paper, patch-wse matchng probabltes P (x U, xv ) n the correspondence structure are ntalzed by: 0, f d(x V, x V ) T d P 0 (x U, x V ) 1, (6) d(x V, otherwse xv ) + 1, where x V s x U s co-located patch n camera B. d(xv, xv ) s the dstance between patches x V and x V. It s defned as the number of strdes to move from x V to x V n the zg-zag order. T d s a threshold whch s set to be 32 n ths paper. Accordng to Eq. 6, P 0 (x U, xv ) s nversely proportonal to the co-located dstance between x V and x V and wll equal to 0 f the dstance s larger than a threshold. Bnary mappng structure selecton. Durng each teraton k n the learnng process, we frst apply correspondence structure Θ k 1 A,B = {P k 1 (x U, xv )} from the prevous teraton to calculate the rank orders of all correct match

5 Algorthm 1 Boostng-based Learnng Process Input: A set of tranng probe mages {U α} from camera A and ther correspondng cross-vew mages {V β } from camera B Output: Θ A,B = {P (x U, y V )}, the correspondence structure between {U α} and {V β } 1: Fnd an optmal bnary mappng structure M α for each probe mage U α, as descrbed n the 4-th paragraph n Sec 5.2 2: Set k = 1. Intalze P 0 (x U, y V ) by Eq. 6. 3: Use the current correspondence structure {P k 1 (x U, x V )} to perform Re-ID on {U α} and {V β }, and select 20 bnary mappng structures M α based on the Re-ID result, as descrbed n the 6-th paragraph n Sec 5.2 4: Compute updated match probablty ˆP k (x U, x V ) by Eq. 7 5: Update the matchng probabltes P k (x U, x V ) by Eq. 12 6: Set k = k + 1 and go back to step 3 f not converged or not reachng the maxmum teraton number 7: Output {P (x U, y V )} mages V α n the tranng set. Then, we randomly select 20 V α where half of them are ranked among top 50% (mplyng better Re-ID results) and another half are ranked among the last 50% (mplyng worse Re-ID results). Fnally, we extract bnary mappng structures correspondng to these selected mages and utlze them to update and boost the correspondence structure. Note that we select bnary mappng structures for both hgh- and low-ranked mages n order to nclude a varety of local patch-wse correspondence patterns. In ths way, the fnal obtaned correspondence structure can sutably handle the varatons n human-pose or local vewponts. Calculatng the updated matchng probablty. Wth the ntroducton of the bnary mappng structure M α, we can model the updated matchng probablty n the correspondence structure by: ˆP k (x U, x V ) = ˆP (x U, x V M α ) P (M α ), (7) M α Γ k where ˆP k (x U, xv ) s the updated matchng probablty between patches x U and x V n the k-th teraton. Γk s the set of bnary mappng structures selected n the k-th teraton. R P (M α ) = n(m α) Mγ Γ k R s the pror probablty for b- n(m γ) nary mappng structure M α, where R n (M α ) s the CMC score at rank n [21] when usng M α as the correspondence structure to perform person Re-ID over the tranng mages. n s set to be 5 n our experments. ˆP (x U, xv M α) s the updated matchng probablty between x U and x V when ncludng the local correspondence pattern nformaton of M α. It can be calculated by: ˆP (x U, x V M α ) = ˆP (x V x U, M α ) ˆP (x U M α ), (8) ˆP (x V xu, M α) s the updated probablty to correspond from x U to x V when ncludng M α, calculated as ˆP (x V x U, M α ) { 1, f m{x U,xV } M α, (9),Mα, otherwse à x V x U where m {x U,x V } s a patch-wse lnk connectng x U and x V. à x V x U = Φ z(x U,xV ),Mα x Vt,m{x U Φz(xU,xVt } Mα,xV t ), where Φ z (x U, xv ) s the average appearance smlarty [27, 9] between patches x U and x V over all correct match mage pars n the tranng set. x V t s a patch that s connected to x U n the bnary mappng structure M α. From Eq. 9, ˆP (x V xu, M α) wll equal to 1 f M α ncludes a lnk between x U and x V. Otherwse, ˆP (x V x U, M α) wll be decded by the relatve appearance smlarty strength between patch par {x U, xv } and all patch pars whch are connected to x U n the bnary mappng structure M α. Furthermore, ˆP (x U M α) n Eq. 8 s the updated mportance probablty of x U after ncludng M α. It can be calculated by ntegratng the mportance probablty of each ndvdual lnk n M α : ˆP (x U M α ) = ˆP (x U m {x U s,x V t }, M α ) m {x U s,x V t } Mα ˆP (m {x U s,x V t } M α ), (10) where m {x U s,x V t } s a patch-wse lnk n M α, as the red lnes n Fg. 4a. ˆP (m{x U s,x V t } M α ) s the mportance probablty of lnk m {x U s,x V t } whch s defned smlar to P (M α ): ˆP (m {x U s,x V t } M α ) = R n (m {x U s,x V t } ) m {xuh,xvg } Mα Rn (m {x Uh,x Vg } ), (11) where R n (m {x U s,x V t } ) s the rank-n CMC score [21] when only usng a sngle lnk m {x U s,x V t } as the correspondence structure to perform Re-ID. ˆP (x U m {x U s,xv t }, M α ) n Eq. 10 s the mpact probablty from lnk m {x U s,x V t } to patch x U, defned as: 0, f d(x U ˆP (x U, x U s ) T d m {x U s,x V t }, M α ) 1 d(x U, xu s ) + 1, otherwse where x U s s lnk m {x U s,x V t } s end patch n camera A. d( ) and T d are the same as Eq. 6. Correspondence structure update. Wth the updated matchng probablty ˆP k (x U, xv ) n Eq. 7, the matchng probabltes n the k-th teraton can be fnally updated by: P k (x U, x V ) = (1 ε)p k 1 (x U, x V ) + ε ˆP k (x U, x V ), (12)

6 (a) (b) (c) (d) (e) (f) Fgure 4. (a): An example of bnary mappng structure (the red lnes wth weght 1 ndcate that the correspondng patches are connected). (b)-(d): Examples of the correspondence structures learned by our approach where (b)-(c) and (d) are the correspondence structures for the VIPeR [6] and 3DPeS [1] datasets, respectvely. The lne wdths n (b)-(d) are proportonal to the patch-wse probablty values. (e): The complete correspondence structure matrx of (d) learned by our approach. (f): The correspondence structure matrx of (d) s dataset obtaned by the smple-average method. (Patches n (e) and (f) are organzed by a zg-zag scannng order. Matrces n (e) and (f) are down-sampled for a clearer llustraton of the correspondence pattern). (Best vewed) where P k 1 (x U, xv ) s the matchng probablty n teraton k 1. ε s the update rate whch s set 0.2 n our paper. From Equatons 7 12, our update process ntegrates multple varables (.e., bnary mappng structure, ndvdual lnks, patch-lnk correlaton) nto a unfed probablty framework. In ths way, varous nformaton cues such as appearances, rankng results, and patch-wse correspondence patterns can be effectvely ncluded durng the model updatng process. Besdes, although the exact convergence of our learnng process s dffcult to analyze due to the ncluson of rank score calculaton, our experments show that most correspondence structures become stable wthn 300 teratons, whch mples the relablty of our approach. Fgures 1 and 4 show some examples of the correspondence structures learned from dfferent cross-vew datasets. From Fgures 1 and 4, we can see that the correspondence structures learned by our approach can sutably ndcate the matchng correspondence between spatal msalgned patches. For example, n Fgures 1 and 4d-4e, the large lower-to-upper msalgnments between cameras are effectvely captured. Besdes, the matchng probablty values n the correspondence structure also sutably reflects the correlaton strength between dfferent patch locatons, as dsplayed by the colored ponts n Fgures 1c and 4e. Furthermore, comparng Fgures 1a and 1b, we can see that the human-pose varaton s also sutably handled by the learned correspondence structure. More specfcally, although mages n Fg 1 have dfferent human poses, patches of camera A n both fgures can correctly fnd ther correspondng patches n camera B snce the one-to-many matchng probablty graphs n the correspondence structure sutably embed the local correspondence varaton between cameras. Smlar observatons can also be obtaned from Fgures 4b and 4c. It should be noted that mages n the dataset of Fgures 4b and 4c are taken by unfxed cameras (.e., cameras wth unfxed locatons). However, the correspondence structure learned by our approach can stll effectvely encode the camera-person confguraton and capture the cross-vew correspondence pattern accordngly. 6. Expermental Results We perform experments on the followng four datasets: VIPeR. The VIPeR dataset [6] s a commonly used dataset whch contans 632 mage pars for 632 pedestrans, as n Fgures 4a-4c and 5d. It s one of the most challengng datasets whch ncludes large dfferences n vewpont, pose, and llumnaton between two camera vews. Images from camera A are manly captured from 0 to 90 degree whle camera B manly from 90 to 180 degree. PRID 450S. The PRID 450S dataset [19] conssts of 450 person mage pars from two non-overlappng camera vews. It s also challengng due to low mage qualtes and vewpont changes. 3DPeS. The 3DPeS dataset [1] s comprsed of 1012 mages from 193 pedestrans captured by eght cameras, where each person has 2 to 26 mages, as n Fgures 4d and 5a. Note that snce there are eght cameras wth sgnfcantly dfferent vews n the dataset, n our experments, we group cameras wth smlar vews together and form three camera groups. Then, we tran a correspondence structure between each par of camera groups. Fnally, three correspondence structures are acheved and utlzed to perform Re-ID between dfferent camera groups. For mages from the same camera group, we smply utlze adacencyconstraned search [27] to fnd patch-wse mappng and calculate the mage matchng score accordngly. Road. The road dataset s our own constructed dataset whch ncludes 416 mage pars taken by two cameras wth camera A montorng an ext regon and camera B montorng a road regon, as n Fgures 1 and 5g. 2 Snce mages n ths dataset are taken from a realstc crowd road scene, the nterferences from severe occluson and large pose varaton sgnfcantly ncrease the dffculty of ths dataset. For all of the above datasets, we follow prevous methods [7, 22, 25] and perform experments under 50%-tranng and 50%-testng. All mages are scaled to The 2 Ths dataset wll be open to the publc soon.

7 patch sze n our approach s The strde sze between neghborng patches s 6 horzontally and 8 vertcally for probe mages, and 3 horzontally and 4 vertcally for gallery mages. Note that we use smaller strde sze n gallery mages n order to obtan more patches. In ths way, we can have more flexbltes durng patch-wse matchng Results for patch matchng We compare the patch matchng results of three methods: (1) The adacency-constraned search method [27, 26] whch fnds a best matched patch for each patch n a probe mage (probe patch) by searchng a fxed neghborhood regon around the probe patch s co-located patch n a gallery mage (Adacency-constraned). (2) The smple-average method whch smply averages the bnary mappng structures for dfferent probe mages (as n Fg. 4a) to be the correspondence structure and combnes t wth a global constrant to fnd the best one-to-one patch matchng result (Smple-average). (3) Our approach whch employs a boostng-based process to learn the correspondence structure and combnes t wth a global constrant to fnd the best one-to-one patch matchng result. Fg. 5 shows the patch mappng results of dfferent methods, where sold lnes represent matchng probabltes n a correspondence structure and red-dashed lnes represent patch matchng results. Besdes, Fgures 4e and 4f show one example of the correspondence structure matrx obtaned by our approach and the smple-average method, respectvely. From Fgures 5 and 4e-4f, we can observe: (1) Snce the adacency-constraned method searches a fxed neghborhood regon wthout consderng the correspondence pattern between cameras, t may easly be nterfered by wrong patches wth smlar appearances n the neghborhood (cf. Fgures. 5d, 5g). Comparatvely, wth the ndcatve matchng probablty nformaton n the correspondence structure, the nterference from msmatched patches can be effectvely reduced (cf. Fgures. 5f, 5). (2) When there are large msalgnments between cameras, the adacency-constraned method may fal to fnd proper patches as the correct patches may be located outsde the neghborhood regon, as n Fg. 5a. Comparatvely, the large msalgnment pattern between cameras can be properly captured by our correspondence structure, resultng n a more accurate patch matchng result (cf. Fg. 5c). (3) Comparng Fgures 4e, 4f wth the last two columns n Fg. 5, t s obvous that the correspondence structures by our approach s better than the smple average method. Specfcally, the correspondence structures by the smple average method nclude many unsutable matchng probabltes whch may easly result n wrong patch matches. In contrast, the correspondence structures by our approach are more coherent wth the actual spatal correspondence pattern between cameras. Ths mples that relable correspon- (a) (b) (c) (d) (e) (f) (g) (h) () Fgure 5. Comparson of dfferent patch mappng methods. Left column: the adacency-constraned method; Mddle column: the smple-average method; Last column: our approach. The sold lnes represent matchng probabltes n a correspondence structure and the red-dashed lnes represent patch matchng results. Note that the mage par n (a)-(c) ncludes the same person (.e., correct match) whle the mage pars n (d)-() nclude dfferent people (.e., wrong match). (Best vewed n color) dence structure cannot be easly acheved wthout sutably ntegratng the nformaton cues between cameras Results for person re-dentfcaton We evaluate person re-dentfcaton results by the standard Cumulated Matchng Characterstc (CMC) curve [21] whch measures the correct match rates wthn dfferent Re- ID rank ranges. The evaluaton protocols are the same as [7]. That s, for each dataset, we perform 10 randomlyparttoned 50%-tranng and 50%-testng experments and average the results. We compare results of four methods: (1) Not applyng correspondence structure and drectly usng the appearance smlarty between co-located patches for person Re- ID (No-structure); (2) Smply averagng the bnary mappng structures for dfferent probe mages as the correspondence structure and utlzng t for Re-ID (Smple-average); (3) Usng the correspondence structure learned by our approach, but do not nclude global constrant when performng Re-ID (No-global); (4) Our approach (Proposed). We also compare our results wth state-of-the-art methods on dfferent datasets: klfda [22], esdc-ocsvm [27], KISSME [19], Salence [26], svmml [13], RankBoost [11]

8 (a) the VIPeR dataset (b) the PRID 450S dataset (c) the 3DPeS dataset Fgure 6. CMC curves for dfferent methods. Table 1. CMC results on the VIPeR dataset Rank klfda[22] KISSME[19] Salence[26] svmml[13] RankBoost[11] esdc-ocsvm[27] LF[18] No-structure Smple-average No-global Proposed and LF [18] on the VIPeR dataset; KISSME [19], EIML [8], SCNCD [25], SCNCDFnal [25] on the PRID 450S dataset; klfda [22], rpcca [22], PCCA [16] on the 3DPeS dataset; and esdc-knn [27] on the Road dataset. Tables 1 4 and Fg. 6 show the CMC results of dfferent methods. From the CMC results, we can see that: (1) Our approach has better Re-ID performances than the state-ofthe-art methods. Ths demonstrates the effectveness of our approach. (2) Our approach has obvously mproved results than the no-structure method. Ths ndcates that proper correspondence structures can effectvely mprove Re-ID performances by reducng patch-wse msalgnments. (3) The smple-average method has smlar performance to the no-structure method. Ths mples that unsutably selected correspondence structures cannot mprove Re-ID performance. (4) The no-global method also has good Re-ID performance. Ths further demonstrates the effectveness of the correspondence structure learned by our approach. Meanwhle, our approach also has superor performance than the no-global method. Ths demonstrates the usefulness of ntroducng global constrant n the patch matchng process. 7. Concluson In ths paper, we propose a novel framework for addressng the problem of cross-vew spatal msalgnments n person Re-ID. Our framework conssts of two key ngre- Table 2. CMC results on the PRID 450S dataset Rank KISSME[19] EIML[8] SCNCD[25] SCNCDFnal[25] No-structure Smple-average No-global Proposed Table 3. CMC results on the 3DPeS dataset Rank klfda[22] rpcca[22] PCCA[16] No-structure Smple-average No-global Proposed Table 4. CMC results on the Road dataset Rank esdc-knn[27] No-structure Smple-average No-global Proposed dents: 1) ntroducng the dea of correspondence structure and learnng ths structure va a novel boostng method to adapt to arbtrary camera confguratons; 2) a constraned global matchng step to control the patch-wse msalgnments between mages due to local appearance ambguty. Extensve expermental results on benchmark show that our approach acheves the state-of-the-art performance. Under ths framework, our future work s devoted to explore new varants of the two components, such as: 1) desgnng other correspondence structure learnng methods that allow for multple structure canddates to enhance ts flexblty; 2) devsng and ncorporatng edge-to-edge smlarty metrcs for solvng the constraned global matchng problem as graph matchng [4, 24], whch has been proven more effectve n many computer vson applcatons.

9 References [1] D. Balter, R. Vezzan, and R. Cucchara. 3dpes: 3d people dataset for survellance and forenscs. In ACM workshop Human gesture and behavor understandng, [2] C. Barnes, E. Shechtman, D. B. Goldman, and A. Fnkelsten. The generalzed patchmatch correspondence algorthm. In ECCV, [3] D. Cheng, M. Crstan, M. Stoppa, L. Bazzan, and V. Murno. Custom pctoral structures for re-dentfcaton. In BMVC, [4] M. Cho, J. Lee, and K. M. Lee. Reweghted random walks for graph matchng. In ECCV, [5] M. Farenzena, L. Bazzan, A. Perna, V. Murno, and M. Crstan. Person re-dentfcaton by symmetry-drven accumulaton of local features. In CVPR, [6] D. Gray, S. Brennan, and H. Tao. Evaluatng appearance models for recognton, reacquston, and trackng. In PETS, , 6 [7] D. Gray and H. Tao. Vewpont nvarant pedestran recognton wth an ensemble of localzed features. In ECCV, , 2, 6, 7 [8] M. Hrzer, P. M. Roth, and H. Bschof. Person redentfcaton by effcent mpostor-based metrc learnng. In AVSS, , 2, 8 [9] M. Kostnger, M. Hrzer, P. Wohlhart, P. M. Roth, and H. Bschof. Large scale metrc learnng from equvalence constrants. In CVPR, , 2, 3, 5 [10] H. W. Kuhn. The hungaran method for the assgnment problem. Naval Research Logstcs Quarterly, , 4 [11] C.-H. Kuo, S. Khams, and V. Shet. Person re-dentfcaton usng semantc color names and rankboost. In WACV, , 2, 7, 8 [12] I. Kvatkovsky, A. Adam, and E. Rvln. Color nvarants for person redentfcaton. IEEE Trans. PAMI, , 2 [13] Z. L, S. Chang, F. Lang, T. S. Huang, L. Cao, and J. R. Smth. Learnng locally-adaptve decson functons for person verfcaton. In CVPR, , 8 [14] C. Lu, S. Gong, and C. C. Loy. On-the-fly feature mportance mnng for person re-dentfcaton. Pattern Recognton, [15] L. Ma, X. Yang, Y. Xu, and J. Zhu. A generalzed emd wth body pror for pedestran dentfcaton. Journal of Vsual Communcaton and Image Representaton, , 2, 4 [16] A. Mgnon and F. Jure. PCCA: A new approach for dstance learnng from sparse parwse constrants. In CVPR, [17] O. Orefe, R. Mehran, and M. Shah. Human dentty recognton n aeral mages. In CVPR, , 2 [18] S. Pedagad, J. Orwell, S. Velastn, and B. Boghossan. Local fsher dscrmnant analyss for pedestran re-dentfcaton. In CVPR, [19] P. M. Roth, M. Hrzer, M. Köstnger, C. Belezna, and H. Bschof. Mahalanobs dstance learnng for person redentfcaton. In Person Re-Identfcaton. Sprnger, , 2, 6, 7, 8 [20] H. Wang, S. Gong, and T. Xang. Unsupervsed learnng of generatve topc salency for person re-dentfcaton. In BMVC, [21] X. Wang, G. Doretto, T. Sebastan, J. Rttscher, and P. Tu. Shape and appearance context modelng. In ICCV, , 7 [22] F. Xong, M. Gou, O. Camps, and M. Sznaer. Person redentfcaton usng kernel-based metrc learnng methods. In ECCV, , 2, 6, 7, 8 [23] Y. Xu, L. Ln, W.-S. Zheng, and X. Lu. Human redentfcaton by matchng compostonal template wth cluster samplng. In ICCV, [24] J. Yan, C. Zhang, H. Zha, X. Yang, W. Lu, and S. M. Chu. Dscrete hyper-graph matchng. In CVPR, [25] Y. Yang, J. Yang, J. Yan, S. Lao, D. Y, and S. Z. L. Salent color names for person re-dentfcaton. In ECCV, , 2, 6, 8 [26] R. Zhao, W. Ouyang, and X. Wang. Person re-dentfcaton by salence matchng. In ICCV, , 3, 7, 8 [27] R. Zhao, W. Ouyang, and X. Wang. Unsupervsed salence learnng for person re-dentfcaton. In CVPR, , 2, 3, 4, 5, 6, 7, 8