Academc Center for Computng and Meda Studes, Kyoto Unversty Collaboratvely Regularzed Nearest Ponts for Set Based Recognton Yang Wu, Mchhko Mnoh, Masayuk Mukunok Kyoto Unversty 9/1/013 BMVC 013 @ Brstol, UK 1
Outlne Why set-based recognton? Related work Regularzed Nearest Ponts (RNP) Collaboratve Regularzed Nearest Ponts (CRNP) Expermental results Fndngs and future work 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan
Why set-based recognton? From sngle-nstance nstance recognton to set-based recognton: Sngle-nstance recognton P1 P Pn Set-based recognton P1 P Pn Who? Who? Who? Who? Who? Collectng a set of mages for recognton becomes ncreasngly convenent. > Takng and sharng pctures/vdeos gets easer The drecton of set based recognton recently gets hotter and hotter. > Face recognton > Person re-dentfcaton (multple-shot) Set based recognton models have the potental to outperform sngle-nstance based recognton approaches under the same condtons. 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 3
Related work Exstng solutons 1. Set-based sgnature generaton -- Largely explored for person re-dentfcaton. -- Compatble wth sngle nstance based learnng algorthms. -- Needs manual desgn, whch s task-dependent and hard. 3. Geometrc dst. fndng 1 -- Manly for face recognton. -- Explores set structure. -- Robust to noses/outlers. -- Unsupervsed (can be supervsed).. Drect set-to-set matchng -- Uses smple mnmum pont-wse dstance for set-to-set matchng. -- Reles on good features for sngle nstances. -- Senstve to noses/outlers. -- Unsupervsed. 3 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 4
Related work Set-to-set dstance fndng X 1 Q -- Query/Probe Set X, {1,, n} -- Gallery Sets X 1 Q Q X Q Q X X n X n (MPD, AHISD/CHISD, SANP/KSANP, SBDR, RNP) (CSA) 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 5
Regularzed Nearest Ponts (RNP) Regularzed Nearest Ponts dstance fndng RNP models each mage set by a regularzed affne hull (RAH): RAH xxα k 1, α, k Yang et al., FG 13 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 6
Regularzed Nearest Ponts (RNP) Regularzed Nearest Ponts dstance fndng RNP models each mage set by a regularzed affne hull (RAH): RAH xxα k 1, α, RNP fnds two nearest ponts from the RAH of Q and the RAH of by solvng αβ, k 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 7 X, respectvely st k k j j 1 mn Qα X β,.. 1, 1, α, β, whch can be solved by where αβ, 1 st k k j j mn Qα X β α β,.. 1, 1, k 1, j 1 help avodng the trval soluton α β 0 k j
Regularzed Nearest Ponts (RNP) Regularzed Nearest Ponts classfcaton After gettng the soluton Q * * α, β, the set-to-set dstance between * * RNP Q X * Qα X * β d, Q and where s the nuclear norm of,.e. the sum of the sngular values of t. * Q The nuclear norm term reflects the representaton ablty (related to the sze) of a set, thus beng able to remove the possble dsturbance unrelated to the class nformaton. X s defned to be Fnally, Q s classfed by: C Q d RNP arg mn. 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 8
Collaboratvely Regularzed Nearest Ponts (CRNP) Collaboratvely Regularzed Nearest Ponts Collaboratve dstance fndng RNP: mn Qα X β α β, st.. 1, 1, αβ, 1 k k j j CRNP solves the followng optmzaton problem: αβ, n 1 k k 1 j mn,.. 1, j Qα Xβ α β st 1, where X[ X,, X ] 1 n β[ β,, β ] T T T 1 n 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 9
Collaboratvely Regularzed Nearest Ponts (CRNP) Collaboratvely Regularzed Nearest Ponts Dstance fndng optmzaton αβ, n j 1 1 k k 1 j mn Qα Xβ α β (1 ) (1 ),, mn zqα Xβ α β, αβ 1 z [ 01,, 1, ] T m T Q [ Q, 1, 0 ] 1 N,1 N,1 T X[ X, 0, 1 ] x q q T N,1 N,1 x T 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 10
Collaboratvely Regularzed Nearest Ponts (CRNP) Collaboratvely Regularzed Nearest Ponts Dstance fndng optmzaton, mn zqα Xβ α β, αβ 1 One-step closedform soluton? But, Yes! -- t s expensve, -- the whole optmzaton s needed for each query/probe set. Iteratve Optmzaton: β α * α P ( ), q zxβ wth ( T ) T 1 P. q Q Q 1 I Q Fx, and optmze : Fx α, and optmze β : * β P ( ), x zqα wth ( T ) T 1 P. x X X I X 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 11
Collaboratvely Regularzed Nearest Ponts (CRNP) Collaboratvely Regularzed Nearest Ponts Dstance fndng optmzaton 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 1
Collaboratvely Regularzed Nearest Ponts (CRNP) Collaboratvely Regularzed Nearest Ponts Classfcaton Lke sparse/collaboratve representaton models for sngle-nstance based * * * recognton, here the set-specfc coeffcents β [ β1,, βn] s mplctly made to have some dscrmnaton power. Therefore, we desgn our classfcaton model as follows. where C Q d CRNP arg mn, * * * CRNP Q X / * Qα X * β β d. Recall that RNP doesn t drectly use the coeffcents themselves whch are actually also dscrmnatve. * * RNP Q X * Qα X * β d, 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 13
Expermental Results Expermental Results Expermental settngs -- datasets Face recognton. Honda/UCSD dataset and CMU MoBo dataset: 1. Honda/UCSD 0 subjects (0 specfed seq. for the gallery, and the other 39 seq. for testng.);. CMU MoBo -- 4 subjects (randomly select 1 seq. out of 4 for each subject for the gallery, and the rest for testng.). 3. The gallery/probe set sze for both datasets s set to be 50 or 100 (collected from the begnnng of each sequence.) Person re-dentfcaton. 3 wdely used datasets: LIDS-MA, LIDS-AA, and CAVIAR4REID. - LIDS-MA: 40 subjects, 1 gallery set & 1 probe set for each, set sze 10; - LIDS-AA: 100 subjects, 1 gallery set & 1 probe set for each, set sze 10; - CAVIAR4REID : 50 subjects, 1 gallery set & 1 probe set for each, set sze 5; 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 14
Expermental Results Expermental Results Expermental settngs -- comparsons Methods. MPD (CVPR10), SRC (TPAMI09), CRC (ICCV11), CHISD (CVPR10), SANP (CVPR11), KSANP (PAMI1), SBDR (ECCV1), CSA (AVSS1), RNP (FG13). Parameters. For CRNP: 4, 1 For other methods: 1 1 - default settngs or orgnally suggested parameters were used. 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 15
Expermental Results Expermental Results Results Face recognton accuracy (%) comparson on the Honda/UCSD dataset. Face recognton accuracy (%) comparson on the CMU MoBo dataset. Performance comparson for person re-dentfcaton on three benchmark datasets. 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 16
Expermental Results Expermental Results Computatonal cost For those methods whch can have (parts of) ther models pre-computed usng the tranng data, the total pre-computaton tme (n seconds) s lsted for comparson. Computatonal cost comparson wth all the related methods on all of the recognton tasks (n the ``mllseconds per sample'' manner, excludng the tme for feature extracton). 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 17
Fndngs and Future Work Fndngs and Future Work Fndngs Collaboratve representaton s effectve for set-based recognton. The computatonally effcent L-norm based regularzaton works well wth collaboratve representaton. Future work A deeper comparson of dfferent norms (ncludng L0, L1, and L) n the same framework of CRNP. Dctonary learnng for performance mprovement. Code: avalable soon on my personal webpage. http://mm.meda.kyoto-u.ac.jp/members/yangwu/ 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 18
Academc Center for Computng and Meda Studes, Kyoto Unversty Thank you! Q & A? 9/1/013 Yang Wu, et al., Kyoto Unversty, Japan 19