Content of the Lecture. Computer Vision 2 Lecture 5. Recap: Tracking-by-Detection. Recap: Elements of Tracking

Transcription

1 9.5.6 Conen of he Lecure Couer Vision Lecure 5 Trcking wih Liner Dnic Models(.5.6 rof. Dr. Bsin Leibe Dr. Jörg Sückler leibe@ision.rwh-chen.de sueckler@ision.rwh-chen.de RWTH Achen Uniersi Couer Vision Grou h:// Single-Objec Trcking Besin Filering Kln Filers EKF ricle Filers Muli-Objec Trcking Visul Odoer Visul SLAM & 3D Reconsrucion Mesureens ie Ses Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Rec: Trcking-b-Deecion Rec: Eleens of Trcking Min ides Al generic objec deecor o find objecs of cerin clss Bsed on he deecions erc objec ernce odels Link deecions ino rjecories 3 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Deecion Where re cndide objecs? D ssociion Which deecion corresonds o which objec? redicion Where will he rcked objec be in he ne ie se? 4 Deecion D ssociion redicion Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Ls lecure Rec: Sliding-Window Objec Deecion Rec: Objec Deecor Design Fleshing ou his ieline bi ore we need o:. Obin rining d. Define feures 3. Define clssifier Trining eles In rcice he clssifier ofen deerines he design. Tes of feures Seedu sregies We looked 3 se-of-he-r deecor designs Bsed on SVMs Bsed on Boosing Cr/non-cr Clssifier Bsed on CNNs Feure ercion 5 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler 6 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Krisen Grun

2 9.5.6 Rec: Hisogrs of Oriened Grdiens (HOG Holisic objec reresenion Loclized grdien orienions 7 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide ded fro Nnee Dll Objec/Non-objec Liner SVM Collec HOGs oer deecion window Conrs norlize oer oerling sil cells Weighed oe in sil & orienion cells Coue grdiens G coression Ige Window Rec: Deforble r-bsed Model (DM Muliscle odel cures feures wo resoluions 8 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: edro Felzenszwlb Score of filer: do roduc of filer wih HOG feures underneh i Score of objec hohesis is su of filer scores inus deforion coss [Felzenszwlb McAlliser Rnn CVR 8] Rec: DM Hohesis Score Rec: Inegrl Chnnel Feures Generlizion of Hr Wele ide fro Viol-Jones Insed of onl considering inensiies lso ke ino ccoun oher feure chnnels (grdien orienions color eure. Sill efficienl reresened s inegrl iges. [Felzenszwlb McAlliser Rnn CVR 8]. Dollr Z. Tu. eron S. Belongie. Inegrl Chnnel Feures BMVC 9. 9 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: edro Felzenszwlb Rec: Inegrl Chnnel Feures Rec: VerFs Deecor Ide : Iner he ele scle s. ige scle relion Generlize lso block couion s order feures: Su of iels in recngulr region. nd -order feures: Hr-like difference of su-oer-blocks Generlized Hr: More cole cobinions of weighed recngles Hisogrs Coued b eluing locl sus on qunized iges. Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler odel 5 ige scles Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler 5 odels ige scle R. Benenson M. Mhis R. Tiofe L. Vn Gool. edesrin Deecion Fres er Second CVR. Slide credi: Rodrigo Benenson

3 9.5.6 Rec: VerFs Deecor Ide : Reduce rining ie b feure inerolion Rec: VerFs Clssifier Consrucion Shown o be ossible for Inegrl Chnnel feures. Dollár S. Belongie eron. The Fses edesrin Deecor in he Wes BMVC. 3 5 odels ige scle Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Rodrigo Benenson 5 odels ige scle score = w h + w h + +w N h N Enseble of shor rees lerned b AdBoos Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Rodrigo Benenson Rec: Eleens of Trcking Tod: Trcking wih Liner Dnic Models Deecion Where re cndide objecs? D ssociion Which deecion corresonds o which objec? redicion Where will he rcked objec be in he ne ie se? 5 Deecion D ssociion redicion Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Ls lecure Tod s oic 6 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Figure fro Forsh & once Toics of This Lecure Trcking wih Dnics Deecion s. Trcking Trcking s robbilisic inference redicion nd Correcion Liner Dnic Models Zero eloci odel Consn eloci odel Consn ccelerion odel The Kln Filer Kln filer for D se Generl Kln filer Liiions Trcking wih Dnics Ke ide Gien odel of eeced oion redic where objecs will occur in ne fre een before seeing he ige. Gols Resric serch for he objec Iroed esies since esureen is reduced b rjecor soohness. Assuion: coninuous oion erns Cer is no oing insnl o new iewoin. Objecs do no diser nd reer in differen lces. Grdul chnge in ose beween cer nd scene. 7 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler 8 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide ded fro S. Lzebnik K. Grun 3

4 9.5.6 Generl Model for Trcking Reresenion The oing objec of ineres is chrcerized b n underling se. Se gies rise o esureens or obserions Y. A ech ie he se chnges o nd we ge new obserion Y. Se s. Obserion Y Y Y Hidden se : reers of ineres Mesureen: wh we ge o direcl obsere 9 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Seln Lzebnik Slide credi: Krisen Grun Trcking s Inference Inference roble The hidden se consiss of he rue reers we cre bou denoed. The esureen is our nois obserion h resuls fro he underling se denoed Y. A ech ie se se chnges (fro - o nd we ge new obserion Y. Our gol: recoer os likel se gien All obserions seen so fr. Knowledge bou dnics of se rnsiions. Ses of Trcking redicion: Wh is he ne se of he objec gien s esureens? Y Y Correcion: Coue n uded esie of he se fro redicion nd esureens. Y Y Trcking Cn be seen s he rocess of roging he oserior disribuion of se gien esureens cross ie. Y Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Krisen Grun Silifing Assuions Onl he iedie s ers Dnics odel Mesureens deend onl on he curren se Y Y Y Y Obserion odel Trcking s Inducion Bse cse: Assue we he iniil rior h redics se in bsence of n eidence: ( A he firs fre correc his gien he lue of Y = ( ( ( Y ( ( ( oserior rob. of se gien esureen Likelihood of esureen rior of he se Y Y Y 3 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler 4 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Seln Lzebnik Slide credi: Seln Lzebnik 4

5 Trcking s Inducion Bse cse: Assue we he iniil rior h redics se in bsence of n eidence: ( A he firs fre correc his gien he lue of Y = Gien correced esie for fre : redic for fre + Correc for fre + Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Seln Lzebnik redic correc 6 Inducion Se: redicion redicion inoles reresening gien Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler d d d Lw of ol robbili A A B db Slide credi: Seln Lzebnik 7 Inducion Se: redicion redicion inoles reresening gien Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Seln Lzebnik d d d Condiioning on A B A B B 8 Inducion Se: redicion redicion inoles reresening gien Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Seln Lzebnik d d d Indeendence ssuion 9 Inducion Se: Correcion Correcion inoles couing gien rediced lue Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler d Bes rule B A A A B B Slide credi: Seln Lzebnik 3 Inducion Se: Correcion Correcion inoles couing gien rediced lue Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Seln Lzebnik d Indeendence ssuion (obserion deends onl on se

6 9.5.6 Inducion Se: Correcion Correcion inoles couing gien rediced lue 3 d Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Seln Lzebnik Condiioning on Sur: redicion nd Correcion redicion: 3 d Dnics odel Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Seln Lzebnik Correced esie fro reious se Sur: redicion nd Correcion redicion: Correcion: 33 d Dnics odel Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Obserion odel Correced esie fro reious se rediced esie d Toics of This Lecure Trcking wih Dnics Deecion s. Trcking Trcking s robbilisic inference redicion nd Correcion Liner Dnic Models Zero eloci odel Consn eloci odel Consn ccelerion odel The Kln Filer Kln filer for D se Generl Kln filer Liiions 34 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Seln Lzebnik Noion Reinder ~ N( μ Σ Rndo rible wih Gussin robbili disribuion h hs he en ecor ¹ nd corince ri. nd ¹ re d-diensionl is d d. d = d = If is D we jus he one reer: he rince ¾ Liner Dnic Models Dnics odel Se undergoes liner rnforion D lus Gussin ~ N D n d nn Obserion odel Mesureen is linerl rnsfored se lus Gussin n ~ N M n n 35 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler 36 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Krisen Grun Slide credi: Seln Lzebnik Krisen Grun 6

7 Ele: Rndol Drifing oins Consider sionr objec wih se s osiion. osiion is consn onl oion due o rndo er. Se eoluion is described b ideni ri D=I Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler D I Slide credi: Krisen Grun 38 Ele: Consn Veloci (D oins Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler ie Mesureens Ses Slide credi: Krisen Grun Figure fro Forsh & once 39 Ele: Consn Veloci (D oins Se ecor: osiion nd eloci Mesureen is osiion onl Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler ( D (greek leers denoe ers M Slide credi: Seln Lzebnik Krisen Grun 4 Ele: Consn Veloci (D oins Se ecor: osiion nd eloci Mesureen is osiion onl Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler ( D (greek leers denoe ers M Slide credi: Seln Lzebnik Krisen Grun 4 Ele: Consn Accelerion (D oins Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Krisen Grun Figure fro Forsh & once 4 Ele: Consn Accelerion (D oins Se ecor: osiion eloci nd ccelerion. Mesureen is osiion onl Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler ( ( ( D (greek leers denoe ers M Slide credi: Seln Lzebnik Krisen Grun

8 9.5.6 Ele: Consn Accelerion (D oins Rec: Generl Moion Models Se ecor: osiion eloci nd ccelerion. ( ( (greek leers denoe ( ers D Mesureen is osiion onl M 43 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Assuing we he differenil equions for he oion E.g. for (undened eriodic oion of liner sring d d Subsiue ribles o rnsfor his ino liner sse d d 3 d d Then we he ( ( 3 ( 3 D Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Seln Lzebnik Krisen Grun Toics of This Lecure Trcking wih Dnics Deecion s. Trcking Trcking s robbilisic inference redicion nd Correcion Liner Dnic Models Zero eloci odel Consn eloci odel Consn ccelerion odel The Kln Filer Kln filer for D se Generl Kln filer Liiions The Kln Filer Kln filer Mehod for rcking liner dnicl odels in Gussin The rediced/correced se disribuions re Gussin You onl need o inin he en nd corince. The clculions re es (ll he inegrls cn be done in closed for. 45 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler 46 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Seln Lzebnik The Kln Filer Know correced se fro reious ie se nd ll esureens u o he curren one redic disribuion oer ne se. Men nd sd. de. of rediced se: Tie ude ( redic Receie esureen Tie dnces: ++ Know redicion of se nd ne esureen Ude disribuion oer curren se. Mesureen ude ( Correc Men nd sd. de. of correced se: Kln Filer for D Se Wn o reresen nd ude N ( N ( 47 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler 48 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Krisen Grun 8

9 osiion rogion of Gussin densiies D Kln Filer: redicion Shifing he en He liner dnic odel defining rediced se eoluion wih ~ N d d Wn o esie rediced disribuion for ne se N ( Ude he en: d for deriions see F& Cher Besin ude Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Incresing he rince Ude he rince: 5 ( ( d d Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Krisen Grun D Kln Filer: Correcion redicion s. Correcion He liner odel defining he ing of se o esureens: Y ~ N Wn o esie correced disribuion gien les esureen: N ( Ude he en: Ude he rince: 5 ( ( Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Krisen Grun Deriions: F& Cher 7.3 ( ( ( ( ( ( Wh if here is no redicion uncerin (? 5 ( ( Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler The esureen is ignored! ( Wh if here is no esureen uncerin (? ( The redicion is ignored! Slide credi: Krisen Grun Recll: Consn Veloci Ele Consn Veloci Model esureens se o se esureen * rediced en esie + correced en esie brs: rince esies before nd fer esureens 53 ie Se is D: osiion + eloci Mesureen is D: osiion Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler 54 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Krisen Grun Figure fro Forsh & once Slide credi: Krisen Grun Figure fro Forsh & once 9

10 9.5.6 Consn Veloci Model Consn Veloci Model o se esureen * rediced en esie + correced en esie brs: rince esies before nd fer esureens o se esureen * rediced en esie + correced en esie brs: rince esies before nd fer esureens 55 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler 56 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide credi: Krisen Grun Figure fro Forsh & once Slide credi: Krisen Grun Figure fro Forsh & once Consn Veloci Model Kln Filer: Generl Cse (>di o se REDICT CORRECT esureen * rediced en esie + correced en esie brs: rince esies before nd fer esureens T T D K M M M residul T D D K d M Kln gin I KM 57 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler for deriions see F& Cher Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler More weigh on residul when esureen error corince roches. Less weigh on residul s riori esie error corince roches. Slide credi: Krisen Grun Figure fro Forsh & once Slide credi: Krisen Grun Sur: Kln Filer ros: Gussin densiies eerwhere Sile udes coc nd efficien Ver esblished ehod er well undersood Cons: Uniodl disribuion onl single hohesis Resriced clss of oions defined b liner odel Wh Is This A Resricion? Mn ineresing cses don he liner dnics E.g. edesrins wlking E.g. bll bouncing 59 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler 6 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler Slide ded fro Seln Lzebnik

11 9.5.6 Bll Ele: Wh Goes Wrong Here? References nd Furher Reding Assuing consn ccelerion odel redicion is oo fr fro rue osiion o coense ossible soluion: Kee ulile differen oion odels in rllel I.e. would check for bouncing ech ie se 6 redicion Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler redicion redicion 3 redicion 4 redicion 5 Correc redicion A er good inroducion o rcking wih liner dnic odels nd Kln filers cn be found in Cher 7 of 6 D. Forsh J. once Couer Vision A Modern Aroch. renice Hll 3 Lecure: Couer Vision (SS 6 Tele-bsed Trcking rof. Dr. Bsin Leibe Dr. Jörg Sückler