Marker-less Motion Capture

Transcription

1 Marker-less Min Capre Seminar n Cmpainal Phgraph and Videgraph WS 9/ Aliaksandr alaika 3... raking Peple wih wiss and Epnenial Maps997 B Chrisph Bregler and Jiendra Malik. Deailed Hman Shape and Pse frm mages7 b Aleandr Balan Lenid Sigal Mihael Blak James Dais and Hrs Hasseker

2 Prblem saemen Perfrms min Perfrms same min Ar Digial mdel Min apre

3 Min apre wih markers rak markers Creae digial mdel Eqipmen: Clhes + Markers Ar

4 Usage f min apre Filmmaking Games B wha if we d n hae an markers?

5 Marker-less min apre Gra-saled image seqene hisri fage rerded b Eadweard Mbridge in 884 mage seqene frm ne r man ameras N markers Era min Era parameers frm min Creae digial mdel

6 mage represenain is image inensibrighness f an image.

7 Opial flw w nseie images s here an min?

8 Opial flw Sbra bh images s here an min? abs

9 mage inensi Assme ha hange in image inensi mes nl frm min. where min mdel Hw ge?

10 mage inensi empral gradien Gradien nsrain eqain fr image inensi Spaial gradien Min mdel

11 Min Min a hange f psiin and rienain. + Rigid bd llein f pariles sh ha disane beween an w remains fied.

12 Min MdelD fr a single rigid bd n ase f affine min mdel: d d a a a a where M d d a a a a d d a a a a ] [ Min mdel Rigid bd in a plane Rigid bd in spae min parameers Hw ge parameers? M

13 Oer-defined eqain ssem Regin f N> piels M M N N N N N eqains J 5 5

14 Newn-Raphsn-sle minimiain J Slin wih linear leas sqares esimain J J J eraie sep:. warp he + image wih a new.. Cmpe new image gradiens 3. Sle wih leas sqares esimain

15 mage sppr map Apprimae rigid bd wih sppr map Eer piel wihin sppr map is par f he rigid bd w W where w w n n w i i {} Appl sppr map WJ J WJ

16 Pse f a single rigid bd Camera frame q [ ] Saled rhgraphi prjein in he image [ im im] s [ ] Obje frame q [ ] Pse f bje represened as bd ransfrmain mari mage ransfrmain mari G r r r 3 r r r 3 r r r d d d q G q Camera Obje pin pin

17 wis wis wih anglar eli wih and p p p q q p q p ˆ ˆ ˆ ˆ 3 3 p q radial eli Anglar eli

18 Rigid bd ransfrmain in epnenial represenain ˆ p Differenial eqain p Radial eli p e ˆ p Slin q p q G q G r r r 3 r r r 3 r r r d d d Rigid bd ransfrmain mari e ˆ ˆ ˆ! ˆ 3! 3 Epnenial represenain

19 Mapping frm bje frame he image Saled rhgraphi prjein in he image [ im im] s [ ] Camera frame q [ ] Obje frame q [ ] [ s ] [ s 3 ] Pse f bje im im s e ˆ ransfrm. mari q Prjein f pin q image lain mage pin Sale far Obje pin

20 wis min mdel im im im im q e s q e s ˆ ˆ q s s

21 wis min mdel fr a rigid bd mage min f a pin nsidering and 3D pin. And we ge r gradien eqain ha we an sle in he same wa as fr D ase. q s s ] [ ] [ ] [ s H q s q s

22 Kinemai Chain Degree f freedmdof = Nmber f jins Link Jin

23 Kinemai Chain Camera frame q [ ] q Obje frame q [ ] Desribe eah jin b a wis hen g G e q ˆ Pse f bje Pse Map bje frame q g q Amn f rain

24 Prd f epnenial maps Chain f K jins is mdeled wih prd f epnenial maps g k k G e ˆ e ˆ k k Radial eli f bd K Veli f a link V V k d ˆ e d wis ˆ e Anglar eli angle k ˆ k Camera frame q [ ] Pse f bje q Obje frame q [ ] q

25 Esimain f pse and anglar hange Min k k k im im im im s where q q e s ] [ ˆ ˆ ˆ And gradien eqain ] [ J H

26 Mliple amera iews Gradien eqain fr ne iew [ H J [ where ] [ s ] k Pse and anglar parameers ] Pse 3 Anglar parameers are same in all iews Pse is differen 4 k

27 Mliple amera iews B mbining all eqains fr all N iews N K N N N J H J H J H

28 Oeriew np images + pse and angles fr G Op pse and angles fr + G k k. Cmpe fr eah image lain he 3D pin q. Cmpe fr eah rigid bd he sppr map W k 3. Se G : G k : k : k 4. erae a Cmpe b Esimae parameers Updae G sing d Warp + wih new k G k

29 Min apre wih wis min mdel Vide

30 SCAPE s. Clindrial mdel Oerlap f image silhees and differen mdels SCAPE has beer erlapping

31 Alernaie apprah : SCAPE mdel Use daabase f deailed 3D sans f mliple peple We an parameerie pse and shape defrmain Adjs hse parameers fi he inp image

32 SCAPE mdel np images erlaid wih esimaed bd mdel. Oerlapellw beween silhee red and esimaed mdel ble. Reered mdel frm eah amera iew.

33 Min apre wih SCAPE mdel Vide

34 Cnlsin w apprahes N markers Ge min parameers sing wis min mdel nsan image inensi sppr maps pimiain Ge min and shape parameers sing SCAPE mdel daabase wih sans fi mdel and image

35 hank Y!