Prof. Feng Liu. Spring /24/2017

Transcription

1 Prof. Feng Lu Sprng 2017 ttp:// 05/24/2017

2 Last me Compostng and Mattng 2

3 oday Vdeo Stablzaton Vdeo stablzaton ppelne 3

4 Orson Welles, ouc of Evl,

5 Images courtesy Peter Sand and Flckr user Carles W. Brown 5

6 6

7 7

8 [Lu et al. 09] 8

9 Input Stablzaton result 9

10

11 Input rajectory Estmaton Moton Model Estmaton Moton Plan Vdeo ransform Output

12 Input rajectory Estmaton Moton Model Estmaton Moton Plan Vdeo ransform Output Feature trajectores

13 Input rajectory Estmaton Moton Model Estmaton Moton Plan Vdeo ransform Output Feature trajectores

14

15 Brgtness Constancy Equaton: Lnearzng te rgt sde usng aylor epanson: ), ( 1),, ( ),, ( ), ( t y y v y u I t y I ), ( ), ( ),, ( 1),, ( y v I y u I t y I t y I y Feature rackng I(,y,t 1) I(,y,t) 0 t y I v I u I Hence, B. Lucas and. Kanade. An teratve mage regstraton tecnque wt an applcaton to stereo vson. In Proceedngs of te Internatonal Jont Conference on Artfcal Intellgence, pp , 1981.

16 Spatal Coerence Constrant I u I y v It 0 How many equatons and unknowns per pel? One equaton, two unknowns How to get more equatons for a pel? Spatal coerence constrant: pretend te pel s negbors ave te same (u,v) 16

17 Solvng te rackng Problem Least squares problem: Wen s ts system solvable? Wat f te wndow contans just a sngle stragt edge? B. Lucas and. Kanade. An teratve mage regstraton tecnque wt an applcaton to stereo vson. In Proceedngs of te Internatonal Jont Conference on Artfcal Intellgence, pp , 1981.

18 Condtons for Solvablty Bad case: sngle stragt edge

19 Lucas-Kanade Flow Least squares problem: Soluton gven by e summatons are over all pels n te wndow B. Lucas and. Kanade. An teratve mage regstraton tecnque wt an applcaton to stereo vson. In Proceedngs of te Internatonal Jont Conference on Artfcal Intellgence, pp , 1981.

20 Lucas-Kanade Flow Recall te Harrs corner detector: M = A A s te second moment matr We can fgure out weter te system s solvable by lookng at te egenvalues of te second moment matr e egenvectors and egenvalues of M relate to edge drecton and magntude e egenvector assocated wt te larger egenvalue ponts n te drecton of fastest ntensty cange, and te oter egenvector s ortogonal to t

21 Interpretng te egenvalues Classfcaton of mage ponts usng egenvalues of te second moment matr: 2 Edge 2 >> 1 Corner 1 and 2 are large, 1 ~ 2 1 and 2 are small Flat regon Edge 1 >> 2 1

22 Unform Regon gradents ave small magntude small 1, small 2 system s ll-condtoned

23 Edge gradents ave one domnant drecton large 1, small 2 system s ll-condtoned

24 Hg-teture or Corner Regon gradents ave dfferent drectons, large magntudes large 1, large 2 system s well-condtoned

25 Feature trackng So far, we ave only consdered feature trackng n a par of mages If we ave more tan two mages, we can track feature from eac frame to te net Gven a pont n te frst mage, we can n prncple reconstruct ts pat by smply followng te arrows

26 rackng over Many Frames Select features n frst frame For eac frame: Update postons of tracked features Dscrete searc or Lucas-Kanade (or a combnaton of te two) ermnate nconsstent tracks Compute smlarty wt correspondng feature n te prevous frame or n te frst frame were t s vsble Fnd more features to track

27 S-omas Feature racker Fnd good features usng egenvalues of secondmoment matr Key dea: good features to track are te ones wose moton can be estmated relably From frame to frame, track wt Lucas-Kanade s amounts to assumng a translaton model for frame-to-frame feature movement Ceck consstency of tracks by affne regstraton to te frst observed nstance of te feature Affne model s more accurate for larger dsplacements Comparng to te frst frame elps to mnmze drft J. S and C. omas. Good Features to rack. CVPR 1994.

28 Input rajectory Estmaton Moton Model Estmaton Moton Plan Vdeo ransform Output Frames 2 2D moton model (omograpy)

29 Homograpy y y epand

30 Fttng a omograpy Equaton for omograpy: y y 0 y y y y y 3 equatons, only 2 lnearly ndependent

31 Drect lnear transform H as 8 degrees of freedom (9 parameters, but scale s arbtrary) One matc gves us two lnearly ndependent equatons Four matces needed for a mnmal soluton (null space of 89 matr) More tan four: omogeneous least squares n n n n n n y y 0 A

32 Input rajectory Estmaton Moton Model Estmaton Moton Plan Vdeo ransform Output

33 Moton Plan S t N t t G, were t t j j Image courtesy: Matsusta et al

34 34

35 35

36 36

37 Input rajectory Estmaton Moton Model Estmaton Moton Plan Vdeo ransform Output 3D reconstructon va structure from moton Voodoo Camera racker (ttp://

38 Voodoo Camera racker (ttp://

39 Input rajectory Estmaton Moton Model Estmaton Moton Plan Vdeo ransform Output

40 Input rajectory Estmaton Moton Model Estmaton Moton Plan Vdeo ransform Output Novel vew syntess va mage based renderng

41 Unstructured lumgrap renderng [Bueler et al. 01] 41

42 42 F Lu, M Glecer, H Jn, A Agarwala. Content-preservng warps for 3D vdeo stablzaton, SIGGRAPH 09

43 Input rajectory Estmaton Moton Model Estmaton Moton Plan Vdeo ransform Output Novel vew syntess mage based renderng

44 Input rajectory Estmaton Moton Model Estmaton Moton Plan Vdeo ransform Output Our metod for novel vew syntess One nput frame One output frame

45 45

46 46

47 nput frame and ponts 47

48 nput frame and ponts output ponts 48

49 nput frame and ponts output frame 49

50 50

51 51

52 52

53 53

54 54

55 [Igaras et al. 05] 55

56 Input Vsual salency map [Itt et al. 99] Vsual salency: te dstnct subjectve perceptual qualty wc makes some tems n te world stand out from ter negbors and mmedately grab our attenton from [Itt 07] 56

57 Input Output 57

58 Input Output teture mappng [Srley et al. 2005] 58

59 Grd mes & ponts Output 59

60 Net me More on Vdeo Stablzaton 3D Vdeo Stablzaton Subspace Vdeo Stablzaton 60