Variational Optical Flow from Alternate Exposure Images

Transcription

1 Variational Optical Flow from Alternate Exposure Images A. Sellent, M. Eisemann, B. Goldlücke, T. Pock, D. Cremers, M. Magnor TU Braunschweig, University Bonn, TU Graz

2 Two Image Optical Flow Pinpoint sharp images t Motion only from image difference Occlusion unexplained Large motion only via image pyramid 2

3 Motion Blurred Images Motion recorded in the blur t High image frequencies in motion direction are lost Explains occlusion Can deal with large motion 3

4 Alternate Exposure Images Motion recorded in the blur t High image frequencies in motion direction Explain occlusion Can deal with large motion 4

5 Alternate Exposure Images Sellent, Eisemann, Magnor (ICCP 2009): Motion Field and Occlusion Time Estimation via Alternate Exposure Flow Point wise minimization More accurate than High-SpeedCamera Optical Flow [Lim2005] Occlusion timings for frame interpolation 5

6 Improved Approach E θ u,v = ρ v Alternate Exposure Images 1 u v 2 u dx 2θ Dual TV L1 Optimization Improved Motion Fields 6

7 Outline How is motion obtained from alternate exposure images? How is the dual TV-L1 optimization applied? What are the benefits? 7

8 Image Formation Without x occlusion x x 1 I B x = I 1 x p x,t dt 0 1 I B x = I 2 x p x,t dt 0 8

9 Image Formation With occlusion x x x x s 1 I B x = I 1 x p x,t dt I 2 x p x,t dt 0 s 9

10 General Image Formation s 1 s B x I B x = I 1 x tw a dt+ I 2 x+tw b dt 0 Measured 0 Data Unknowns I1 Forward motion wa I2 Backward motion wb B Occlusion time s 10

11 Energy Formulation: Data s ρ 1 w a,wb,s =B I 1 x tw a dt+ 0 1 s I 2 x+tw b dt 0 ρ 2 wa,wb,s =I 1 x swa I 2 x+ 1 s w b E data wa,wb,s = ρ 1 +γ ρ 2 dx 11

12 Energy Formulation: Smoothness 2 E smooth wa,wb,s = α i=1 wa,i wb,i +β s dx 12

13 Minimization Method Zach, Pock, Bischof (DAGM 2007): A duality based approach for realtime TV L1 optical flow E θ wa,ψa,wb,ψb,s,σ =E data ψ a,ψ b,σ E smooth wa,wb,s 1 ψ a wa 2 ψ b wb 2 σ s 2 θ ψb ψa wa wb 13

14 Minimization Method Zach, Pock, Bischof (DAGM 2007): A duality based approach for realtime TV L1 optical flow E θ wa,ψa,wb,ψb,s,σ =E data ψ a,ψ b,σ E smooth wa,wb,s 1 ψ a wa 2 ψ b wb 2 σ s 2 θ ψb ψa wa wb 14

15 Minimization Method Zach, Pock, Bischof (DAGM 2007): A duality based approach for realtime TV L1 optical flow E θ wa,ψa,wb,ψb,s,σ =E data ψ a,ψ b,σ E smooth wa,wb,s 1 ψ a wa 2 ψ b wb 2 σ s 2 θ ψb ψa wa wb 15

16 Minimization Method Zach, Pock, Bischof (DAGM 2007): A duality based approach for realtime TV L1 optical flow E θ wa,ψa,wb,ψb,s,σ =E data ψ a,ψ b,σ E smooth wa,wb,s 1 ψ a wa 2 ψ b wb 2 σ s 2 θ ψb ψa wa wb 16

17 Results: ground truth comparison More accurate motion fields 17

18 Results: ground truth comparison More accurate motion fields Mean Angular Error Ben Windmill Corner Sand, Tellers 8,42 6,78 6,40 Zach et al. 5,81 4,87 5,05 AEF pointwise 6,31 8,64 12,87 AEF TV L1 4,27 4,56 4,57 18

19 Results: real scenes More accurate motion fields Motion in occluded/ disoccluded areas 19

20 Summary Alternate exposure images provide more information than using only one type of images Global TV-L1 optimization results in more accurate motion fields than point-wise optimization Flow calculation more accurate than state-of-the-art Optical Flow algorithms 20

21 Thank you. graphics. tu bs. de The authors gratefully acknowledge funding by the German Science Foundation from project DFG MA2555/