smooth coefficients H. Köstler, U. Rüde

Size: px

Start display at page:

Download "smooth coefficients H. Köstler, U. Rüde"

Arlene Strickland
5 years ago
Views:

1 A robust multigrid solver for the optical flow problem with non- smooth coefficients H. Köstler, U. Rüde

2 Overview Optical Flow Problem Data term and various regularizers A Robust Multigrid Solver Galerkin coarsening Line/Block Smoothers Matrix-dependent Transfer Operators Iterant Recombination Experimental Results Parallelization Results 2

3 Goal Develop a general framework for the solution of systems of PDEs in regular domains resulting from variational formulations in image processing like Optical Flow or Image Registration Use (non)-linear multigrid solver 2D-4D Provide easy to use interface Focus on hard to solve cases including singularities and nearly singular problems 3

4 Parallel C++ Code The general scheme Implementation read frames image preprocessing solver setup solve visualize flow ffmpeg, normalization, set Operator, MG, FMG, plot vector raw formats smoothing, RHS, multilevel, field, compute derivatives Coarsening time dependent plot norm, (non)-linear LIC 4

5 Optical Flow Problem An approximation of the motion Optical Flow Motion The optical flow at the pixel (x,y) is the 2D-velocity vector 5

6 Definitions Dimension d, Image domain Ω R d Time T = [0,... t max ] Image intensity function Displacement vector field (motion field) Spatial and temporal derivatives of an image 6

7 Optical Flow Data Term Assumption: Objects keep the same intensity over time Taylor Expansion gives in 2D The optical flow constraint equation (OFCE) 7

8 Variational Problem Formulation Non-parametric approach: We try to minimize the functional Data term Regularizer Smoothing parameter 8

9 Regularization I Regularize problem to get a unique solution Diffusion based: Elastic: Fluid: Curvature based: 9

10 Regularization II Include temporal, spatial or flow information in regularizer For example for diffusion based regularizer: Spatio-temporal: smoothes also in time in video sequence Flow-driven: distinguishes between image edges and real edges 10

11 Regularization III Image-driven (isotropic): smoothes little in the vicinity of image edges so that motion boundaries are not blurred Image-driven (anisotropic): smoothes along edges but not perpendicular to them 11

12 Euler Lagrange Equations I Solution of the minimization problem is a system of PDE's (here for diffusion regularizer and optical flow data term): 12

13 Euler Lagrange Equations II Using Finite Differences for discretization we have to solve the linear system Matrix A h corresponds to a standard discretization for the Laplacian with 2x2-matrices as entries, e.g. on the diagonal The solution u h and the RHS f h are 2D vectors 13

14 Difficulties for the solver When α 0 the matrix gets singular For highly textured images or small moving objects the intensity function I is not smooth, thus problems when doing the Taylor expansion for the Data Term and computing the image derivatives I x, I y, I t (presmooth images?) Problems with fast motions, big time steps How about including information from landmarks or other features? The situation gets worse for more advanced regularizers (e.g. anisotropic, nonlinear) Apply several advanced multigrid techniques 14

15 Parallel Multigrid Scheme I Multigrid components Vertex-centered grid with ghost layer and standard or semicoarsening (Coupled) red-black/multi-color point Gauss-Seidel or Jacobi smoother Full weighting and bilinear interpolation or matrix dependent transfer operators Galerkin coarse grid approximation (GCA approach) or direct discretization Iterant recombination Dirichlet or Neumann boundary conditions FMG, V-, W- cycles FAS, Newton-MG 15

16 Parallel Multigrid Scheme Solve A h u h = f h using a hierarchy of grids Relax on Compute Correct Trade ghost cells Restrict Trade ghost cells Interpolate Solve by recursion 16

17 Choosing the coarse grid operator: Galerkin coarsening Standard coarsening leads to bad convergence rates or even divergence, so the coarse grid operator is chosen then as For our system, we get Galerkin approach leads to high memory cost! 17

18 Lumping, Direct Coarsening Lumping to reduce the memory cost = reduce the size of the stencils by summing up the off-center entries and add it to the center value Direct coarsening = simply restrict image derivatives to coarse grid and use 5/7 point stencil 18

19 Block Gauss-Seidel MG: Direct coarsening MG: Lumping MG: Galerkin Memory cost 19

20 Small regularization parameters - Moving point - V(2,2)-cycle - Jacobi smoother 20

21 Results for Different Smoothers - Moving point - α = 1 - V(2,2)-cycle 21

22 Matrix-dependent Transfer Operators I Split into coarse and fine-without-coarse variables Schur complement 22

23 Matrix-dependent Transfer Operators II Idea: perform approximately a block Gaussian elimination of the operator A Replace (A FF A FC ) by (A FF A FC ) such that A FF is easy to invert -A FF -1 A FC is an approximation of -A FF -1 A FC the prolongation is a local operator 23

24 Matrix-dependent Transfer Operators III Split e.g. in 2D Ω hg = Ω h FC Ω h CF approximate inverse by diagonal or upper triangular matrix 24

25 Results for different Transfer Operators - Moving point - α = 1 - V(2,2)-cycle - ILU-smoother 25

26 Iterant Recombination Similar to Preconditioning Consider linear combination of the latest approximations Determine coefficients by 26

27 Results for Iterant Recombination - Moving point - α = 1 - V(2,2)-cycle - Jacobi smoother 27

28 Results for combined methods - Moving point,α = 1,V(2,2)-cycle Matrix. Dep. Transfer op. Matrix. Dep. Transfer op. and It. Rec. 28

29 System of PDEs A Simple Illustration 29

30 Office sequence I Ground trouth motion field computed by raytracer and original image 30

31 Office sequence II Ground trouth motion field computed by raytracer and original image 31

32 Error metrics Computed Flow field Angular error 32

33 Experimental Results I The Hamburg taxi sequence (256x190) The flow field for diffusion based regularizer with α =

34 Experimental Results II Diffusion based regularizer, α = 0.01 Diffusion based regularizer, α =

35 Experimental Results III Diffusion based regularizer, α = 0.01 Diffusion based regularizer, α =

36 Experimental Results IV Higher order derivative of I Gaussian derivative of I, σ = 1, 9 points 36

37 Experimental Results V Reconstruction of video sequence by optical flow (P. Münch) 37

38 Experimental Results VI Diffusion based regularizer (I.Christadler) 38

39 Experimental Results VII Image-driven isotropic diffusion based regularizer (I.Christadler) 39

40 Experimental Results VIII Flow-driven regularizer (I.Christadler) 40

41 Including landmarks I Using landmark information (trails) from feature tracking software lgf3 (LS Inf , G. Greiner, H. Niemann, Ch. Vogelgsang, I. Scholz) Set for a given u i = (u fix, v fix ) 41

42 Including landmarks II Without Landmarks With Landmarks Data: SFB 603/A7,Christoph Winter,Volker Daum, Fraunhofer Institut, Thread 42

43 Particle image velocimetry I 43

44 Particle image velocimetry II 44

45 PIV: Line integral convolution 45

46 Parallelization Results I Multigrid V(2,2)-cycle, Galerkin coarsening using MPI on SGI Altix 3700 EPIC (Itanium2) 1.3 GHz 28 CPUs 112 GByte RAM 46

47 HPC-Cluster Cluster The cluster consists of 9 dual-, 8 quad-nodes and a fileserver CPU: AMD Opteron, 2.2 GHz, 1 MB L2 cache RAM: DDR 333, 4 GB for dual-, 16 GB for the quad-nodes All nodes are connected with a GBit-interface, the quad-nodes additionally with Infiniband. Latter provides a bandwidth of up to 10 GBit/s. 47

48 Parallelization Results II Times for one multigrid V(2,2)-cycle Scali-MPI Standard coarsening 48

49 Application to medical data I MRI (magnetic resonance imaging) scan images preoperative intraoperative 49

50 Application to medical data II 50

51 Conclusions & Outlook GUI for user interaction Inspect influence of landmarks (small island problem) on multigrid solver Mathematical analysis of optical flow problem (P. Kornprobst) Current projects: video compression (diploma thesis), 3D LIC for PIV, 4D medical image registration (M. Prümmer), motion blur (project work) Use different (statistical) data terms Combine different regularizers? 51

Contents. I The Basic Framework for Stationary Problems 1

Contents. I The Basic Framework for Stationary Problems 1 page v Preface xiii I The Basic Framework for Stationary Problems 1 1 Some model PDEs 3 1.1 Laplace s equation; elliptic BVPs... 3 1.1.1 Physical experiments modeled by Laplace s equation... 5 1.2 Other