How to fit a surface to a point cloud?

Size: px

Start display at page:

Download "How to fit a surface to a point cloud?"

Alexandrina Grant
6 years ago
Views:

1 TRICS seminar Comuter Science Deartment How to fit a surface to a oint cloud? or otimization of surface functionals in comuter vision Yuri Boykov

2 Otimization of surface functionals in comuter vision The University of Comuter vs. human vision Model fitting in comuter vision temlates, ictorial structures, trees, deformable models, contours/snakes, meshes, surfaces, comlexes, grahs, weak-membrane model, Mumford-Shah, Potts model, Otimization in comuter vision dynamic rogramming, gradient descent, PDEs, shortest aths, min. sanning trees, linear and quadratic rogramming, rimal-dual schema, network flow algorithms, QPBO,... Alications segmentation, stereo, multi-view reconstruction, otical flows surface fitting

3 Contours

4 +Shading 3D shae understanding M.C. Escher Drawing hands

5 +Color Recognition Da Vinci Madonna Litta

6 +Texture recognizing material Magritte Souvenir de Voyage

7 +Texture recognizing 3D ersective The New Yorker Album of Drawings, The Viking Press, NY, 1975

8 What do humans get by looking? basic image cues: Contours Shading Color Texture J. Vermeer The Guitar Player

9 What do humans get by looking? basic image cues: Contours Shading Color Texture higher-level ercetion: Segmentation Motion 3D shae ercetion 3D scene geometry Detection/Recognition

10 What do comuters get by looking? x I(x,y) x y y y 3D lot of image intensity I(x,y) x

11 What do comuters get by looking? Intensity discontinuities (contours) Intensity gradients (shading) Multi-valued intensities (color) Filtering (e.g. texture) basic image cues: P. Picasso The Guitar Player higher-level grouing?

12 Bayesian aroach Fit some rior model into data Prior + Data high-level knowledge (global icture) Low-level cues (local info)

13 Rigid Temlate Matching image image Face temlate translation, rotation, scaling In matching we estimate osition of a rigid temlate in the image Position includes global location arameters of a rigid temlate: - translation, rotation, scale,

14 Non-rigid (arametric) matching anorama mosaicing 1. Pick one image (red) 2. War the other images to match it (homograhic transform) 3. Blend

15 e.g. using homograhies

16 e.g. using flexible temlates In flexible temlate matching we estimate osition of each rigid comonent of a temlate For tree-structured models, efficient global otimization is ossible via DP (Felzenswalb&Huttenlocher 2002)

17 tracking arameters => activity recognition Bottom-u tracker

18 Kass, Witkin, Terzooulos 1987 deformable contours ( snakes ) 2D curve which matches to image data Initialized near target, iteratively refined Can restore missing data initial intermediate final Otimization gets harder when a loo is introduced. DP does not aly. One solution: gradient descent 5-18

19 local minima, fixed contour toology Cremers, Tischhäuser, Weickert, Schnörr, Diffusion Snakes, IJCV 6-19 '02

20 Osher&Sethian 1989 Imlicit reresentation of contours Level set function u(x,y) is normally discretized/stored over image ixels Values of u() can be interreted as distances or heights of image ixels u u( x, y ) A contour may be aroximated from u(x,y) with near subixel accuracy C

21 [Visualization is courtesy of O. Juan] Simle evolution dc N du u The University of Morhological Oeration: Erosion

22 Visualization is courtesy of O. Juan Examle of gradient descent evolution The University of Gradient descent w.r.t. Euclidean length u ( dt) 2 u 2 x 2 u 2 y 6-22

23 Examle of gradient descent evolution Gradient descent w.r.t. Euclidean length u ( dt) 2 u 2 x 2 u 2 y Lalacian Osher&Sethian

24 [examle from Goldenberg, Kimmel, Rivlin, Rudzsky, IEEE TIP 01] Geodesic Active Contours via Level-sets E ( C) C g( ) ds u ( dt) ( g xu) x ( g y u) y 6-24

25 Other geometric energy functionals besides length [courtesy of Ron Kimmel] The University of Geometric measures commonly used in segmentation weighted length Functional E( C ) E ( C) C g( ) ds gradient descent evolution dc N ~ g g, N weighted area E ( C) f da ~ f alignment (flux) E ( C) C v, N ds ~ div( v ) 6-25

otimization - gradient descent Tyical roblems: - local minima

26 in 3D deformable meshes, level-sets, The University of Estimation of osition for mesh oints Many loos. otimization - gradient descent Tyical roblems: - local minima (clutter, outliers) -over-smoothing GOALS: global otima (?) right functional (?)

27 Global Otimization and Surface Functionals

28 More generally... Estimate labels for grah nodes The University of observed noisy image I NOTE: similar to robust regression model estimation L I along one scan line in the image image labeling L (restored intensities)

29 (simle examle) Piece-wise smooth restoration The University of Markov Random Fields (MRF) aroach (Continuous analogue: Mamford-Shah functional, 1989) weak membrane model (Geman&Geman 84, Blake&Zisserman 83,87) E( L) ( L I ) 2 (, q) V( L N, L q ) L Lq T T discontinuity reserving rior otimizing E(L) is NP hard!

30 (simle examle) Piece-wise constant restoration The University of observed noisy image I L I image labeling L (restored intensities) along one scan line in the image

31 (simle examle) Piece-wise constant restoration The University of Potts model Boykov Veksler Zabih 01 Greig et al. 89 (for 2 labels) E( L) i ( L I ) i 2 i (, q) N ( L L q ) 2 { : L 2} 1 { : L 1} 0 L 0 Lq { : L 0 } ercetual grouing global otimization is still NP hard, but there are fast rovably good combinatorial aroximation algorithms, linear and quadratic rogramming, QPBO, rimal-dual schema

32 Percetual grouing from stereo constant label = lane (Birchfield &Tomasi 99)

33 Binary labeling L (binary image restoration) E( L) {0,1} Greig Porteous Seheult 89 D ( L ) (, q) N ( L The University of L q ) D ( L ) ( L I ) 2 Pr( I L ) original binary image I otimal binary labeling L Globally otimal solution is ossible using combinatorial grah cut algorithms seudo-boolean otimization Hammer 65, Picard&Ratlif 75

34 Binary labeling (object extraction) L {0,1} The University of object segmentation L 0 L 1 left ventricle of heart

35 Binary labeling (object extraction) L {0,1} Boykov&Jolly 01 E( L) D ( L ) (, q) w N q ( L L q ) surface extraction 0 C 1 left ventricle of heart Globally otimal solution is ossible using grah cut algorithms seudo-boolean otimization (Hammer 65, Picard&Ratlif 75)

36 Imlicit surface reresentation via grah-cuts Any contour (or surface in 3D) satisfying labeling of exterior/interior oints (ixel centers) is accetable if some exlicit surface has to be outut

37 Global otimization of geometric surface functionals The University of E( L ) D ( L ) V( L,Lq ) (,q ) N E( C) C g( ) ds C edge alignment N, v x ds f int( C) ( x) d Geometric length any convex, symmetric metric (e.g. Riemannian) Flux any vector field v Regional bias any scalar function f Tight characterization for geometric functionals of contour C that can be globally otimized by grah cut algorithms (Kolmogorov&Boykov 05) disclaimer: for airwise interactions only

38 Globally otimal surfaces in 3D Volumetric segmentation (BJ01,BK 03,KB 05)

39 Binary labeling (object extraction) L {0,1} The University of iteratively re-estimate color models e.g. using mixture of Gaussians Blake et al. 04, Rother et al. 04

40 Segmentation for Image Blending

41 Segmentation for Image Blending

42 Otimal surfaces in 3D 3D reconstruction Vogiatzis, Torr, Ciola 05 Local cues: voxel s hotoconsistency Prior: smoothness, rojective geometry constraints

43 Globally otimal surfaces in 3D from a chea digital camera Lemitsky&Boykov, 2006

44 Globally otimal surfaces in 3D 3D model (texture maed) multi-view reconstruction set u Furukawa&Ponce 2006

45 Surface fitting to oint cloud a cloud of 3D oints (e.g. from a laser scanner) surface fitting: 3D model:

46 Surface fitting to oint cloud

47 Surface fitting functional Fitting a surface into a cloud of oriented oints (Lemitsky&Boykov, 2007) E( C) N, v ds 1 ds C C data fit rior n i i S

48 Otimal surfaces in 3D Fitting a surface into a cloud of oriented oints (Lemitsky&Boykov, 2007) E( C) N, v ds 1 ds C C From 10 views No initialization is needed

49 Global vs. local otimization Fitting a surface into a cloud of oriented oints (Lemitsky&Boykov, 2007) E( C) div v d 1 ds int( C) C regional otentials f div(v) initial solution local minima global minima

50 Fitting to sarse data

51 Fitting to sarse data

52 Fitting to sarse data

53 Summary Global otimization -Your solution is as good as your functional -No need to worry about initial guess or convergence issues -Polynomial algorithms, but many ractical issues (efficient data structures, memory limitations, arallelization, dynamic alications, ) - Many useful functionals are NP hard (lots of aroximation methods are develoed) - New aroaches allowing global otimization are introduced (including new version of level-sets)

Discrete Optimization Methods in Computer Vision CSE 6389 Slides by: Boykov Modified and Presented by: Mostafa Parchami Basic overview of graph cuts

Discrete Optimization Methods in Computer Vision CSE 6389 Slides by: Boykov Modified and Presented by: Mostafa Parchami Basic overview of graph cuts [Yuri Boykov, Olga Veksler, Ramin Zabih, Fast Approximation