Image Segmentation II Advanced Approaches

Size: px

Start display at page:

Download "Image Segmentation II Advanced Approaches"

MargaretMargaret Hunter
5 years ago
Views:

1 Image Segmentation II Advanced Approaches Jorge Jara W. 1,2 1 Department of Computer Science DCC, U. of Chile 2 SCIAN-Lab, BNI

2 Outline 1. Segmentation I Digital image processing Segmentation basics 2. Segmentation II Advanced techniques

3 Segmentation basics Some times more information is needed in order to achieve a good segmentation

4 Designed to optimize a cost or fitness function Interest features detection (e.g. edges, points) SIFT Template matching Hough transform Pixel clustering Statistics, probabilities Graph cuts Machine learning (support vector machine, neural networks ) Differential equations, calculus of variations Contour properties Region properties Segmentation - Advanced This can be a VERY long list

Segmentation - Advanced Template matching Classic model Hough transform Applies to circles, line segments and a variety of shapes If we can detect edges we can approximate a

5 Segmentation - Advanced Template matching Classic model Hough transform Applies to circles, line segments and a variety of shapes If we can detect edges we can approximate a circle (center, radius) Hough P (1959) n connected edges and m (=n or <> n) circles A test is performed to determine the circles with best fit Aguilar P, Hitschfeld N (DCC, 2010)

Segmentation - Advanced Variational methods Based on energy minimization, defining integral models Idea: to include desirable features on segmented images (like homogeneus regions, short or smooth

6 Segmentation - Advanced Variational methods Based on energy minimization, defining integral models Idea: to include desirable features on segmented images (like homogeneus regions, short or smooth ROI boundaries) Optimum solutions found by partial differential equations Examples: Mumford-Shah, Ambrosio-Tortorelli, Chan-Vese (details in the book from Aubert & Kornprobst 2006) image I main discontinuties in I ROI boundaries B piecewise smooth image J The Mumford & Shah functional (1989)

rigidity (bending, cornering) force field - repulsion - attraction output: force

7 Segmentation - Advanced Active contour models Optimization of different properties input image +initial guess contour C(s) - elasticity (contraction) - rigidity (bending, cornering) force field - repulsion - attraction output: force balance minimal energy First active contours approach: Kass, Witkin & Terzopoulos (1988) Snakes

8 Segmentation - Advanced 2D active contours or snakes Snakes Contour model [2D discrete point curve] Image [2D: I = I(x, y)] Resolution control Interpolation Approximation Adjusted contour Description Shape, topology

9 Segmentation - Advanced 1 contour function 1 ROI (this is called parametric approach) 2D parametric curve C = C(s) = [x(s), y(s)] s є [0, 1] (arbitrary length) 2D discretization C = {[x i, y i ]; i = 0..n}

Segmentation - Advanced Snakes: optimization derived from a variational approach Minimization of an integral functional a snake minimizes its energy 1 2 1 2 2 C( s) C( s) E Eext [ C( s)] ds 2 2 s s 0

10 Segmentation - Advanced Snakes: optimization derived from a variational approach Minimization of an integral functional a snake minimizes its energy C( s) C( s) E Eext [ C( s)] ds 2 2 s s 0 Elasticity term (coefficient α) Internal energy, contour dependant Rigidity term (coefficient β) External energy, image dependant Kass et al (1988) Snakes: active contour models Int. J. of Computer Vision 1(4):

11 Segmentation \ Active contour models Elasticity - α

12 Segmentation \ Active contour models Rigidity - β

$\ Active$ contour

13 Segmentation \ Active contour models Resolution (f)

14 Intensity gradient vectors V 0 =[I x, I y ] 14

$Segmentation \ Active$ fields can be constructed

15 Segmentation \ Active contour models Attraction fields can be constructed from the intensity gradients Edgemap image Image intensity Or a gaussian-smoothed version

$Segmentation \ Active contour models$ active contour models) Gradient vector

16 Segmentation \ Active contour models Image force fields (external forces in active contour models) Gradient vector flow (GVF) Generalized GVF (GGVF) Xu & Prince (1998) Active contours and gradient vector flow Xu & Prince (1998) Generalized gradient vector flow external forces for active contours

$image force field Segmentation \$ Active contour models Edge

17 Defined by an energy functional to be minimized ), ( ), ( ], [ ), ( y x v v y x u u v u y x V Let GVF image force field Segmentation \ Active contour models Edge similarity term Diffusion (global smoothness) term

$Segmentation \$ models The GVF

18 Segmentation \ Active contour models The GVF (GGVF) energy minimization Diffusion term Edge term g h 18

19 Campo de gradientes GVF V 0 =[I g( I ) x, I y ] = μ μ = 0.05 > 0 h( I ) = I 2 19

20 Generalized GVF (GGVF) g( I ) = e (- I /μ) μ = 0.05 >0 h( I ) = 1 - g( I ) 20

21 Segmentation - Advanced Snakes formulation Energy minimization condition: Euler-Lagrange equation 2 C( s) 2 s 4 C( s) 4 s E ext [ C( s)] 0 Internal forces External force

22 Snakes numerical solution Dynamic formulation C(s) C(s, t) Time equilibrium leads to solution 0 ), ( t t s C y t x )], ( [ ), ( ), ( ), ( t s C E s t s C s t s C t t s C ext Internal forces External force Time evolution Segmentation - Advanced

23 Segmentation - Advanced Snakes numerical solution Parameters α (elasticity), β (rigidity) 2 C 2 s 4 C E [ C] 0 4 ext s An * nc Eext[ C] 0 κ (added as the external force weight), γ (contour viscosity) C t AC t E ext ( Ct 1) ( Ct Ct 1) Iterative scheme, solved computationally A is a 5-diagonal matrix, suitable for numeric solution C( s, t) t 1 ( A) ( Ct 1 E ext ( Ct 1)) In order to computationallly solve this system: matrix inversion, array data structures, adaptive algorithms, 0

24 Segmentation - Advanced α = 0 = 0.01 = 3 = 0.01 = 0.05 Iterations = 5 f = 0.15

25 Segmentation - Advanced

26 Segmentation - Advanced Some problems with the snakes method The quantity of contours must not change, given the 1-1 correspondence of the model Dependecy on the initial guess Local optima for the functional minimization A bad initialization of the snakes can lead to unstability and undesired results Control of the snake parameters is crucial

27 α = 0.01 = 0.5 = 0.6 = 5 = 0.05 Iterations = 5 f = 0.15 Bad parameter selection.

28 A bad 3D case (the gold standard segmentation is the green sphere)

29 Segmentation - Advanced Alternatives Arbitrary contour initialization automation Inference (machine learning approaches)

30 Segmentation - Advanced Topology adaptive snakes (or surfaces) McInerney & Terzopoulos, 1995 (2D), 2002 (3D) Topological changes handling: splitting, merging, collapse

31 Between deformations, the contours are re-parametrized by using a grid to detect topology changes

32 This is an algorithmic approach, additional to the mathematical optimization model. The computation becomes more intensive

33 Segmentation - Advanced Hardest cases: initialization by manual ROI sketching Raw Treatment image

34 Boundary model construction 2D Freeman chain code Segmentation - Advanced 3D mesh models (voxel based): marching cubes (surface meshes), tethraedra (volume meshes) 2D polygon chain code Voxel ( 3D pixel ) model 3D polygon mesh

35 Segmentation - Advanced A typical 3D surface mesh model is formed by: Nodes or vertices Polygons Other models Polynomial surfaces (splines, Bézier, NURBS, ) Primitives composition

36 Segmentation - Advanced Surface mesh model construction Many approaches for construction Many approaches for modeling Voxel ( 3D pixel ) model Triangle mesh Triangular and quarilateral mesh

37 Sample 3D ROI

38 References David Marr. Vision MIT press, 1982 John Russ. The image processing handbook, 4 th ed. CRC Press, 2002 Nixon, Aguado. Feature extraction & image processing, 1 st 2 nd 3 rd ed. Academic Press, Aubert & Kornprobst. Mathematical Problems In Image Processing Springer, 2006 From SCIAN-Lab (see publications section on the website) Jara, D/3D active contours Olmos, D topology adaptive active contours (t-snakes)

39 References Some free and open source software tools Java based (Java runtime requried) ImageJ ( public domain) Fiji ( GPL license) Icy ( GPLv3 license) Others CellProfiler ( GPL, BSD licenses) Slicer ( BSD license) IPOL (Image Processing Online): open access electronic journal with peer reviewed articles + code (in C language) + examples ( BSD / GPL / LGPL licenses or similar)

O4 Digital Imaging 1

O4 Digital Imaging 1 Jorge Jara 1,2, Mauricio Cerda 2 1 Department of Computer Science DCC, U. of Chile 2 SCIAN-Lab, BNI Outline 1. O4a - Introduction Digital imaging Segmentation 2. O4b - Descriptors