Vision for Vision. Learning image analysis from the brain to prevent blindness. Prof. Bart ter Haar Romeny

Size: px

Start display at page:

Download "Vision for Vision. Learning image analysis from the brain to prevent blindness. Prof. Bart ter Haar Romeny"

Alexandrina Jennings
5 years ago
Views:

1 Vision for Vision Learning image analysis from the brain to prevent blindness Prof. Bart ter Haar Romeny Northeastern University - Eindhoven University of Technology

2 toppoints Edge focusing scale graph theory MR slice hartcoronair

3 Structures exist at their own scale: Original σ = e 0 px σ = e 1 px σ = e 2 px σ = e 3 px Noise edges

4 The graph of the sign-change of the first derivative of a signal as a function of scale is denoted the scale-space signature of the signal. Zero-crossings of the second order derivative = max of first order derivative, as a function of scale

5 The notion of longevity can be viewed of a measure of importance for singularities [Witkin83]. The semantical notions of prominence and conspicuity now get a clear meaning in scale-space theory. In a scale-space we see the emergence of the hierarchy of structures. Positive and negative edges come together and annihilate in singularity points.

6 Example: Lysosome segmentation in noisy 2-photon microscopy 3D images of macrophages.

7 Marching-cubes isophote surface of the macrophage. Preprocessing: - Blur with σ = 3 px - Detect N strongest maxima slice 24 slice 23 slice 24 slice 20 slice 18 slice 24 slice 24 slice 21 slice 25 slice 18 slice 22 slice 21

8 We interpolate with cubic splines interpolation 35 radial tracks in 35 3D orientations

9 The profiles are extremely noisy: Observation: visually we can reasonably point the steepest edgepoints.

10 Edge focusing over all profiles. Choose a start level based on the task, i.e. find a single edge.

11 Detected 3D points per maximum. We need a 3D shape fit function.

12 The 3D points are least square fit with 3D spherical harmonics: 1 2, sin, cos, sin, sin2, cos sin, cos 2 1, 15 2 cos sin, 1 4 sin

13 Resulting detection:

14 An efficient way to detect maxima and saddlepoints is found in the theory of vector field analysis (Stoke s theorem)

15 Topological winding numbers N-D p L i 1 dl i2... dl i n i 1 i 2... i n L j p L j p n 2 2-D p L 1 dl 2 L 2 dl 1 L 1 2 L 2 2 is the wedge product (outer product for functionals)

In 2D: the surrounding of the point P is a closed

rotates over 2π when we walk over a closed maximum:

16 In 2D: the surrounding of the point P is a closed path around P. BMIE The winding number ν of a point P is defined as the number of times the image gradient vector rotates over 2π when we walk over a closed path around P. maximum: ν = 1 minumum: ν = 1 regular point: ν = 0 saddle point: ν = -1 monkey saddle: ν = -2

17 Winding number = +1 extremum Winding number = -1 saddle The notion of scale appears in the size of the path.

18 Generalised saddle point (5 th order): (x+i y) 5 Winding number = - 4 monkey saddle The winding numbers add within a closed contour, e.g. A saddle point (-1) and an extremum (+1) cancel, i.e. annihilate. Catastrophe theory

Decrease of structure over scale scales with the dimensionality. Slopefor MR image: 1.66555 Slopefor whitenoise: 1.

20 Decrease of structure over scale scales with the dimensionality. Slopefor MR image: Slopefor whitenoise: The number of extrema and saddlepoints decrease as e -N over scale

21 Application: Computer-Assisted Human Follicle Analysis for Fertility Prospects with 3D Ultrasound Fertility Prospects In most developed countries a postponement of childbearing is seen. E.g. in the Netherlands: Average age of bearing first child is 30 years. ter Haar Romeny et al., IPMI 1999

22 Female reproductive anatomy

23 Ovary Oviduct Uterus wall Endometrium Uterus Uterus neck

24 The number of follicles decreases during lifetime

25 1. As female fecundicity decreases with advancing age, an increasing number of couples is faced with unexpected difficulties in conceiving. Approx couples visit fertility clinics annually In 70% of these cases age-related fecundicity decline may play a role A further increase is expected 2. In our emancipated society a tension between family planning and career exists. Being young, till what age can I safely postpone childbearing? Getting older, at what age am I still likely to be able to conceive spontaneously? A further increase is expected Menopausal age

26 A follicle s life Resting 0.03 mm initiation of growth > 120 days? Early growing mm Preantral mm basal growth ~ 65 days Antral mm Selectable 2-5 mm rescued by FSH window ~ 5 days Selected 5-10 mm Dominance mm maturation ~ 15 days Ovulation

27 3D Ultrasound is a safe, cheap and versatile appropriate modality Kretz Medicor 530D

28 Two 3D acquisition strategies: 1. Position tracker on regular probe 2. Sweep of 2D array in transducer Regular sampling from irregularly space slices Trans-vaginal probe

30 Manual counting is very cumbersome Automated follicle assessment 2-5 mm hypodense structures structured noise vessels may look like follicles ovary boundary sometimes missing

31 Detection of a singularity (i.e. a minimum) From theory of vector fields several important theorems (Stokes, Gauss) exist that relate something happening in a volume with just its surface. We can detect singularities by measurements around the singularity. ξ i ξ i 1-D: difference of signs of the gradient ξ i zero crossing or extremum P The surrounding of the point P are just 2 points left and right of P 1D sphere. ξ ξ i =, x ξ y

32 = ν ξε. dξ W i ij j ij ε = In subscript notation: ij dα= ξε. dξ where ε ij is the antisymmetric tensor. i j dα = ξ dξ ξ dξ ξ + ξ

33 Example of a result: 1 cm Dataset 256 3, radius Stokes sphere 1 pixel, blurring scale 3 pixels

Detection of follicle boundaries: generation of 200-500 rays in a homogeneous orientation

spherical harmonics to get an analytical description of the shape calculate volume and

34 Detection of follicle boundaries: generation of rays in a homogeneous orientation distribution determine most pronounced edge along ray by winding number focusing fit spherical harmonics to get an analytical description of the shape calculate volume and statistics on shape US intensity Scale Scale Distance along ray Distance along ray Distance along ray

35 3D scatterplot of detected endpoints 3D visualisation of fitted spherical harmonics function

Validation with 2 bovine ovaria anatomincal coupes high resolution MR 3D

(pixels) volume from spherical harmonics (mm 3 ) volume from MRI volume

0 29.8 113 55 74 41.6 56.2 54.9 59.3 v01 33 41 66 25.3 242.3 47 22 75 44.

36 Validation with 2 bovine ovaria anatomincal coupes high resolution MR 3D ultrasound Follicle # x center y center z center distance to neighbor (pixels) volume from spherical harmonics (mm 3 ) volume from MRI volume from anatomy v v v

37 Conclusions: 3D ultrasound is a feasible modality for follicle-based fertilitiy state estimation automated CAD is feasible, more clinical validation needed winding numbers are robust (scaled) singularity detectors a robust class of topological properties emerges

38 Multi-scale watershed segmentation Watershed are the boundaries of merging water basins, when the image landscape is immersed by punching the minima. At larger scale the boundaries get blurred, rounded and dislocated.

39 Regions of different scales can be linked by calculating the largest overlap with the region in the scales just above.

40 The method is often combined with nonlinear diffusion schemes E. Dam, ITU

41 Nabla Vision is an interactive 3D watershed segmentation tool developed by the University of Copenhagen. Sculpture the 3D shape by successively clicking precalculated finer scale watershed details.

Requirements for region detection

Region detectors Requirements for region detection For region detection invariance transformations that should be considered are illumination changes, translation, rotation, scale and full affine transform