Selected Topics in Biomedical Image Analysis

Size: px

Start display at page:

Download "Selected Topics in Biomedical Image Analysis"

Ami Matthews
5 years ago
Views:

Selected Topics in Biomedical Image Analysis Zeyun Yu Department of Computer Sciences University of Wisconsin-Milwaukee http://www.uwm.

adaptive method Image segmentation Multi-seeded fast marching method Image skeleton extraction Boundary-free approach J. F. OBrien and N.

1 Selected Topics in Biomedical Image Analysis Zeyun Yu Department of Computer Sciences University of Wisconsin-Milwaukee Department of Computer Science University of Wisconsin-Milwaukee Slides 1 Selected Topics Image contrast enhancement A fast and adaptive method Image segmentation Multi-seeded fast marching method Image skeleton extraction Boundary-free approach J. F. OBrien and N. F. Ezquerra, Proc. SPIE Conf. Visualization in Biomed. Computing, 1994 Department of Computer Science University of Wisconsin-Milwaukee Slides 2 1

Contrast Enhancement (1): Examples CT image

image Department of Computer Science University

Enhancement (2): Prior Work Global contrast

equalization (Pizer 87; Caselles 98; Stark 00)

σ Retinex model (Jobson 97) Single scale

2 Contrast Enhancement (1): Examples CT image Mammography image MRI image Cell image Virus image Department of Computer Science University of Wisconsin-Milwaukee Slides 3 Contrast Enhancement (2): Prior Work Global contrast manipulation Linear Nonlinear Histogram equalization (Pizer 87; Caselles 98; Stark 00) Global Local I( x, y) R = log I( x, y)* G ( x, y) σ Retinex model (Jobson 97) Single scale Multi-scale Department of Computer Science University of Wisconsin-Milwaukee Slides 4 2

3 Contrast Enhancement (3): Our Method Motivations Local contrast manipulation Adaptive transfer function Multi-scale contrast enhancement Steps Compute local statistics (min/max/avg) Design transfer function Update the intensity pixel-by-pixel Original image Local minimum Local maximum Contrast enhanced Department of Computer Science University of Wisconsin-Milwaukee Slides 5 Contrast Enhancement (4): Local Statistics Propagation scheme for average (Deriche 90; Young 95) Top-down For each row: (m-1, n) (m, n) 3. Conditional propagation scheme for local minimum/maximum (Yu 04) Bottom-up For each row: C: conductivity [0, 1] 3. Department of Computer Science University of Wisconsin-Milwaukee Slides 6 3

Contrast Enhancement (5): Comparison Advantages Fast to calculate Smooth results Results by local

Computer Science University of Wisconsin-Milwaukee Slides 7 Contrast Enhancement (6): Transform

function lmin Img lavg lmax If I(x,y) > A(x,y), choose convex transfer function 0 ω α = Anew I new

4 Contrast Enhancement (5): Comparison Advantages Fast to calculate Smooth results Results by local propagation: searching: Original image Local minimum Local maximum Contrast enhanced Department of Computer Science University of Wisconsin-Milwaukee Slides 7 Contrast Enhancement (6): Transform Rescale the window size Local contrast manipulation If I(x,y) < A(x,y), choose concave transfer function lmin Img lavg lmax If I(x,y) > A(x,y), choose convex transfer function 0 ω α = Anew I new 128 Y = f (x) 255 ω 0 lw 255 X Department of Computer Science University of Wisconsin-Milwaukee Slides 8 4

Contrast Enhancement (7): Extensions Anisotropic propagation 3D contrast

1] Color contrast enhancement RGB, HSV Department of Computer Science

Original image Histogram equalization Our method (isotropic) Original image

5 Contrast Enhancement (7): Extensions Anisotropic propagation 3D contrast enhancement (m-1, n) (m, n) R: resistance [0.01, 0.1] Color contrast enhancement RGB, HSV Department of Computer Science University of Wisconsin-Milwaukee Slides 9 Contrast Enhancement (8): Results Original image Histogram equalization Our method (isotropic) Original image Histogram equalization Our method (anisotropic) Department of Computer Science University of Wisconsin-Milwaukee Slides 10 5

6 Contrast Enhancement (9): Results Original image Isotropic propagation Anisotropic propagation Original image C = 0.95 C = 0.85 C = 0.75 Department of Computer Science University of Wisconsin-Milwaukee Slides 11 Contrast Enhancement (10): Results 3D example: Hair bundle cellular image (left: original right: enhanced) RGB based Color example: Chinese painting (left: original right: enhanced) Department of Computer Science University of Wisconsin-Milwaukee Slides 12 6

hair bundle (hearing organelle of the hair

Restricted shapes Region merging Graph cut

Parametric (snakes) Geometric (level set

7 Image Segmentation (1): Examples Single object Multiple objects Tip link between hair bundle (hearing organelle of the hair cell) Rice dwarf virus trimer (segmented into three monomers) Map provided by W. Chiu Map provided by M. Auer Department of Computer Science University of Wisconsin-Milwaukee Slides 13 Image Segmentation (2): Prior Work Edge detection Discontinuity Model matching Restricted shapes Region merging Graph cut Clustering Active/deformable contour Parametric (snakes) Geometric (level set method) McInerney & Terzopoulos,1996 Department of Computer Science University of Wisconsin-Milwaukee Slides 14 7

Image Segmentation (3): Fast Marching Method The fast marching

certain speed r r T ( ) F( ) = 1 where F is the speed function,

fast marching method (Bajaj jj & Yu & Auer, 2003; Sifakis &

Topology-changing for the same seed Topology-preserving between

Wisconsin-Milwaukee Slides 15 Image Segmentation (4): Fast

segmentation Automatic seed generation Medical images: source

classification) Post-processing (region merging) Molecular

8 Image Segmentation (3): Fast Marching Method The fast marching method (Sethian, 1996, 1999) Start from a seed and propagate by certain speed r r T ( ) F( ) = 1 where F is the speed function, which can be defined as: r r α I ( ) F( ) = e α > 0 Multi-seeded fast marching method (Bajaj jj & Yu & Auer, 2003; Sifakis & Tziritas, 2001) 1 marching simultaneously from each seed Topology-changing for the same seed Topology-preserving between seeds 2 3 Department of Computer Science University of Wisconsin-Milwaukee Slides 15 Image Segmentation (4): Fast Marching Method Multiple seeds assigned to one object Better segmentation Automatic seed generation Medical images: source points (Yu 02) Two approaches Pre-processing (seed classification) Post-processing (region merging) Molecular images: sink points (Bajaj&Yu 03; Yu 05) Department of Computer Science University of Wisconsin-Milwaukee Slides 16 8

9 Image Segmentation (6): Seed Classification Marching distance MD ( A, B) = Min{ f e A B r f ( ) ds} Multiple seeds assigned to one object along all paths from A to B (Yu & Bajaj, 2005; Bajaj & Yu & Auer, 2003) marching simultaneously from each seed Topology-changing for seeds in the same class Topology-preserving for seeds in different class A B Department of Computer Science University of Wisconsin-Milwaukee Slides 17 Image Segmentation (7): Seed Classification P22 Phi29 Correlation Score: 5-fold: fold: 0.79 Collaborators: Wah Chiu (BCM), Tim Baker (UCSD) Department of Computer Science University of Wisconsin-Milwaukee Slides 18 9

Image Segmentation (8): Seed Classification

Science University of Wisconsin-Milwaukee

Original image 105 seeds/regions 14 regions

adjacent regions Edge weight: (1) intensity

common boundary between two regions Use a

This works in a similar way to the region

10 Image Segmentation (8): Seed Classification CTA segmentation Department of Computer Science University of Wisconsin-Milwaukee Slides 19 Image Segmentation: Region Merging Original image 105 seeds/regions 14 regions after merging Node: each region Edge: two adjacent regions Edge weight: (1) intensity difference (2) gradient magnitude on the common boundary between two regions Use a greedy method to find the minimum cut. This works in a similar way to the region merging approach Department of Computer Science University of Wisconsin-Milwaukee Slides 20 10

11 Image Segmentation: Constrained Region Merging A user-guided, semi-automatic method The user picks some seed points and assigns the types, which will be given the highest priority during the segmentation The regions containing the selected seeds are assigned the corresponding types. Regions containing different types are NOT allowed to merge, no matter how similar they are. All other regions are free to merge, but whenever they are merged to a region with a type, they will be assigned with the same type too. The weighted graph Graph cut allowed Graph cut not allowed Department of Computer Science University of Wisconsin-Milwaukee Slides 21 A Quick Demo of Seed Selection BIMoS Biomedical Image-based Modeling and Simulation (Molecule) (Image) (Mesh) (Simulation) Department of Computer Science University of Wisconsin-Milwaukee Slides 22 11

Image Segmentation: Results Time step #1

pixel: treated as a square (four edges) 3D

quadrilateral surface mesh Department of

12 Image Segmentation: Results Time step #1 Time step #3 Department of Computer Science University of Wisconsin-Milwaukee Slides 23 Geometric Modeling Pixels 2D pixel: treated as a square (four edges) 3D voxel: treated as a cube (six faces) a quadrilateral surface mesh Department of Computer Science University of Wisconsin-Milwaukee Slides 24 12

Mesh Smoothing Least square based quadric surface fitting: 2 2 huv (, ) = au + buv+

13 Quadrilateral Surface Mesh Generation Time step #1 Time step #3 Department of Computer Science University of Wisconsin-Milwaukee Slides 25 Quadrilateral Surface Mesh Smoothing Least square based quadric surface fitting: 2 2 huv (, ) = au + buv+ cv + eu+ fv+ g Department of Computer Science University of Wisconsin-Milwaukee Slides 26 13

14 Mesh Smoothing Results Department of Computer Science University of Wisconsin-Milwaukee Slides 27 The Heart Motion Time step #1 Time step #3 Time step #5 Time step #10 Department of Computer Science University of Wisconsin-Milwaukee Slides 28 14

15 Skeleton Extraction (1): Boundary-Based Boundary-based Methods Topology-preserving thinning (Lam 92) Distance transform (Arcelli 92; Malandain 98) Voronoi diagram (Ogniewicz 95; Amenta 01) Voxel Coding (Zhou & Toga 99) Physically-based simulations (Leymarie 92; Grogorishin 96) Boundary of object Skeletons Department of Computer Science University of Wisconsin-Milwaukee Slides 29 Skeleton Extraction (2): Boundary-Free Intensity-based Methods Contouring the edge-strength functions (Tari 97) Scale-space theory (Lindeberg 98, Pizer 98) Level set method: surface motion (Chung 00) Pseudo-distance map (Jang 01) Morse complexes (Milnor 63) Filaments Skeletons Department of Computer Science University of Wisconsin-Milwaukee Slides 30 15

Skeleton Extraction (3): Our Method Four steps: Initialize gradient vector field Diffuse gradient

given image r is the neighbor with lowest intensity For dark features: I (*) is the given image r is

Slides 31 Skeleton Extraction (4): Our Method Four steps: Initialize gradient vector field Diffuse

u 2 2 2 = μ u ( u f x )( f x + f y ) t v 2 2 2 = μ v ( v f )( + ) y f x f y t Anisotropic Diffusion (Yu

16 Skeleton Extraction (3): Our Method Four steps: Initialize gradient vector field Diffuse gradient vector field Compute skeleton strength map Skeleton eeo Tracing r r For bright features: I (*) is the given image r is the neighbor with lowest intensity For dark features: I (*) is the given image r is the neighbor with highest intensity Department of Computer Science University of Wisconsin-Milwaukee Slides 31 Skeleton Extraction (4): Our Method Four steps: Initialize gradient vector field Diffuse gradient vector field Compute skeleton strength map Skeleton eeo Tracing r Isotropic Diffusion (Xu 98) u = μ u ( u f x )( f x + f y ) t v = μ v ( v f )( + ) y f x f y t Anisotropic Diffusion (Yu 04) u = μ ( g ( α ) u) t v = μ ( g ( α ) v) t Department of Computer Science University of Wisconsin-Milwaukee Slides 32 16

Skeleton Extraction (5): Our Method Four steps: Initialize gradient vector field

bright features: Conventional gradients Isotropic diffusion For dark features:

Slides 33 Skeleton Extraction (6): Our Method Four steps: Initialize gradient vector

Two tracing approaches: Canny s method Pixel-based Discontinuity Isotropic diffusion:

y G I xi z G σ σ I I G x 2 I G y σ σ I I G y y z σ I I G x 2 I G σ σ I I G z y z z σ

17 Skeleton Extraction (5): Our Method Four steps: Initialize gradient vector field Diffuse gradient vector field Compute skeleton strength map Skeleton eeo Tracing r For bright features: Conventional gradients Isotropic diffusion For dark features: Anisotropic diffusion Department of Computer Science University of Wisconsin-Milwaukee Slides 33 Skeleton Extraction (6): Our Method Four steps: Initialize gradient vector field Diffuse gradient vector field Compute skeleton strength map Skeleton eeo Tracing Two tracing approaches: Canny s method Pixel-based Discontinuity Isotropic diffusion: Canny s method Local structure tensor Path-tracing Continuity guaranteed 2 I x Gσ I xi y G I xi z G σ σ I I G x 2 I G y σ σ I I G y y z σ I I G x 2 I G σ σ I I G z y z z σ Anisotropic diffusion: Canny s method Department of Computer Science University of Wisconsin-Milwaukee Slides 34 17

18 Skeleton Extraction (7): Results Original image Skeleton strength map Skeletons Skeletons on map Original image Edge strength map Skeleton strength map Skeletons on map Department of Computer Science University of Wisconsin-Milwaukee Slides 35 Skeleton Extraction (8): Results Department of Computer Science University of Wisconsin-Milwaukee Slides 36 18

19 Skeleton Extraction (9): Bio-Application 1BBH 1DXT Department of Computer Science University of Wisconsin-Milwaukee Slides 37 Questions? BIMoS (Will be available later this year) Department of Computer Science University of Wisconsin-Milwaukee Slides 38 19

A Segmentation-Free Approach for Skeletonization of Gray-Scale Images via Anisotropic Vector Diffusion

A Segmentation-Free Approach for Skeletonization of Gray-Scale Images via Anisotropic Vector Diffusion Zeyun Yu and Chandrajit Bajaj Department of Computer Science, University of Texas at Austin, Austin,