C2: Medical Image Processing Linwei Wang

Transcription

1 C2: Medical Image Processing Linwei Wang

2 Content Enhancement Improve visual quality of the image When the image is too dark, too light, or has low contrast Highlight certain features of the image Restoration Reduce blur or noise in a degraded image Morphological image processing

3 Image Enhancement - Content Enhancement Spatial-domain image enhancement Transformations Single-point transformations Multi-frame transformations Filtering Smoothing filters Sharpening filters Nonlinear filtering Frequency-domain image enhancement Smoothing filters Sharpening filters

4 Restoration Image Restoration - Content Noise models Noise-only spatial filters Inverse Filter Wiener Filter (minimum mean square error filtering) Constrained least square filtering Geometric mean filter Total-variation image de-noising (*)

5 Morphological Processing- Content Morphological processing Dilation & erosion Opening & closing Basic morphological algorithms Boundary extraction Region filling Thinning Thickening Skeletons Pruning

6 Medical Images As Matrices

7 Medical Image Enhancement Spatial Domain

8 Single-Point Transformation One pixel in the input à one pixel in the output Only improve visual content, does not add information content No new information is obtained; might lose information if multiple pixels produces the same output Objective information is of secondary importance; subjective information is key to present to physicians & improve their understanding

9 Single-Point Transformation Map input gray levels to output gray levels through a look-up table (LUT) Window Scaling Selecting only necessary information for analyzing Visually or algorithmically simplify the task Fast speed The transformation function can be defined graphically by the user (by looking-up tables)

10 Basic Single-Point Transformations Linear Negative Logarithmic Log Inverse-log Nega%ve Nth root Log Iden%ty Power-law N-th power Nth power Inverse- log N-th root r: input gray level s: output gray level s =!(r)

11 Image Negatives Suited for enhancing white or gray detail embedded in dark regions of an image, especially when the black areas are dominant in size s = L!1! r s = L!1! r Mammogram with lesion nega%ve DIP Fig 3.4, Courtesy of GE medical system

12 Log Transformation s = clog(1+ r), r! 0 Expand the values of dark pixels & compress the higher-level values Maps a narrow range of low gray-level values into a wider range of output level Maps a wide range of high gray-level values into a narrower range of output level Compresses the dynamic range of images with large variations in pixel values Log Inverse- log

13 Power-Law Transformation Gamma-correction s = cr!, c,! > 0 Contrast manipulation Υ <1: ~ log: Expand the dark; images are brighter; lower contrast Υ >1: ~inverse log: Compress the dark; images are darker; higher contrast

14 Example original r=0.6 r=0.4 r=0.3 Reduced contrast washed- out

15 Example original r=0.6 r=0.4

16 Piecewise-Linear Transformation Contrast stretching To increase the dynamic range of the gray levels Can be arbitrarily complex; consider more user inputs

17 Image Statistics Global image statistics is useful for designing an image-specific LUT for the entire image Local image statistics is useful for adaptive contrast enhancement, with varying LUT across image Neighborhood statistics is useful for noise processing and filtering, not contrast enhancement Global Local Neighborhood

18 Single-Point Transformation Histogram Equalization Histogram equalization A standard technique for guiding single-point transformation Histograms provide a wealth of information about the image Dark Low contrast Bright Well readable

19 Histogram of an image f of size m by n: p(f k ) = n k /(mn) Let f be a random variable in [0, L-1]: p(f k ): probability density function (pdf) k Cumulative density function (cdf): monotonic function! j=0 Goal: to achieve uniform distribution of the pdf Let g be the output image after histogram equalization is applied on f: g = T(f) p(g) = p( f ) df Single-Point Transformation: Histogram Equalization dg = p( f ) dg df n j mn g = CDF( f ) = f # p(w)dw $ dg = p( f ) $ p(g) =1!" df

20 Single-Point Transformation: Histogram Equalization Discrete cases: images pdf: cdf: p( f k ) = p( f = k) = n k MN, CDF( f k ) =! p( f i ) = Output image g: Input image with arbitrary pdf -> Output image with uniform pdf Stretching the histogram for better contrast Straightforward; invertible Not computational expensive Indiscriminate k i=0 k! i=0 0! k < L n i MN g k = CDF( f k ) = k! j=0 n j MN

21 Example Single-Point Transformation Histogram Equalization 8 by 8 small image Frequency (histogram) CDF T(i) = round( CDF(i)!CDF min mn! CDF min " (L!1))

22 Single-Point Transformation Histogram Equalization Adaptive histogram equalization Compute several histograms for local sections of the image Regional contrast is increased to its maximum. Can produce significant noise, especially in homogeneous regions Contrast limited adaptive histogram equalization (CLAHE) Limit the regional contrast to a desired level Limit the maximum number of counts for each grayscale to a clip limit ; the extra counts beyond this limit are uniformly distributed among grayscale with counts less than the clip limit

23 Single-Point Transformation Histogram Equalization Early detection of ischemia in brain injury A: Directly after an injury B: After histogram equalization: Early diagnosis of ischemia C: Confirmed by CT scan 4 days later: treatment would have been too late

24 Multi-Image Transformation Single pixels from several input images à single pixel in one output image

25 Subtraction Multi-Image Transformation: Subtraction Angiography Why subtraction in addition to contrast media? Digital subtraction angiography (DSA) Both images show exactly the same area of the body Produces image with better contrast and quality; requires much less contrast medium. Can be applied to a sequence of images M j = I j!1 + I j!2 +!I j!n f j = I j! M j Information with temporal frequencies below 1/(n t) are removed, including stationary objects (0 Hz) or slow movements caused by breathing (low frequency)

26 Image Filtering Filtering: contextual/neighborhood operation (input several pixels in a neighborhood à output a single pixel) Linear filter: Additive Uniform!( f + g) =!( f )+!(g)!(" f ) = "!( f ), "! R Constancy in relation to a shift Usually operate according to convolution rule Nonlinear filter Filtering is useful for noise reduction!( f! h ) = [!( f )]! h Note: Results might need to be scaled (look-up table) to make sure the pixel values are integers within certain level

27 Image Filtering 1D Convolution: Continuous Discrete Example: $ # ( f! g)(x) = f (y)g(x " y)dy "# # $ ( f! g)[n] = f [m]g[n " m] m="#

28 Image Filtering 2D Convolution Continuous Discrete $ # $ # f (x, y)! g(x, y) = f (!,")g(x "!, y "")d! d" "# "# # f [m, n]! g[m, n] = f (i, j)g(m " i, n " j) # $ $ j="# i="# Example:

29 Low-Pass Filters: Mean Filters Low-Pass Filters The kernel (convolution mask) is filled with only positive values of coefficients Averaging filter / Mean filter! # # # " $ & & & %! # # # " $ & & & %

30 Low-Pass Filters: Gaussian Filters Gaussian kernel (smoothing) 1D: 1 2D: G(x) = G(x, y) = 2!" e! x 2 2" 2 x 1 2 +y 2 2!" e! 2" 2 Discrete approximation Smoothing / blurring images, similar to mean filter (weighted mean) Reduce the contrast of images Eliminate some material information about the edges or the texture of selected areas or structures

31 High-Pass Filters To isolate elements responsible for rapid brightness changes, e.g., contours and edges Often directional First-order derivative based Differentiation / finite difference Image Gradient Second-order derivative based Laplacian

32 High-Pass Filters: Differentiation Recall:!f!x = lim!x"0 f (x +!x, y)# f (x, y)!x " Possible kernels : directional discretize $ " $$$!f!x % f (x i#1, y)# f (x i+1, y) 2&x = ' x ( f " $ $ $ # ! % ' ' ' & " $ $ $ # 0 0 0! % ' ' ' & " $ $ $ # !1 0 % ' ' ' &

33 High-Pass Filters: Differentiation Responds strongly to noise Image noise usually results in pixels very different from neighbors Solutions Smoothing before differential filters Do we need to apply two filters (convolutions)? No, we can apply the derivative of Gaussian filter, because convolutions are associative! x "(G " f ) = (! x "G)" f! x "G! y "G

34 High-Pass Filters: Image Gradient Recall: let f(x,y) be a function of two variables Gradient at point (x,y)!f = [ The absolute value is large at boundaries "f "x "f "y % ] #! $ f ( x ' * &! y $ f )!f = ( "f "x )2 + ( "f "y )2 # (! x $ f ) 2 + (! y $ f ) 2 The direction corresponds to the steepest ascent Normally orthogonal to image boundaries small gradient

35 High-Pass Filters: Image Gradient Sobol derivative kernels # 1 0 "1 & % (! x : % 2 0 "2 ( % 1 0 "1 ( $ ' Directional variations à Edge detection # & % (!y :% ( % "1 "2 "1 ( $ '

36 High-Pass Filters: 2-nd Derivative 2 nd derivatives of images (divergence of gradient)!f = "2 f "x 2 + "2 f "y 2 = [ " "x " "y ] [ "f "x "f "y ] f = # #f Rota%onally Invariant

37 High-Pass Filters: 2-nd Derivative Using Laplacian for image enhancement #% g(x, y) = f (x, y)! "2 f (x, y) $ &% f (x, y)+ " 2 f (x, y) If center coefficient of Laplacian mask is nega%ve If center coefficient of Laplacian mask is posi%ve 2 nd derivative vs. gradient: 2 nd derivative: Superior in enhancing fine details Produce noisier results than the gradient Gradient: Has a stronger response in areas of significant gray-level transitions The response to fine details and noises are lower

38 Laplacian of Gaussian (LoG) Images should be smoothed a bit first! "G LoG(x, y) =! 1!" (1! x2 + y 2 )e! x 2 +y 2 2" 2 4 2" 2

39 Unsharp Masking Subtract a blurred version of an image from the image itself f s (x, y) = f (x, y)! f smoothed (x, y) High-boost filtering g(x, y) = wf (x, y)! f smoothed (x, y) = (w!1) f (x, y)+ f s (x, y) w: weight Laplacian is just one specific type of standard unsharp masking with w-1 =1 With increasing value of w, sharpening process becomes less and less important

40 Unsharp Masking Parameters Mask size: determine objects to be enhanced Small mask size: enhancement of small objects (vessels), high frequency components Large mask size: enhancement of large objects (vertebra), low frequency components Weight: determine strength of enhancement Unsharp mask kernal can be seen as a difference of two Gaussians (DoG) with 1 >> 2

41 Combining Spatial Enhancement Methods Laplacian of the image to obtain fine details Rather noisy: how to remove? Use a smoothed version of the gradient of the original image a. original b. Laplacian c. Laplacian sharpened (a+b)

42 d Sobel e smoothed sobel f laplacian sharpened smoothed (c) by e Combining Spatial Enhancement Methods To obtain a smoothed gradient To multiply it by the Laplacian image (works as a mask) Preserve details while reducing noise in the relatively flat area Roughly as combining the best features of the Laplacian and the gradient

43 Combining Spatial Enhancement Methods Add back to the original image for a sharpened image Power-law transformation to increase the dynamic range of the sharpened images g sharpened h Power- law transforma%on (r=0.5)

44 Combining Spatial Enhancement Methods a b c d e f g h

45 Combining Spatial Enhancement Summary Methods Utilize Laplacian to highlight fine details Utilize the gradient to enhance prominent features Smoothed version of the gradient is used to mask the Laplacian image to reduce noises Single-point transformation is used to increase the dynamic range of the gray levels

46 Filtering in MatLab Most of the filters / kernels mentioned earlier can be constructed in matlab by fspecial Average Gaussian Laplacian LoG Sobel Unsharp

47 Medical Image Enhancement Frequency Domain

48 Filtering in Frequency Domain Power spectrum of an image

49 Filtering in Frequency Domain Review: Discrete Fourier transform (DFT): F(u,v) F(u, v) = 1 MN N!1 M!1 "" y=0 x=0 xu! j2! ( M f (x, y)e + yv N ) Power spectrum: F(u,v) 2 High frequency Low frequency u v

50 Filtering in Frequency Domain Review 2: If F(u,v) = F [f(x,y)], H(u, v) = F [h(x,y)] Then F [f(x,y)*h(x,y)] = F(u,v) H(u,v) (element-by-element) FT of convolution of the image f(x,y) and the kernel h(x,y) is equal to the multiplication of each FT

51 Filtering in Frequency Domain Computational complexity: DFT: N-point data N 2 ultiplication, N(N-1) additionà O(N 2 ) F(u) = 1 N!1 xu! j2! ( N " f (x)e ) u = 0,1,..., N!1 N FFT: dividing into odd-number and even-number x=0 F(u) = 1 N = 1 N = 1 N n=0 N!1 " x=0 xu! j2! ( N f (x)e ) N /2!1 " f (2n) e! j2! ( 2nu N ) + 1 N N /2!1 " f (2n) e! j2! ( n=0 nu N /2 ) +e N /2!1 " f (2n +1) e! j2! ( n=0! j2! ( u N ) 1 N N /2!1 (2n+1)u ) N " f (2n +1) e! j2! ( n=0 nu N /2 ) = DFT N /2 ( f (0), f (2),... f (N! 2))+W N n DFT N /2 ( f (1), f (3),... f (N!1))

52 Filtering in Frequency Domain Computational complexity DFT: O(N 2 ) FFT: O(NlogN)

53 Filtering in Frequency Domain Flowchart of filtering in frequency domain Input image Real domain FFT Filtering (mul%plica%on) frequency domain Inverse FT Output image Real domain

54 Filtering in Frequency Domain 2D Filters in the frequency domain LPF HPF BPF f 0, f 1, f 2 are cut- off frequencies

55 Filtering in Frequency Domain X Original image A`er FFT

56 Smoothing Frequency-Domain Filters Attenuating a specified range of high-frequency components Ideal LPF! H(u, v) = " 1 # 0 If D(u,v) <= D 0 Otherwise Blurring: center component Ringing: concentric components

57 Smoothing Frequency-Domain Filters Butterworth LPF 1 H(u, v) = 1+ (D(u, v) / D 0 ) 2n n = 1, 2, 5, 20 n = 1: no ringing, LP effect is low n =2: imperceptible ringing but much less pronounced than ILPF n >2: more and more ringing effect BLPF of order 2 is a good compromise between effective lowpass filtering and acceptable ringing characteristics

58 Smoothing Frequency-Domain Filters Gassian lowpass filter H(u, v) = e!d2 (u,v)/2! 2 σ = D 0 is the cutoff frequency No ringing, suitable for applications where artifact is not acceptable (such as medical images!) Less smoothing compared to 2-order BLPF at the same cut-off frequency: BLPF is more suitable in applications where tight control of the transition around the cut-off frequency is needed

59 Sharpening Frequency-Domain Filters H hp (u, v) =1! H lp (u, v) Ideal HPF! H(u, v) = " 0 # 1 If D(u,v) <= D 0 Otherwise Ringing: concentric components Butterworth HPF Smoother than IHPF Gaussian HPF H(u, v) =1! e!d2 (u,v)/2! 2 1 H(u, v) = 1+ (D 0 / D(u, v)) 2n Smoother than IHPF & BHPF Can be constructed as DoG: allow more control over filter shape

60 Sharpening Frequency-Domain Filters Unsharp Masking & high-boost filtering f hb (x, y) = wf (x, y)! f lp (x, y) = (w!1) f (x, y)+ f hp (x, y) H hp (u, v) =1! H lp (u, v) H hb (u, v) = (w!1)+ H hp (u, v) w "1 high-frequency emphasis filtering H hfe (u, v) = a + bh hp (u, v), a! 0, b > a a: offset b: emphasizing high-frequency components

61 Nonlinear Filters Order-statistics filters Based on the ordering (ranking) the pixels in the neighborhood & replacing the value of the center pixel with the ranking results Median filter: for the processed point of the output image, the median filter selects a middle (median) value of the sorted values of the point and its neighbors in the input image Considered very radical Usually does not reduce the sharpness of object edges Does not introduce new values into the image, so no scaling is needed Unavoidable erosion of particularly small fragments Time-consuming Most frequently used to eliminate salt-and-pepper noise (impulse noise)

62 Geometric Mean Filters ˆf (x, y) = [ " (s,t)!s x,y g(s,t)] 1 MN " Achieve smoothing comparable to the arithmetic mean filter, but tends to lose less image details