Digital Image Processing

Size: px

Start display at page:

Download "Digital Image Processing"

Maximillian Roberts
6 years ago
Views:

1 Outline Digital Image Processing Digital Imaging and PACS: Applications in Radiolog 4-5 March 6 B Assistant Proessor Charnchai Pluempitiwiriawej Department o Electrical Engineering Chulalongkorn Universit Digital Images Digital Image Processing Image Acquisition Image Enhancement Image Restoration Image Segmentation What is an image? An image is a picture that represents some inormation. Representing a Digital it Image An image can be represented as a D unction ( x, ) ) Intensit (Gra Level) Spatial Coordinates For a digital image, x,, and are all inite and discrete. 3 4 Pixel Binar (Black & White) Image Picture element or image element is called a pixel (,) (,) (,3) (,4) (column) : R {,} (46,9) = (47,9) = (46,9) = (47,9) = (46,93) = (47,93) = Pixel (46,95) is on (46,94) = (47,94) = (46,95) = (47,95) = (,3) (,3) (,3) (3,3) (,) (,) (,) (3,) (,) (,) ) (,3) (,4) (3,) (3,) (3,3) (3,4) (48,9) = (48,9) = (48,93) = (48,94) = (48,95) = (,) (,) (,) (3,) (4,) (4,) (4,3) (4,4) (49,9) 9) (49,9) 9) (49,93) 93) (49,94) 94) (49,95) 95) = = = = = (,) (,) (,) (3,) x x (row) onl bit or each pixel is needed (5,9) (5,9) (5,93) (5,94) (5,95) = = = = = 5 Pixel (5,94) is o 6

2 Gra Scale (Intensit) Image : R R (4,4) = (4,5) (4,6) = 3 (4,7) = (4,8) = True Color Image 3 The intensit it value or pixel (4,35) 55 (4,35) is a vector with 3 components (channels) : R R 3 n bit-depth = n gra levels (5,4) (5,5) = 3 (5,6) = 8 (5,7) = 4 (5,8) (6,4) (6,5) (6,6) (6,7) (6,8) = = 3 5 = (7,4) (7,5) (7,6) (7,7) (7,8) = = 3 = 4 = (8,4) 5 (8,5) = 45 (8,6) = 45 (8,7) = 35 (8,8) = Image Intensit Or Gra Level R = G = B 5 R=55 G= B= R=5 G=5 B= How man colors can a 4-bit-color image represent? 6,777,6 7 copright b Charnchai Pluempitiwiriawej 7 8 Indexed d Image (46,9) (46,9) = (46,93) = (46,94) (46,95) (47,9) (47,9) (47,93) (47,94) (47,95) = = (48,9) (48,9) (48,93) (48,94) (48,95) = 4 = 4 = 3 = = (49,9) = 4 (49,9) (49,93) = 6 (49,94) (49,95) Color Map R G B Medical Images Provide inormation about the shape and unction o organs in human bod One o the most important t mean or establishing the diagnosis A special mean or controlling the therapeutic action (5,9) (5,9) (5,93) = 6 (5,94) = 6 (5,95) D Images 3D images (x,, z) ) or (x,, t) ) 4D Images 4D image (x,, z, t) )

3 Image = Inormation Digital it Image Processing (DIP) Manipulation o digital images b mean o computers or storage, transmission, and representation or autonomous machine perception Image Compression, Representation, ti etc. to improve pictorial inormation or human interpretation Image Enhancement, Restoration, Segmentation, etc. 3 4 Related Fields Computerized Processes Image Processing Image Analsis Computer Visioni Low level processing Noise reduction, Contrast enhancement, Image sharpening both inputs and outputs are images Mid-level Image segmentation, Image classiication Outputs = attributes extracted rom the input image High level Making sense o the ensemble o the recognized objects visual cognition 5 6 Fundamentals o DIP Medical Image Acquisition Knowledge based Image Acquisition Image Pre-processing Image Segmentation Image Interpretation Image Enhancement Image Restoration 7 The process b which h phsicians i evaluate an area o the subject's bod that is not externall visible Dierent Medical Imaging Modalities Radiograph (X-ra), Fluoroscop, Tomograph Magnetic Resonance Imaging (MRI) Ultrasound (Sonograph) Electron Microscop 8

4 Image Pre-processing Transorms input images into output images To change or improve qualit o the image Techniques which manipulate the pixel values o an image or some particular purpose. contrast or eature enhancement, color correction noise removal (iltering), artiact correction, image scaling (change size), image warping (change shape). Point Processing A unction T is applied on the intensit o each individual pixel (x, ). The gra value o the output image g at (x, ) depends on the gra value o the input image at (x, ) onl T is the gra level transormation unction g = T ( ) (x,) g(x,) intensit mapping 9 Gra Level Transormations g g g g Negative Contrast stretching n th root g n th power Piece-wise linear g Log Inverse log Non-monotonic unction is not recommended g = or [,] or g = L or =,,,L L Suitable or enhancing white or gra detail embedded in dark regions when the black area is dominant Negative Image Original mammogram showing a small lesion o a breast Negative Image : gives a better vision to analze the image Gamma Correction Images & Their Histograms Plots o g = c or various values o Input gra level =.4 =.3 =.6 3 Dark image: Components o the histogram are concentrated on the low side o the gra scale. Bright image Components o histogram are concentrated on the high side o the gra scale. Low contrast image Histogram is narrow High contrast image Histogram is broad and uniorm 4

5 Histogram Equalization Histogram Equalization g k k n j n j L or k,,..., L bit-image image g k n k k n j j k n j k j n n k g k Mask Mode Angiograph Neighborhood Processing M(x,, ) (x,, t) g(x,, t) = (x,, t) M(x,, ) The output t gra value g at pixel (x,) not onl depends on the input gra value at pixel (x,) but also the input gra values at the neighboring pixels o (x,). (x,) g(x,) (a) mask image (b) image ater injection o contrast medium (e.g., iodine) (c) Enhanced image (b) (a) g (x, ) = T{ (x, ), (x, ), (x, ), (x, ), } 7 8 Window A subset o pixels in an image can be represented b a binar matrix called a window (mask, template) The pixel o interest (x,) is oten (though not necessar) located at the center o the window The location o other pixels in the mask are measured relative to the pixel o interest (x,) (x,) ( x, ),( x, ),( x, ), w( x, ) ( x, ),( x, ) Sliding Window The center o the window is moved rom pixel to pixel throughout the input image. 9 3

6 Spatial Filtering Process When a unction is applied to the gra values o the pixels in the window, we call it a ilter or kernel. The response o the ilter (result o the unction) ) will be placed at the (x, ) location o the output image g(x,) = max{w x }+ min{w x } 7 Linear Filtering I the unction is linear (a linear ilter), the weights are oten shown as the mask coeicients w w w 3 w ( x, ) w ( x, ) w3 ( x, ) w 4 w 5 w 6 g( x, ) w4 ( x, ) w5 ( x, ) w6 ( x, ) w 7 w 8 w 9 w 7 ( x, ) w 8 ( x, ) w 9 ( x, ) Use zero-padding 3 3 Spatial Filtering Spatial iltering i is local l enhancement technique using window/mask. Smoothing Spatial Filters Linear Filters Averaging & Weighted Averaging Filters (Lowpass) Order Statistics Filters (non-linear) Median Filter, Min/Max Filters Sharpening Spatial Filters Gradient Filters & Laplacian Filters (Highpass) Composite Laplacian Filters, Unsharp Masking, & High-boost Filters Averaging Filter a c e b d a) original image 5x5 pixels b) - ) results o smoothing with square averaging ilter masks o size n = 3, 5, 9, 5 and 35, respectivel. Note: Big mask eliminates small objects in the image. The size o the mask establishes the relative size o the objects that will be blended with the background Eects o Laplacian Filter Prewitt Filters Highlights gra-level discontinuities in an image Deemphasizes regions with slowl varing gra levels Thereore, it tends to produce images that have graish edge lines and other discontinuities, all superimposed on a dark, eatureless background. 4 = x x z 7 z 8 z 9 z3 z6 z9 z z4 z z z z 3 7 w w w 3 w 4 w 5 w 6 w 7 w 8 w

7 Sobel Filters Sobel operators, 3x3 w w w x w w w w w w x w w w w w w w w w 3 w 4 w 5 w 6 w 7 w 8 w 9 Prewitt Comparison o Edge Detectors the weight value is to achieve smoothing b giving more important to the center point 37 Original image Sobel Vertical Horizontal Sum 38 a = original b = Laplacian o a e = d thru 5x5 averaging ilter Psudo-Coloring = c x e h = g.5 c = a + b d = Sobel o a g = + a Courtes o G.E. Medical Sstems 39 4 Image Registration & Fusion Combining inormation rom + images + SPECT MRI Image Segmentation Subdivides an image into constituent regions. Result = Object + Background + CT MRI 4 4

8 Liver Segmentation Cardiac MR Image Segmentation ti Find the volume o the heart chamber and dthe thickness o the heart wall copright b Charnchai Pluempitiwiriawej 44 4D Cardiac MR Image Segmentation Image Segmentation Methods Histogram-based Methods Thresholdings Gradient-based Methods Edge Detectors Region-based Methods Region Growing, Region Split & Merge Active Contour Methods Snake, GVF snakes, ACWE, LRES, LREK, Other Methods Watershed Thresholding Methods Investigate the image s histogram and choose a threshold that separate the image intensit into or more groups. Thresholds ma be chosen heuristicall or automaticall applied globall or locall Estimated Global Threshold Select an initial estimate or T Segment the image using T, produce two groups o pixels G and G Compute the average gra level values m and m o pixels in each group Compute a new threshold value T = (m +m )/ Repeat step until the dierence in T in successive iterations is small 47 48

9 Otsu s Optimal Thresholding Estimate a threshold T that minimizes some segmentation error criteria T T P pzdz P p z e T dz Flaws o Thresholding Onl group regions with similar intensit but no constrains on connected pixels. Oten ail with nonuniorm background 49 5 Gradient-Based Methods Appl the gradient operator onto the image to ind regions with abrupt change (discontinuit). Quite sensitive to noise (sudden change o intensit values). Pre-process b lowpass ilter. Laplacian o Gaussian (LoG) Smooth the image irst with a Gaussian r h r e Appl Laplacian r r r h r 4 e Find zero-crossing Threshold the image 5 5 Gradient vs. LoG Region Growing Method Procedure Start with a set o seed points Grow regions b appending to each seed those neighbor pixels with similar properties. Challenges How to select appropriate seeds Criteria used to grow regions Descriptors Connectedness Stopping rule

10 input image 3D Liver Segmentations manual segmentation thresholdingh region growing Watershed Method Consider an image as a 3D surace Find its local minima Flood the image (rom below) As the water rises, i the water rom one catchment basin (or watershed) is about to overlow into another basin, we build a dam to divide them orming watershed lines Watershed Algorithm Dam Construction (a) Flooding step n C M C M n n (b) Flooding step n (c) Perorm dilation q around C n (M ) and C n (M ) such that Onl on pixels in q I an dilating point will cause the two region to merge, the dam is built Result o Watershed Active Contour (Snake) a curve Cs () [(), xs s ()], s[,] that moves through the spatial domain o an image to minimize an energ unction E( C) E ( C) E ( C) ds internal C F internal F t external external Evolution equation F internal F external controls the smoothness o the contour. pushes the contour toward the object in the image. 59 6

11 Internal Force To control the smoothness o the contour. 4 C C E C C internal( C) F internal s s 4 s s.and are the weights controlling the snake s tension and rigidit, respectivel., Initial Contour Result 6 External Force To push the contour toward the object in the image. Edge-Based - Traditional Snake - GVF Snake - DDGVF Snake Region-Based - Region-Based Snake 6 Traditional Snake Gradient Vector Flow (GVF) I ( x, ) ( x, ) Edge Map ( x, ) G ( x, ) I( x, ) potential orces F F external traditional - Limited Capture Range - Poor Convergence F F V external ( x, ) GVF ield V( x, ) [ u( x, ), v( x, )] GVF Vt V ( V ), V potential orces - Large Capture Range - Good Convergence G.is a -D Gaussian unction with standard deviation..is the gradient operator. 63.is a regularization parameter..is the Laplacian operator. 64 Traditional vs. GVF Snakes Dnamic Directional GVF Snake Traditional & GVF Edge Map ( x, ) G ( x, ) I( x, ) DDGVF Edge Map ( x, ) x ( x, ), x ( x, ), ( x, ), ( x, ) x 4 directions x x Positive Edge Negative Edge I( x, ) potential orce ield potential orce ield 65 66

12 Positive vs. Negative Edges x I( x, ) Positive Edge x ( x, ) x 4 directions x ( x, ) x ( x, ) ( x, ) DDGVF Force Fields Positive Edges ( x, ) x ( x, ), x ( x, ), ( x, ), ( x, ) DDGVF ield Vx (, ) u( x, ), u( x, ), v( x, ), v( x, ) B A u, v u, v Fexternal FDDGVF [ Fx, F ] x x, x, D B C A x, x, Negative Edge ( x, ) ( x, ) x x ( x, ) ( x, ) C u, v D u, v x max cos( ), min cos( ), max sin( ), minsin( ), F u u F v v cos( ). is the normal vector s component in the x direction. sin( ). is the normal vector s component in the direction. 67 potential orces 68 Results o DDGVF External Force o Region-Based F external F DDGVF+ F external F DDGVF- Fexternal Fregion in ( I Rin ) out ( I Rout ) n n.is the unit outward normal vector o the contour. (a) (b) (c) (a) Initial snake position (b) Result o boundar detection using positive edge (c) Result o boundar detection using negative edge Initial Contour Segmentation Result 69 7 Edge-based vs. Region-based Snakes on MR Cardiac Images GVF Snake (Edge-Based) Region-Based Snake F external FGVF external region F F Initial Contour Traditional Snake GVF Snake 7 DDGVF Snake (Positive Edge) DDGVF Snake (Negative Edge) Region-Based Snake 7

13 Double Snakes Contour A To trace endocardial boundar. Contour B To trace epicardial boundar. - Segment both boundaries simultaneousl. - Overcome papillar muscles. - No training needed. - The two contours do not merge and intersect. B Sopon Phumeechana & Charnchai Pluempitiwiriawej 73 Contour B Inter-Contour Force (To control the distance between Contour A and Contour B) Weighting parameters C t B F w F wf internal R reg ion I inter-contour Region-Based Force (To push the contour toward the boundar) Internal Force (To control the smoothness o the contour) 74 Contour B Inter-Contour Force r Pixels Contour A + Contour B Region division ision o the average intensit in the image. R 4 R 3 R R 75 The distance and the relationship o the both contours. 76 Experiments Snthetic Image 6 Cardiac MR Images Snthetic Image r 5 w R R BW BW R 3 Pixels.7 Gra Scale Image w w I R Number o Iterations x pixels 5 x 5 pixels Compare our results to the manuall segmented image set. Area similarit Shape similarit (Threshold =.5) 77 Binar Image phase phase phase3 78

14 r Inter-Contour Distance () r Pixels r 5 Pixels r Pixels r 5 Pixels r Inter Contour Distance () r 3 Pixels r 35 Pixels r 4 Pixels r 45 Pixels 79 8 r Inter-Contour Distance (3) r 5 Pixels r 55 Pixels r 6 Pixels 6 Cardiac MR Images Frame Frame Frame9 Frame Apex Middle Base 8 8 Example : Apex w R R BW BW R 3 r Pixels Example : Middle w R R r Pixels BW BW R 3 Apex Slice Image Binar Image (Threshold =.4) w w I R Middle Slice Image Binar Image (Threshold =.4) w w I R Initial Contours Segmentation Result 83 Initial Contours Segmentation Result 84

15 Tagline Detection Mocardial Motion Detection D/4D Rendering Lesion Detection Medical Image Processing Input data is usuall 3D or 4D Human-Machine Partnerships Computers are to assist, to acilitate, NOT to replace human experts Detection o lesions and pathologies is oten done qualitativel and mostl subjective But computers can measure quantitativel, e.g., tumor volume, length o a bone racture MIP or Diagnosis () Quantitative measurement o several image parameters (size, color, shape, texture) Change detection among images acquired in dierent time instants. The time interval can be a ew seconds (angiographic sequence) or several months or ollow-up purposes using images o the same modalit Data usion o dierent imaging i modalities to allow combination o complementar inormation o the same patient 89 9

16 MIP or Diagnosis () Comparison o images with the same imaging modalit but dierent patients: useul or studing a particular patholog Indexing an image database Image movement characterization and articulation o human organs Data visualization o volume and dnamic scenes Three Basic Areas o MIP Image Pre-processing Image enhancement, restoration (iltering), Image Segmentation Partitioning an image into contiguous regions with cohesive properties Image Registration Aligning multiple images to use dierent inormation or more powerul diagnostic tool 9 9 More Advanced Areas o MIP Motion Analsis Angiographic sequences, drug perusion Image Visualization D/3D/4D representation, volume rendering Surger Simulation Geometric and biomechanical models o organs and tissues or training Medical Robotic Robotic Surger, Telemedicine 93

Digital Image Processing. Image Enhancement in the Spatial Domain (Chapter 4)

Digital Image Processing. Image Enhancement in the Spatial Domain (Chapter 4) Digital Image Processing Image Enhancement in the Spatial Domain (Chapter 4) Objective The principal objective o enhancement is to process an images so that the result is more suitable than the original