Overview Module 03. Image Restoration and Freq. Filtering & Color Imaging and Transformations. Model of Image Degradation/Restoration

Transcription

1 Introduction to Medical Image Processing (5XSA), Module 3 Image Restoration and Freq. Filtering & Color Imaging and Transformations Peter H.N. de With (p.h.n.de.with@tue.nl ) slides version. Overview Module 3 Part a: Noise and filtering Noise model Spatial filtering and periodic noise Part b: Special filtering techniques Frequency-Band Filtering, Adaptive filtering Part : Color imaging and transformations Color models and systems Color representations and associated processing Part 3: Cases with medical image restoration Case : Ultrasound imaging, Case : Cancer detection, analysis 3 Model of Image Degradation/Restoration 4 Module 3 Part Image Restoration & Reconstruction Noise model spatial noise filter freq.-domain noise filter projection imaging and CT. Model communication imperfections and noise insertion by a channel with degradation and restoration filters.. Degradation can be derived by measuring and analysis 3. Model is linear (pos. invarian filter process & addition of noise, so g( = h( f ( + n( G( = H ( F( + N( Noise Sources and Considerations 5 Widely Used Noise Models () 6. Noise is generated when capturing the image, caused by. Sensor: intrinsic noise and temperature. Light level of the scene 3. Insufficient resolution processing (cost reduction). Noise has spatial and frequency propertie so it can be analyzed with the DFT and in time domain 3. For analysis purposes (also here), noise is assumed to be spatially and signal independent. This is sometimes invalid (e.g. X-ray and nuclear medicine imaging) g( = h( f ( + n( G( = H ( F( + N(. Gaussian (normal) noise. Mathematically tractable. Note mean µ and st.dev. σ. Rayleigh noise p( z. Often in comm. problems. Has mean and variance with 3. Erlang (Gamma) noise. Has mean and variance. a> and b positive integer p( z) = e σ π ) = ( z a) e b ( z a) / b ( z µ ) / σ ( z a) z = µ = a + πb / 4 ; σ = b(4 π ) / 4 b b a z p( z) = e ( b )! az z = µ = b σ = ( z ) / a; b / a

2 Widely Used Noise Models () 7 Illustrated noise models 8. Exponential noise. Mathematically tractable. Note mean µ and st.dev. σ az p( z) = ae ( a, z ) µ = σ / a ; = / a. Uniform noise p( z) = ; ( a z b). Often in analysis problems b a. Has mean and variance with µ = ( a + b) / ; σ = ( b a) / 3. Impulse (salt and pepper) noise p( z) = Pa. Noise pulses can be pos. or negative p( z) = Pb. b>a, intensity b will be light dot 3. If either probability is zero: unipolar noise ( z = a) ( z = b) Adding noise models to images () 9 Adding noise models to images () Gaussian Rayleigh Gamma Exponential Uniform Impulse (S&P) Periodic noise Periodic noise Visual example Periodic noise is spatially dependent (exception!) Very suited for filtering in the frequency domain! Periodic noise is spatially dependent Model is sinusoid, with r( = Asin[πu ( x + B ) / M + πv ( y + B ) / N] x y u and v are frequencie B s are phase (displacements) Note spectrum dots! Derive the frequency domain representation Each sine wave gives a conjugate pair in freq. domain

3 Measuring noise parameters Take selected area(s) of image which have no detail and L measure the local histogram µ = z p ( z ) i S i i= Measure e.g. mean and variance L σ = ( z µ ) p ( z ) S i i= 3 Spatial mean filtering / restoration of noise Covers the case of noise only. Arithmetic mean filter. Simple filter, average in area. Rectangular window mxn g ( = f ( + n( fˆ( = mn g( ( Sxy 4. Geometric mean filter. Perform similar to mean filter. Tends to loose less details 3. Harmonic mean filter fˆ( =. Works well for salt noise, not pepper. Performs well also for Gaussian noise fˆ( = g( ( t ) Sxy mn mn ( t ) Sxy g( Mean filtering Example X-ray image 5 Mean filtering Example of X-ray image 6 fˆ( = g( ( S xy g( ( t ) Sxy Q+ Contraharmonic mean filter reduces salt-&-pepper noise Q Mean filtering Example of X-ray image 3 7 Order-statistic filters / Median etc. Covers the case of noise only. Order-statistic filters are spatial filters whose response is based on ordering/ranking the pixels. Is a non-linear process.. Median filter. Widely applied. Selects the median of a set of samples 3. Excellent results for speckle noise, no blur fˆ( = median{ g( } ( Sxy 8 In general, arithmetic an geometric mean filters are well suited for random noise (such as Gaussian, uniform). The contraharmonic filter does well for impulse noise, but the dark/light color has to be known.. Max and min filter. Useful for finding bright/dark points. Max filter reduced pepper noise 3. Min filter reduced salt noise fˆ( = max{ g( } ( t ) Sxy fˆ( = min { g( )} t ( s, t ) S xy 3

4 Example results with 3x3 Median filters 9 Example results with max/min filters Median, one iter. Median, two iter. Median, three iter. Note the brightening effect of the max filter and the darkening effect of the min filter! Sample results with mean & median filters Adaptive filters Adaptive filters Perform better than static filters as presented earlier They have a higher complexity, since measurements are required Operate within a window and then adapt to the image contents. Adaptive local noise reduction filter, operating in area S, depending on 4 quantities. g( the value of the noisy image. σ η, the variance of the noise corrupting f( 3. the local mean m L of the pixels in the window 4. σ L, the local variance of the pixels in the window Local noise reduction filter 3 Local noise reduction filter - Result 4 The desired behavior of the filter should be that. If σ η = the filter should return simply the value of g(. If the local variance is high relative to σ η, the filter should return a value close to g(. Edges should be preserved. 3. If the two variances are equal, the filter should return the arithmetic mean value of the pixels in S. This means that the local area has the same properties as the overall image and local noise then averaged. The filter could be as follows σ η L [ g( y m ] fˆ( = g( ) σ L Adaptive filters are always better! 4

5 Adaptive median filter () Normal median filter can handle low-density noise Varying density noise requires other filtering An adaptive median filter may change the filter window S The filter still replaces one pixel with the filter output Consider that z min = min of window, z max = max of window, z med = median of window, S max = max window size 5 Adaptive median filter () / Algorithm Stage A A = z med z min ; A = z med z max. If A > and A <, go to stage B Else increase window size If window size <= S max repeat stage A Else output z med Stage B B = z xy z min ; B = z xy z max. If B > and B <, output z xy Else output z med Determine in this stage, whether output z med is impulse or not Test whether data is impulse or not, if so, then filter 6 Adaptive median filter (3) / Results 7 Frequency domain filters Bandreject 8 Sometime the spectral noise location is known, e.g. with periodic noise. Then define specific spectral noise filters Static filter shows a significant loss of detail (broken conn.)! Additive periodic noise can be approximated by -d sinusoidal functions. Remember the DFT of sine is two conjugate impulses! Frequency domain filters Bandreject 9 Frequency domain filters Bandpass 3 Bandpass filter is opposite of bandreject, hence H BP (= - H BR ( Bandpass filter is opposite of bandreject, hence H BP =-H BR 5

6 Frequency domain filters Notch filters A notch filter rejects (passes) frequencies in a neighborhood about a center 3 Freq.- domain filters Notch filter results 3 Estimation of the degradation function 33 Example: blur caused by linear motion - 34 Obtain the degradation by Observation Function is not known Measure in rectangles in foreground and background Experimentation Perform experiments with equipment to emulate conditions Impose image impulse, measure impulse response, H=G(/A Mathematical modeling Test with modeled filter such as Gaussian, Laplacian etc. T Consider the shift g( = f [( x x (, y y ( ]dt Integration constant with time T and g the blurred image Taking the Fourier transform and reversing the integration over T with that of Fourier gives G( = F( e T j π [ ux( t ) + vy ( t )] dt = F( H ( Hence, H is the integral over de exponential If the motion variable are known, then H is computed Example: blur caused by linear motion - 35 Example: blur caused by linear motion Consider the function x ( = at / T Taking the Fourier transform and deriving further T T j πux t j πuat T T jπua H ( = e ( ) / dt = e dt = sin( πua) e πua This result can be extended in D with y ( bt / T = 6

7 Inverse filtering Suppose we have degradation function H( Simplest approach: inverse filtering F h ( = G( / H(, then with add. noise, we find F h ( = F( +N( / H( Note that N( is not known! Also: when H( is zero, then the ratio explodes., this happens frequently One way around: limit filter frequencies to values close to origin where H is high typically This slide is intentionally left blank 39 4 This slide is intentionally left blank This slide is intentionally left blank 4 4 This slide is intentionally left blank This slide is intentionally left blank 7

8 Module 3 Part Color image processing 43 Color processing Full-color processing Involves TV, broadcast and consumer camera etc Pseudo color processing Often used in medical domain: generate color to show 44 Color fundamental triangle, color model color conversion Color light wavelengths 45 Color mixtures 46 Human eye cones measure light 65% red, 33% green, % blue All cones not equal! RGB are thus primary colors Blue=435nm Green= 546nm Red=7nm Primary colors can be mixed to secondary colors Magenta=R+B Cyan=G+B Yellow=R+G Mixing secondary color with an opposite primary (or 3 prim.) gives white Color triangle 47 Typical color models 48 Colors have been specified by CIE In a triangle x (red) and y (green) z (blue) = - (x+ Color is specified in models RGB, Red Green Blue, for TV & cameras CMYK, Cyan Magenta Yellow Black, for printing The color black in CMYK is extra HSI, Hue Saturation Intensity, for grayscale printing, old TV, etc Any point on boundary is fully saturated 8

9 RGB is based on Cartesian coordinate RGB on axe CMY on corner Black at origin RGB Color Model 6M colors! 49 RGB safe color cube for coloring images RGB safe color cube: only colors at the surface of the cube, total 6 colors Guarantee colors at every 8-bit computer/display Each surface has 6x6 colors All 8-bit gray levels are included supports webbased & pseudocolor imaging 5 CMY and CMYK Color Models CMY are secondary color or primary colors of pigments Printing popularity comes from: cyan does not reflect red from white light illumination Most devices using colored pigments require CMY input Conversion from RGB is simple, and results from (C, M, Y) T =(,, ) T (R, G, B) T Based on assumption that all colors have been normalized to unit interval Note that first component equation reflects absence of red, magenta reflects no green, and yellow no blue. Likewise, RGB can be easliy obtained from CMY 5 HSI Color Model HSI originates from the wish to describe images in Intensity (black & white or gray level image) Saturation (strength of the pure color componen Hue (color temperature, simply the type of color) HSI is much more suited for describing color image rather than RGB, that is more suited for color image generation Convert from RGB cube, by tilting it such that black is at the bottom and white at the top. 5 HSI Color understanding the RGB cube Convert from RGB cube, by tilting it such that black is at the bottom and white at the top. Intensity is covered by passing a plane perpendicular to the intensity axi and hue is in the shaded plane Shaded plane contains both BW and cyan 53 HSI Color and Saturation () Convert from RGB cube, cross planes are triangular or hexagonal Hue is angle leftwise from red orientation Saturation is the length of the vector Origin is at intersection with intensity plane 54 9

10 HSI Color and Saturation () 55 Converting color from RGB to HSI () 56 From each RGB pixel, the H value H = θ H = 36 θ B G B > G The angle comes from a.o. goniometry analysis / [( R G) + R B)] θ = arccos{ } / [( R G) + ( R B)( G B)] The saturation and intensity are given by 3 S = [min( R, G, B)] I = ( R + G + B) ( R + G + B) 3 Converting color from RGB to HSI () Note the following properties - RGB colors in (a) have fixed saturation - Intensity grows when more color is added Sat. 57 Hue Int. Pseudo color - Intensity slicing Let the gray scale vary between (black) and L- (white) Assume P planes perpendicular to the intensity axis P+ partitions of gray scale are made Each value intensity l k converts to color c k 58 Geometric interpretation of intensity slicing 59 Color / Example of intensity slicing 6 Alternative view f ( = c if f ( k V k

11 Module 3 Part 3 Medical Cases with Image Restoration and Frequency Case : UltraSound imaging of needles & Contributed by ir. Arash Poutaherian Case : Frequency analysis of colon cancer Contributed by ir. Fons van der Sommen 6 Case : Ultrasound-guided interventions Ultrasound-guided interventions with needles Contributed by ir. Arash Poutaherian 6 Case / Example US - Challenges Limited field of view in D ultrasound: Any motion may exclude parts of the needle from ultrasound field. Considerable training is required to avoid wrong needle placement. Extremely challenging, tense, lengthy procedure. Existing tracking systems (electromagn., optical, robotics) require extra equipment, specific skill add to the costs. Ultrasound Image 63 Case : Ultrasound Image Proc. Challenges Low signal to noise, e.g. spurious speckle noise occurs Various imaging artifacts (e.g. ringing, limited depth) Needle visibility decreases with the insertion angle During an intervention, only short parts of the needle are visible in initial frames 64 Case : Ultrasound Proposed System 3D ultrasound imaging Extra dimension: Larger field of view Coarse placement of the 3D transducer Image-based needle tracking Without utilizing external tracking devices No additional setup is required in the operating room Similar procedure for the operators 65 Case : Ultrasound image restoration () Assuming the speckle noise as an impulsive effect, median filtering can be used to reduce the speckles. 66

12 Case : Ultrasound image restoration () Anisotropic-diffusion filters reduce speckles by smoothing noisy regions with a Gaussian filter. The rate of diffusion is controlled by local statistics such as the image gradient. 67 Case : Ultrasound image restoration (3) Adaptive histogram equalization minimizes the difference in needle brightness at different insertion angles. 68 Case : Esophageal cancer detection with visual imaging Risk factors: obesity, reflu genetics This case is contributed by ir. Fons van der Sommen 69 Case : Detection esophageal cancer dream 7 Fastest rising type of cancer in the Western world 5 th most prevalent cancer, -5% five-year survival rate 8% of patients are diagnosed in a late stadium Esophagectomy up to % die due to surgery complications High morbidity trouble with eating etc. Gastroenterologist uses Endoscope for inspection System: lesion detected Supportive detection system for the detecion of early esophageal cancer. Early detection: % five-year-survival-rate* instead of -5% for esophageal cancer. * Patients who have died due to other implications are not included. Case : Esophageal cancer / Objective 7 Case : Esophagus cancer evolves gradually 7 Create a real-timecomputer aided detection system which aids the physician to identify early cancer in the esophagus Applications Barrett Low-grade dysplasia High-grade dysplasia Early cancer Real-time detection: endoscopy examination and surgery Off-line detection: training gastroenterologists quality assurance in existing databases Advanced stage cancer

13 Case : Detection cancer special direct. filters Use specially tuned filters to quantize deviating texture related to early lesions. Employ machine learning techniques to differentiate between cancerous and non-cancerous tissue. 73 Case : Detection cancer Experiment. results Latest study with 5 international experts Detection performance close to expert level. Precision/recall: System:.7/.88 to./.7 Experts:.85/.8 to./.7 Patient-based detection of early cancer Early carcinoma High-grade dysplasia False detections System ps 9 / 3 6 / 8 System ps / 3 7 / 8 6 System ps3 3 / 3 8 / 8 9 Expert 3 / 3 6 / 8 Expert 3 3 / 3 5 / 8 4 Expert 4 3 / 3 8 / 8 8 Expert 5 3 / 3 7 / 8 4 Expert System (Gold standard) Expert Expert 3 Expert 4 Expert 5 74 Case : Frequency analysis () 75 Case : Frequency analysis () 76 Cancerous tissue Healthy tissue Example: detection of cancerous tissue Known from medical literature: difference in texture. Texture can be considered as a combination of frequencies. We can analyze frequencies using the Fourier transform! Question: Can we observe the difference between cancerous tissue and healthy tissue in the frequency domain? If we can, we can develop filters for these frequencies to discriminate between cancer and healthy tissue. Image annotated by medical expert Analyze spectrum for the cancer blocks (red) and the healthy blocks (green). Exclude ambiguous blocks from analysis (yellow). Case : Frequency analysis (3) 77 Case : Frequency analysis (4) 78 Cancer spectrum Healthy spectrum Difference observed in low frequencies Can we find the exact frequencies? 3

14 Case : Frequency analysis (5) 79 Case : Image restoration / Reflections () 8 Resulting spectra: Blue > Specular reflections of the light source Detect reflections with a threshold Find a smart threshold by analysis of color channels Use interpolation to find the missing pixel values Case : Image restoration / Reflections () 8 Specular reflections in endoscopic image Image after restoration 4