Numerical Methods on the Image Processing Problems

Transcription

1 Numerical Methods on the Image Processing Problems Department of Mathematics and Statistics Mississippi State University December 13, 2006

2 Objective Develop efficient PDE (partial differential equations) based mathematical models and their numerical algorithms for 1 Noise removal Enhance the quality of images 2 Image segmentation Edge (2D) or surface (3D) detection

3 Objective Develop efficient PDE (partial differential equations) based mathematical models and their numerical algorithms for 1 Noise removal Enhance the quality of images 2 Image segmentation Edge (2D) or surface (3D) detection

4 Objective Develop efficient PDE (partial differential equations) based mathematical models and their numerical algorithms for 1 Noise removal Enhance the quality of images 2 Image segmentation Edge (2D) or surface (3D) detection

5 Applications - Noise Removal Image with 20% impulse noise (left) and denoised image (right)

6 Applications - Noise Removal Image with 10% impulse noise (left) and denoised image (right)

7 Applications - Noise Removal Color image with impulse noise (left) and denoised image (right)

8 Applications - Edge detection

9 Applications - Edge detection Medical Imaging

10 Applications - Edge detection Image Analysis in Materials Science

11 Outline 1 History - PDE based Mathematical Image Processing 2 Image Denoising Conventional approach - Total variation (TV) minimization New models and their numerical procedure - Non-convex diffusion model - Kim and Lim ( 05) - Anisotropic diffusion model - Lim and Williams ( 06) 3 Image Segmentation Conventional Approach Method of Background Subtraction (MBS) for Medical Image Segmentation - Kim and Lim ( 05) 4 Conclusions

12 Outline 1 History - PDE based Mathematical Image Processing 2 Image Denoising Conventional approach - Total variation (TV) minimization New models and their numerical procedure - Non-convex diffusion model - Kim and Lim ( 05) - Anisotropic diffusion model - Lim and Williams ( 06) 3 Image Segmentation Conventional Approach Method of Background Subtraction (MBS) for Medical Image Segmentation - Kim and Lim ( 05) 4 Conclusions

13 Outline 1 History - PDE based Mathematical Image Processing 2 Image Denoising Conventional approach - Total variation (TV) minimization New models and their numerical procedure - Non-convex diffusion model - Kim and Lim ( 05) - Anisotropic diffusion model - Lim and Williams ( 06) 3 Image Segmentation Conventional Approach Method of Background Subtraction (MBS) for Medical Image Segmentation - Kim and Lim ( 05) 4 Conclusions

14 Outline 1 History - PDE based Mathematical Image Processing 2 Image Denoising Conventional approach - Total variation (TV) minimization New models and their numerical procedure - Non-convex diffusion model - Kim and Lim ( 05) - Anisotropic diffusion model - Lim and Williams ( 06) 3 Image Segmentation Conventional Approach Method of Background Subtraction (MBS) for Medical Image Segmentation - Kim and Lim ( 05) 4 Conclusions

15 Outline 1 History - PDE based Mathematical Image Processing 2 Image Denoising Conventional approach - Total variation (TV) minimization New models and their numerical procedure - Non-convex diffusion model - Kim and Lim ( 05) - Anisotropic diffusion model - Lim and Williams ( 06) 3 Image Segmentation Conventional Approach Method of Background Subtraction (MBS) for Medical Image Segmentation - Kim and Lim ( 05) 4 Conclusions

16 History of PDE based Mathematical Image Processing Short history, but has strong impact Image denoising, deconvolution (deblurring), segmentation (edge/surface detection) Image denoising - Total variation (TV) minimization (Osher ( 92), Lions ( 97), Chan ( 98), Kim ( 01)) - Weakness: Edges of images can be easily smeared out due to diffusion property of TV minimization original image (left) and smeared image (right)

17 History of PDE based Mathematical Image Processing Short history, but has strong impact Image denoising, deconvolution (deblurring), segmentation (edge/surface detection) Image denoising - Total variation (TV) minimization (Osher ( 92), Lions ( 97), Chan ( 98), Kim ( 01)) - Weakness: Edges of images can be easily smeared out due to diffusion property of TV minimization original image (left) and smeared image (right)

18 History of PDE based Mathematical Image Processing Short history, but has strong impact Image denoising, deconvolution (deblurring), segmentation (edge/surface detection) Image denoising - Total variation (TV) minimization (Osher ( 92), Lions ( 97), Chan ( 98), Kim ( 01)) - Weakness: Edges of images can be easily smeared out due to diffusion property of TV minimization original image (left) and smeared image (right)

19 History of PDE based Mathematical Image Processing Short history, but has strong impact Image denoising, deconvolution (deblurring), segmentation (edge/surface detection) Image denoising - Total variation (TV) minimization (Osher ( 92), Lions ( 97), Chan ( 98), Kim ( 01)) - Weakness: Edges of images can be easily smeared out due to diffusion property of TV minimization original image (left) and smeared image (right)

20 History of PDE based Mathematical Image Processing Color image denoising (Kim ( 02), Osher ( 03)) - Use RGB color component Color image with 15% impulse noise (left) and denoised image (right)

21 Image Denoising Conventional approach TV minimization model u t σ u γ+1 ( ) u = β (u o u) u Efficiently removes noise Image loses sharpness since the frequency of noise and edges are similar Produces a staircasing (locally constant) effect and nonphysical dissipation

22 Image Denoising Conventional approach TV minimization model u t σ u γ+1 ( ) u = β (u o u) u Efficiently removes noise Image loses sharpness since the frequency of noise and edges are similar Produces a staircasing (locally constant) effect and nonphysical dissipation

23 Image Denoising Conventional approach TV minimization model u t σ u γ+1 ( ) u = β (u o u) u Efficiently removes noise Image loses sharpness since the frequency of noise and edges are similar Produces a staircasing (locally constant) effect and nonphysical dissipation

24 Image Denoising Conventional approach TV minimization model u t σ u γ+1 ( ) u = β (u o u) u Efficiently removes noise Image loses sharpness since the frequency of noise and edges are similar Produces a staircasing (locally constant) effect and nonphysical dissipation

25 Image Denoising - NC Model Non-convex diffusion model Control of nonphysical dissipation Consider min F ɛ,p (u), where u F ɛ,p (u) = Ω ɛ u p dx + λ 2 f u 2. Then ( ) u p ɛ u 2 p λ(f u) = 0, where ɛ u = (u 2 x + u 2 y + ɛ 2 ) 1/2. Sharp (left) and blurry image (right)

26 Image Denoising - NC Model Non-convex diffusion model F ɛ,p Sharp image Blurry image F 0, = = 0.5 F 0, = = 1 F 0.1, F 0, = The strictly convex minimization (p > 1) makes images blurrier. The TV model itself (p = 1 and ɛ = 0) may not introduce blur but its regularization (p = 1 and ɛ > 0) does. When p < 1(non-convex), the model can make the image sharper.

27 Image Denoising - NC Model Non-convex diffusion model F ɛ,p Sharp image Blurry image F 0, = = 0.5 F 0, = = 1 F 0.1, F 0, = The strictly convex minimization (p > 1) makes images blurrier. The TV model itself (p = 1 and ɛ = 0) may not introduce blur but its regularization (p = 1 and ɛ > 0) does. When p < 1(non-convex), the model can make the image sharper.

28 Image Denoising - NC Model Non-convex diffusion model F ɛ,p Sharp image Blurry image F 0, = = 0.5 F 0, = = 1 F 0.1, F 0, = The strictly convex minimization (p > 1) makes images blurrier. The TV model itself (p = 1 and ɛ = 0) may not introduce blur but its regularization (p = 1 and ɛ > 0) does. When p < 1(non-convex), the model can make the image sharper.

29 Image Denoising - NC Model Non-convex diffusion model F ɛ,p Sharp image Blurry image F 0, = = 0.5 F 0, = = 1 F 0.1, F 0, = The strictly convex minimization (p > 1) makes images blurrier. The TV model itself (p = 1 and ɛ = 0) may not introduce blur but its regularization (p = 1 and ɛ > 0) does. When p < 1(non-convex), the model can make the image sharper.

30 Image Denoising - NC Model Non-convex diffusion model New non-convex (NC) model ( ) u u t ɛ u 1+ω ɛ u 1+ω = β (f u), ω ( 1, 1), β 0 Numerical procedures Linearized θ- method. Alternating Direction Implicit (ADI) method Theorem (Stability) The θ- method for the new NC model is stable and holds the maximum principle.

31 Image Denoising - NC Model Non-convex diffusion model New non-convex (NC) model ( ) u u t ɛ u 1+ω ɛ u 1+ω = β (f u), ω ( 1, 1), β 0 Numerical procedures Linearized θ- method. Alternating Direction Implicit (ADI) method Theorem (Stability) The θ- method for the new NC model is stable and holds the maximum principle.

32 Image Denoising - NC Model Non-convex diffusion model New non-convex (NC) model ( ) u u t ɛ u 1+ω ɛ u 1+ω = β (f u), ω ( 1, 1), β 0 Numerical procedures Linearized θ- method. Alternating Direction Implicit (ADI) method Theorem (Stability) The θ- method for the new NC model is stable and holds the maximum principle.

33 Numerical Experiments - NC Model Image with 20% mean zero noise (left), conventional method (middle), new NC method (right)

34 Numerical Experiments - NC Model Image with 20% mean zero noise (left), conventional method (middle), new NC method (right)

35 Numerical Experiments - NC Model Original brain image (left) and enhanced image by new NC method (right)

36 Numerical Experiments - NC Model Horizontal line cuts of brain image and its restored images

37 Image Denoising - AD Model Anisotropic diffusion model Speckle noise is multiplicative and it can be modeled as (Krissian et. al. 04, 05): f = u + un, Corresponding time marching equation: u t u2 ( u ) f + u u u = λ u (f u). u2 f +u u/2 makes the diffusion faster in the lighter region (where the image values are high) and slower in the darker region (where the image values are low). unrealistic and ineffective in practice!

38 Image Denoising - AD Model Anisotropic diffusion model Speckle noise is multiplicative and it can be modeled as (Krissian et. al. 04, 05): f = u + un, Corresponding time marching equation: u t u2 ( u ) f + u u u = λ u (f u). u/2 makes the diffusion faster in the lighter region (where the image values are high) and slower in the darker region (where the image values are low). unrealistic and ineffective in practice! u2 f +u

39 Image Denoising - AD Model Anisotropic diffusion model Speckle noise is multiplicative and it can be modeled as (Krissian et. al. 04, 05): f = u + un, Corresponding time marching equation: u t u2 ( u ) f + u u u = λ u (f u). u/2 makes the diffusion faster in the lighter region (where the image values are high) and slower in the darker region (where the image values are low). unrealistic and ineffective in practice! u2 f +u

40 Image Denoising - AD Model Anisotropic diffusion model Speckle noise is multiplicative and it can be modeled as (Krissian et. al. 04, 05): f = u + un, Corresponding time marching equation: u t u2 ( u ) f + u u u = λ u (f u). u/2 makes the diffusion faster in the lighter region (where the image values are high) and slower in the darker region (where the image values are low). unrealistic and ineffective in practice! u2 f +u

41 Image Denoising - AD Model Consider f = u + ( u u s ) n, where u s : smoothed version of the noised image f. New Anisotropic Diffusion (AD) Model u ( u ) t C u u s α u = β (f u), C > 0, 1/2 < α < 2 u On the regions where noise is present, u u s is relatively big. Diffusion is big enough to reduce the noise efficiently. On the regions where noise is not present, u u s is small. Diffusion is relatively slower.

42 Image Denoising - AD Model Consider f = u + ( u u s ) n, where u s : smoothed version of the noised image f. New Anisotropic Diffusion (AD) Model u ( u ) t C u u s α u = β (f u), C > 0, 1/2 < α < 2 u On the regions where noise is present, u u s is relatively big. Diffusion is big enough to reduce the noise efficiently. On the regions where noise is not present, u u s is small. Diffusion is relatively slower.

43 Image Denoising - AD Model Consider f = u + ( u u s ) n, where u s : smoothed version of the noised image f. New Anisotropic Diffusion (AD) Model u ( u ) t C u u s α u = β (f u), C > 0, 1/2 < α < 2 u On the regions where noise is present, u u s is relatively big. Diffusion is big enough to reduce the noise efficiently. On the regions where noise is not present, u u s is small. Diffusion is relatively slower.

44 Image Denoising - AD Model Consider f = u + ( u u s ) n, where u s : smoothed version of the noised image f. New Anisotropic Diffusion (AD) Model u ( u ) t C u u s α u = β (f u), C > 0, 1/2 < α < 2 u On the regions where noise is present, u u s is relatively big. Diffusion is big enough to reduce the noise efficiently. On the regions where noise is not present, u u s is small. Diffusion is relatively slower.

45 Numerical Procedure - AD Model 1 TFR (texture-free residual) parametrization 1 Set β as a constant: β(x, 0) = β 0. 2 For n = 2, 3, Compute the absolute residual and a quantity G n 1 Res : R n 1 = f u n 1, G n 1 Res = max 0, S m(r n 1 ) R n 1, where S m is a smoothing operator and R n 1 denotes the L 2 -average of R n 1. Update: β n = β n 1 + γ n G n 1 Res, where γ n is a scaling factor having the property: γ n 0 as n.

46 Numerical Experiments - AD Model Cuba Missile Crisis: The original (left above) and restored images by using ITV (right above), ITV-TFR (left below), and AD-TFR (right below) model

47 Medical Image Segmentation - MBS Motivation Medical images can involve noise, diverse artifacts, and unclear edges. Conventional segmentation methods show difficulties when applied to medical imagery. When an appropriate background is subtracted from the given image, the residue can be considered as an essentially binary image.

48 Medical Image Segmentation - MBS Procedure of method of background subtraction

49 Medical Image Segmentation - MBS Procedure of the construction of background 1 Select a coarse mesh {Ω ij } for the image domain Ω and choose a coarse image U c on {Ω ij }. Each element Ω ij in the coarse mesh corresponds to m x m y pixels. 2 Smooth U c. 3 Prolongate U c to the original mesh Ω, for U f. 4 Smooth U f. Assign the result for the background Ũ. Strategies for background construction In step I, choose U c on Ω ij as ra ij + (1 r)m ij, 0 r 1, a ij : arithmetic average, m ij : minimum of U 0 on Ω ij. Ũ must contain only background Information, not objects information. Thus select m = m x = m y such that number of blocks in U c corresponding to objects are smaller than the number of smoothing iterations in step II.

50 Medical Image Segmentation - MBS Procedure of the construction of background 1 Select a coarse mesh {Ω ij } for the image domain Ω and choose a coarse image U c on {Ω ij }. Each element Ω ij in the coarse mesh corresponds to m x m y pixels. 2 Smooth U c. 3 Prolongate U c to the original mesh Ω, for U f. 4 Smooth U f. Assign the result for the background Ũ. Strategies for background construction In step I, choose U c on Ω ij as ra ij + (1 r)m ij, 0 r 1, a ij : arithmetic average, m ij : minimum of U 0 on Ω ij. Ũ must contain only background Information, not objects information. Thus select m = m x = m y such that number of blocks in U c corresponding to objects are smaller than the number of smoothing iterations in step II.

51 Numerical Experiments - MBS Heart: Original (left), Conventional Method (middle), Conventional approach with MBS (right)

52 Numerical Experiments - MBS Hand: Original (left), Conventional Method (middle), Conventional approach with MBS (right)

53 Numerical Experiments - MBS Leukemia: Original (left), Conventional Method (middle), Conventional approach with MBS (right)

54 Conclusions 1 Regularization of u can be a significant source of nonphysical dissipation in TV-based models. 2 Non-convex (NC) diffusion model has been introduced in order to simultaneously suppress the noise and enhance edges. 3 Anisotropic diffusion (AD) model is more efficient on SAR image denoising than conventional models. 4 Numerical procedures of NC and AD models are stable. 5 The method of background subtraction (MBS) can be efficiently used as a pre-process of various segmentation methods for medical image segmentation.

55 Conclusions 1 Regularization of u can be a significant source of nonphysical dissipation in TV-based models. 2 Non-convex (NC) diffusion model has been introduced in order to simultaneously suppress the noise and enhance edges. 3 Anisotropic diffusion (AD) model is more efficient on SAR image denoising than conventional models. 4 Numerical procedures of NC and AD models are stable. 5 The method of background subtraction (MBS) can be efficiently used as a pre-process of various segmentation methods for medical image segmentation.

56 Conclusions 1 Regularization of u can be a significant source of nonphysical dissipation in TV-based models. 2 Non-convex (NC) diffusion model has been introduced in order to simultaneously suppress the noise and enhance edges. 3 Anisotropic diffusion (AD) model is more efficient on SAR image denoising than conventional models. 4 Numerical procedures of NC and AD models are stable. 5 The method of background subtraction (MBS) can be efficiently used as a pre-process of various segmentation methods for medical image segmentation.

57 Conclusions 1 Regularization of u can be a significant source of nonphysical dissipation in TV-based models. 2 Non-convex (NC) diffusion model has been introduced in order to simultaneously suppress the noise and enhance edges. 3 Anisotropic diffusion (AD) model is more efficient on SAR image denoising than conventional models. 4 Numerical procedures of NC and AD models are stable. 5 The method of background subtraction (MBS) can be efficiently used as a pre-process of various segmentation methods for medical image segmentation.

58 Conclusions 1 Regularization of u can be a significant source of nonphysical dissipation in TV-based models. 2 Non-convex (NC) diffusion model has been introduced in order to simultaneously suppress the noise and enhance edges. 3 Anisotropic diffusion (AD) model is more efficient on SAR image denoising than conventional models. 4 Numerical procedures of NC and AD models are stable. 5 The method of background subtraction (MBS) can be efficiently used as a pre-process of various segmentation methods for medical image segmentation.

59 Conclusions 1 Regularization of u can be a significant source of nonphysical dissipation in TV-based models. 2 Non-convex (NC) diffusion model has been introduced in order to simultaneously suppress the noise and enhance edges. 3 Anisotropic diffusion (AD) model is more efficient on SAR image denoising than conventional models. 4 Numerical procedures of NC and AD models are stable. 5 The method of background subtraction (MBS) can be efficiently used as a pre-process of various segmentation methods for medical image segmentation.