Digital Image Processing Chapter 5: Image Restoration

Transcription

1 Digital Image Processing Chapter 5: Image Restoration

2 Concept of Image Restoration Image restoration is to restore a degraded image back to the original image while image enhancement is to maniplate the image so that it is sitable for a specific application. Degradation model: g x y f x y h x y + η x y where hxy is a system that cases image distortion and ηxy is noise. Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

3 y x y x h y x f y x g η + Concept of Image Restoration Concept of Image Restoration y x y x h y x f y x g η + We can perform the same operations in the freqency domain where conoltion is replaced by mltiplication domain where conoltion is replaced by mltiplication and addition remains as addition becase of the linearity of the Forier transform. N H F G + If we knew the ales of H and N we cold recoer F by writing the aboe eqation as N G F H G F Howeer as we shall see this approach may not be practical. Een thogh we may hae some statistical information abot the noise we will not know the ale of nxy or N for all or l A ll di idi b Hi j ill diff lti een any ales. As well diiding by Hij will case difflties if there are ales which are close to or eqal to zero.

4 Noise We may define noise to be any degradation d in the image signal cased by external distrbance. If an image is being sent electronically from one place to another ia satellite or wireless transmission or throgh networked cable we may expect errors to occr in the image signal. These errors will appear on the image otpt in different ways depending on the type of distrbance in the signal. Usally we know what type of errors to expect and hence the type of noise on the image; hence we can choose the most appropriate method for redcing the effects. Cleaning an image corrpted by noise is ths an important area of image restoration. We will inestigate some of the standard noise forms and the different methods of eliminating i or redcing their effects on the image.

5 Salt and pepper noise Also called implse noise shot noise or binary noise. This degradation d can be cased by sharp sdden distrbances in the image signal; its appearance is randomly scattered white or black or both pixels oer the image. To add noise we se the Matlab fnction imnoise which takes a nmber of different parameters. To add salt and pepper noise: >> cameramanimread'cameraman.tif'; >> figre;imshowcameraman; >> cameraman_spimnoisecameraman'salt & pepper'; >> figre;imshowcameraman_sp; to add more or less noise we inclde an optional parameter being a ale between 0 and 1 indicating the fraction of pixels to be corrpted. Thsfor example >> cameraman_sp0imnoisecameraman'salt & pepper'0.; >> figre;imshowcameraman_sp0; i

6 Gassian noise Gassian noise is an idealized d form of white noise which h is cased by random flctations in the signal. We can obsere white noise by watching a teleision which is slightly mistned to a particlar channel. Gassian noise is white noise which is normally distribted. If the image is represented as I and the Gassian noise by N then we can model a noisy image by simply adding the two: I + N >> cameraman_gnimnoisecameraman gassian'; i i >> figre;imshowcameraman_gn; The gassian parameter also can take optional ales giing the mean and ariance of the noise. The defalt ales are 0 and 0.1

7 Speckle noise Whereas Gassian noise can be modelled dlldby random ales added ddd to an image; speckle noise or more simply jst speckle can be modelled by random ales mltiplied by pixel ales hence it is also called mltiplicatie noise. Speckle noise is a major problem in some radar applications. As aboe imnoise can do speckle: >> cameraman_spkimnoisecameraman'speckle'; >> figre;imshowcameraman_spk; In Matlab speckle noise is implemented as I1 + N Althogh Gassian noise and speckle noise appear sperficially similar they are formed by two totally different methods and as we shall see reqire different approaches for their remoal.

8 Periodic noise If the image signal is sbject to a periodic rather than a random distrbance we might obtain an image corrpted by periodic noise. The effect is of bars oer the image. The fnction imnoise does not hae a periodic option bt it is qite easy to create by adding a periodic matrix sing a trigonometric fnction to an image: >> ssizecameraman; >> [xy]meshgrid1:s11:s; 1:s; >> psinx/3+y/5+1; >> cameraman_pnimdoblecameraman+p//; >> figre;imshowcameraman_pn i Salt and pepper noise Gassian noise and speckle noise can all be cleaned by sing spatial filtering techniqes. Periodic noise howeer reqires the se of freqency domain filtering. This is becase whereas the other forms of noise can be modelled dlldas local ldegradations dti periodic idi noise is a global effect.

9 Noise Models Ni Noise cannot tbe predicted ditdbt bt can be approximately tl described ibdin statistical way sing the probability density fnction PDF Gassian noise: p z 1 e πσ z μ / σ Rayleigh noise p z z b 0 a e z a / b for z for z a < a Erlang Gamma noise p z b b 1 a z az z a e for z 0 b 1! 0 for z < 0

10 Noise Models cont. Exponential noise Uniform noise p z ae az p z 1 b - a 0 for a z otherwise b Implse salt lt& pepper noise Pa for z a p z Pb for z b 0 otherwise

11 PDF: Statistical Way to Describe Noise PDF tells how mch each z ale occrs. Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

12 Image Degradation with Additie Noise g x y f x y + η x y Degraded images Original image Histogram Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

13 Image Degradation with Additie Noise cont. g x y f x y + η x y Degraded images Original image Histogram Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

14 Estimation of Noise We cannot se the image histogram to estimate noise PDF. It is better to se the histogram of one area of an image that has constant intensity to estimate noise PDF. Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

15 Periodic Noise Periodic noise looks like dots In the freqency domain Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

16 Periodic Noise

17 Periodic Noise

18 Periodic Noise Redction by Freq. Domain Filtering Degraded image DFT Periodic noise can be redced by setting freqency components corresponding to noise to zero. Band reject filter Restored image Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

19 Band Reject Filters Use to eliminate i freqency components in some bands Periodic noise from the preios slide that is Filtered ot. Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

20 Notch Reject Filters A notch reject filter is sed to eliminate i some freqency components. Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

21 Notch Reject Filter: Notch filter Degraded image DFT freq. Domain Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition. Noise Restored image

22 Example: Image Degraded by Periodic Noise Degraded image DFT no shift DFT of noise Noise Restored image Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

23 Mean Filters Mean Filters Degradation model: g y x y x h y x f y x g η + To remoe this part Arithmetic mean filter or moing aerage filter from Chapter 3 1 S xy t s t s g mn y x f 1 ˆ y Geometric mean filter mn size of moing window mn t s g y x f 1 ˆ S t s xy t s g y x f

24 Geometric Mean Filter: Example Original image Image corrpted by AWGN Image obtained sing a 3x3 arithmetic mean filter Image obtained sing a 3x3 geometric ti mean filter AWGN: Additie White Gassian Noise Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

25 Harmonic and Contraharmonic Filters Harmonic mean filter mn fˆ x y 1 Works well llfor salt noise bt fails for pepper noise g s t s t S xy Contraharmonic mean filter mn size of moing window fˆ g s t Q+ 1 Positie Q is sitable for s t Sxy x y eliminating i pepper noise. Q Q the filter order s t S xy g s t Negatie Q is sitable for eliminating salt noise. For Q 0 the filter redces to an arithmetic mean filter. For Q -1 the filter redces to a harmonic mean filter.

26 Contraharmonic Filters: Example Image Image corrpted corrpted by pepper by salt noise with noise with prob prob. 0.1 Image obtained sing a 3x3 contra- harmonic mean filter With Q 1.5 Image obtained sing a 3x3 contra- harmonic mean filter With Q-1.5 Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

27 Contraharmonic Filters: Incorrect Use Example Image Image corrpted corrpted by pepper by salt noise with noise with prob prob. 0.1 Image obtained sing a 3x3 contra- harmonic mean filter With Q-1.5 Image obtained sing a 3x3 contra- harmonic mean filter With Q1.5 Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

28 Order-Statistic Filters: Reisit Original image sbimage Statistic parameters Mean Median Mode Min Max Etc. Moing window Otpt image

29 Order-Statistics Filters Median filter ˆ f x y { g s t } median s t S xy Max filter Min filter fˆ x fˆ x y y { g s t } max Redce dark noise s t S xy pepper noise { } Rd bih min g s t Redce bright noise salt noise s t S xy Midpoint filter 1 ˆf x y { g s t } + { g s t } } f max s t S min s t xy S xy

30 Median Filter : How it works A median filter is good for remoing implse isolated noise Pepper noise Median Salt noise Degraded image Salt noise Pepper noise Moing window Sorted array Filter otpt Normally implse noise has high magnitde and is isolated. When we sort pixels in the moing window noise pixels are sally at the ends of the array. Therefore it s rare that the noise pixel will be a median ale.

31 Median Filter : Example 1 Image corrpted by saltand-pepper noise with p a p b Images obtained sing a 3x3 median Images filter from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

32 Max and Min Filters: Example Image Image corrpted corrpted by pepper by salt noise with noise with prob. 0.1 prob. 0.1 Image obtained sing a 3x3 max filter Image obtained sing a 3x3 min filter Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

33 Alpha-trimmed Mean Filter Formla: fˆ x y 1 mn d s t g S xy r s t where g r st represent the remaining mn-d pixels after remoing the d/ highest and d/ lowest ales of gst. Lowest Gray Leel Total mn pixels d/ d/ Highest Gray Leel This filter is sefl in sitations inoling mltiple types of noise sch as a combination of salt-and-pepperand and Gassian noise.

34 Alpha-trimmed Mean Filter: Example Image additionally corrpted by additie salt-and- pepper noise Image corrpted by additie niform noise 1 Image Image obtained obtained sing a 5x5 sing a 5x5 arithmetic ti geometric mean filter mean filter Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

35 Alpha-trimmed Mean Filter: Example cont. 1 Image additionally corrpted by additie salt-and- pepper noise Image corrpted by additie niform noise Image obtained sing a 5x5 median filter Image obtained sing a 5x5 alpha- trimmed mean filter with d 5

36 Alpha-trimmed Mean Filter: Example cont. Image obtained sing a 55 5x5 arithmetic mean filter Image obtained sing a 55 5x5 geometric mean filter Image obtained sing a 5x5 median filter Image obtained sing a 5x5 alpha- trimmed mean filter with d 5

37 Adaptie Filter General concept: -Filter behaior depends on statistical characteristics of local areas inside id mxn moing window - More complex bt sperior performance compared with fixed filters Statistical characteristics: Local mean: Local ariance: 1 m L g s t mn σ L s t S xy 1 g s t m L mn s t S xy Noise ariance: σ η

38 Adaptie Local Noise Redction Filter Prpose: want to presere edges Concept: 1. If σ η is zero No noise the filter shold retrn gxy becase gxy fxy. If σ L is high relatie to σ η Edges shold be presered the filter shold retrn the ale close to gxy 3. If σ L σ η Areas inside objects the filter shold retrn the arithmetic mean ale m L Formla: σ η η fˆ x y g x y σ L g x y m L

39 Adaptie Noise Redction Filter: Example Image corrpted by additie Gassian noise with zero mean and σ 1000 Image obtained sing a 7x7 geometric mean filter Image obtained sing a 7x7 arithmetic mean filter Image obtained sing a7 7x77 adaptie noise redction filter Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

40 Adaptie Median Filter Prpose: want to remoe implse noise while presering edges Algorithm:Leel A: Leel B: A1 z median z min A z median z max If A1 > 0 and A < 0 goto leel B Else increase window size If window size < S max repeat leel A Else retrn z xy B1 z xy z min B z xy z max If B1 > 0 and B < 0 retrn z xy where Else retrn z median z min minimm gray leel ale in S xy z max maximm gray leel ale in S xy z median median of gray leels in S xy z xy gray leel ale at pixel xy S max maximm allowed size of S xy

41 Adaptie Median Filter: How it works Leel la: A1 z median z min Determine A z median z max whether z median If A1 > 0 and A < 0 goto leel l B is an implse or not Else Window is not big enogh increase window size If window size < S max repeat leel A Else retrn z xy Leel B: z median is not an implse Determine B1 z xy z min whether z xy B z xy z max is an implse or not If B1 > 0 and B < 0 z xy is not an implse retrn z xy to presere original details Else retrn z median to remoe implse

42 Adaptie Median Filter: Example Image corrpted by salt-and-pepper noise with p a p b 0.5 Image obtained sing a 7x7 median filter Image obtained sing an adaptie median filter with S max 7 More small details are presered Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

43 Estimation of Degradation Model Degradation model: or g x y f x y h x y + η x y G F H + N Prpose: to estimate hxy or H Why? If we know exactly hxy regardless of noise we can do deconoltion to get fxy back from gxy. Methods: 1. Estimation by Image Obseration. Estimation by Experiment 3. Estimation by Modeling Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

44 Estimation by Image Obseration Original image nknown Degraded image fxy fxy*hxy gxy Obseration Sbimage Estimated Transfer DFT G s g s x y fnction G Restoration s H H process pocessby s F ˆ estimation s DFT This case is sed when we Reconstrcted Fˆ know only gxy and cannot s Sbimage repeat the experiment! ˆ x y f s

45 Estimation by Experiment Used when we hae the same eqipment set p and can repeat the experiment. Response image from Inpt implse image the system System H Aδ x y DFT { Aδ x y } A DFT H G A g x y G DFT Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

46 Estimation by Modeling Used when we know physical mechanism nderlying the image formation process that can be expressed mathematically. Oi Original i limage Seere trblence Example: Atmospheric Trblence model Mild trblence k Low trblence H e k + 5/ 6 k k Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

47 Estimation by Modeling: Motion Blrring Assme that camera elocity is x0 t y0 t The blrred image is obtained by where T exposre time. T g x y f x + x0 t y + y0 0 t dt G g x T T 0 0 f f y e x + x + j π x + y dd dxdy x0 t y + y0 t dt e x0 t y + y0 t e jπ x+ y jπ x+ y dxdy dxdy dt

48 Estimation by Modeling: Motion Blrring cont. G T 0 T 0 f x + x0 t y + y0 t e [ jπ x0 t + y0 t F e ]dt F T 0 e j π x t + y 0 0 t dt jπ x+ y Then we get the motion blrring transfer fnction: H T 0 e j π x t + y 0 0 t For constant motion x 0 t y0 t at bt H T e 0 dt dt T π a + sin i π a b dxdy dt + b e j π a + b j π a + b

49 Motion Blrring Example For constant motion H T π a + b sin π a + b e jπ a+ b Original image Motion blrred image a b T 1 Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

50 Motion Blr in Matlab >> fcheckerboard8; >> PSFfspecial'motion'745; >> PSF PSF >> gb imfilterfpsf 'circlar'; >> imshowf;figre;imshowgb; Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

51 Inerse Filter From degradation model: G F H + N after we obtain H we can estimate F by the inerse filter: ˆ G F F + H N H Noise is enhanced when H is small. In practical the inerse filter is not Poplarly sed. To aoid the side effect of enhancing noise we can apply this formlation to freq. component with in a radis D 0 from the center of H. Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

52 Inerse Filter >> wimread'cameraman.tif';; if' >> wffftshiftfftw; >> blbtter565615; ; >> wbwf.*b; % image degraded >> wbaabsifftfftshiftwb; >> wbaint855*matgraywba; b >> figre;imshowwba; % recoer >> w1fftshiftfftwba./b; >> w1aabsifftfftshiftw1; >> figre;imshowmatgrayw1a; i Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

53 Inerse Filter: Example Original image Reslt of applying the fll filter Reslt of applying the filter with D 0 40 Blrred image Reslt of applying Reslt of applying De to Trblence the filter with D 0 70 the filter with D 0 85 H e / 6

54 Wiener Filter: Minimm Mean Sqare Error Filter Wiener Filter: Minimm Mean Sqare Error Filter Objectie: optimize mean sqare error: { } ˆ f f E Objectie: optimize mean sqare error: { } f f E e Wiener Filter Formla: ˆ * G S H S S H F f f + η / * G S S H H f η 1 / G H S S H f + η / 1 G S S H H f + η where where H Degradation fnction S Power spectrm of noise Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition. S η Power spectrm of noise S f Power spectrm of the ndegraded image

55 Approximation of Wiener Filter Approximation of Wiener Filter Wiener Filter Formla: 1 ˆ G H F Wiener Filter Formla: / S S H H f + η Approximated Formla: Difficlt to estimate 1 ˆ G K H H H F + K H H + Practically K is chosen manally to obtained the best isal reslt!

56 Wiener Filter: Example Original image Reslt of the fll inerse filter Reslt of the inerse filter with D Blrred image De to Trblence Reslt of the fll Wiener filter

57 Wiener Filter: Example cont. Reslt of the inerse Original image filter with D 070 Blrred image Reslt of the De to Trblence Wiener filter

58 Example: Wiener Filter and Motion Blrring Image Reslt of the Reslt of the degraded inerse filter Wiener filter by motion blr + AWGN σ η η 650 σ η 35 Note: K is chosen manally σ η 130 Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

59 Degradation model: W itt i t i f Constrained Least Sqares Filter Constrained Least Sqares Filter Degradation model: y x y x h y x f y x g η + Written in a matrix form η Hf g + Objectie: to find the minimm of a criterion fnction [ ] 1 1 M N y x f C [ ] 0 0 x y y x f C Sbject to the constraint ˆ η Hf g We get a constrained least sqare filter where w w w T ˆ * G P H H F We get a constrained least sqare filter P H + γ where P Forier transform of pxy

60 Constrained Least Sqares Filter: Example Constrained least sqare filter * ˆ H F G H P + γ γ is adaptiely adjsted to achiee the best reslt. Reslts from the preios slide obtained from the constrained least sqare filter

61 Constrained Least Sqares Filter: Example cont. Image Reslt of the Reslt of the degraded Constrained Wiener filter by motion blr + Least sqare filter AWGN σ η η 650 σ η 35 σ η 130

62 Constrained Least Sqares Filter:Adjsting γ Define r g Hf ˆ It can be shown that φ γ r T r r We want to adjst gamma so that 1. Specify an initial ale of γ r η ± a 1 where a accracy factor. Compte r 3. Stop if 1 is satisfied Otherwise retrn step after increasing γ if r < η a or decreasing γ if r Use the new ale of γ to recompte * ˆ H F G H + γ P > η + a

63 Constrained Least Sqares Filter:Adjsting Constrained Least Sqares Filter:Adjsting γ cont. cont. * ˆ * G P H H F + γ ˆ F H G R r For compting M x N y y x r MN r y M N y x m η M N 0 0 x y y x MN m η η η For compting [ ] x y m y x MN η η η σ η For compting [ ] η η σ m MN η

64 Constrained Least Sqares Filter: Example Oi Original i limage Use correct noise parameters Correct parameters: Initial γ 10-5 Correction factor 10-6 a 0.5 σ η η 10-5 Blrred image De to Trblence Use wrong noise parameters Wrong noise parameter σ - η 10 Reslts obtained from constrained least sqare filters Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

65 Geometric Mean filter Geometric Mean filter This filter represents a family of filters combined into a 1 α This filter represents a family of filters combined into a single expression ˆ 1 * * G H H F α α G S S H H F f η β + S f α 1 the inerse filter α 1 the inerse filter α 0 the Parametric Wiener filter α 0 β 1 the standard Wiener filter β 1 α < 0.5 More like the inerse filter β 1 α > 0.5 More like the Wiener filter Another name: the spectrm eqalization filter

66 Geometric Transformation These transformations ti are often called rbber-sheet transformations: ti Printing an image on a rbber sheet and then stretch this sheet according to some predefine set of rles. A geometric transformation consists of basic operations: 1. A spatial transformation : Define how pixels are to be rearranged in the spatially transformed image.. Gray leel interpolation : Assign gray leel ales to pixels in the spatially transformed image.

67 Spatial Transformation

68 Spatial Transformation One of the most commonly sed forms of spatial transformations is affine transform which can be written. This transfomation can scale rotate translate or shear a set of points depending di on the ales chosen for the elements of T which can be create in Matlab :

69 Spatial Transformation

70 Spatial Transformation

71 Image Registration Image registration i method seek to align two images of the same scene. For example it may be of interest to align two ore more images taken at roghly the same time bt sing different instrments sch an MRI scan and a PET scan.

72 Image Registration

73 Geometric Transformation : Algorithm x y x y Image f to be restored 1 3 Distorted image g 1. Select coordinate xy in f to be restored. Compte x r x y 4. get pixel ale at g x y By gray leel interpolation y s x y 3. Go to pixel x y 5. store that ale in pixel fxy in a distorted image g 5

74 Spatial Transformation To map between pixel coordinate xy of f and pixel coordinate x y of g x r x y y s x y For a bilinear transformation mapping between a pair of Qadrilateral regions x r x y c x + c y + c xy + c y s x y c x + c y + c xy + c x y x y To obtain rxy and sxy we need to know 4 pairs of coordinates x y and its corresponding x y which are called tiepoints. Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

75 Gray Leel Interpolation: Nearest Neighbor Since x y may not be at an integer coordinate we need to Interpolate the ale of g x y Example interpolation methods that can be sed: 1. Nearest neighbor selection. Bilinear interpolation 3. Bicbic interpolation Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

76 Gray Leel Interpolation: Bilinear interpolation

77 Geometric Distortion and Restoration Example Original image and tiepoints Tiepoints of distorted image Distorted image Restored image Use nearest neighbor intepolation Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

78 Geometric Distortion and Restoration Example cont. Original image and tiepoints Tiepoints of distorted image Distorted image Restored image Use bilinear it intepolation lti Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.

79 Example: Geometric Restoration Original image Geometrically distorted Image Use the same Spatial Trans Bilinear. as in the preios example Restored image Difference between aboe images Images from Rafael C. Gonzalez and Richard E. Wood Digital Image Processing nd Edition.