Filtering. Yao Wang Polytechnic University, Brooklyn, NY 11201

Transcription

1 Spatia Domain Linear Fitering Yao Wang Poytechnic University Brookyn NY With contribution rom Zhu Liu Onur Gueryuz and Gonzaez/Woods Digita Image Processing ed

2 Introduction Outine Noise remova using ow-pass iters Sharpening by edge enhancement Edge detection using high-pass iters Edge enhancement by high emphasis iters Edge detection First order gradient Second order gradient Summary Yao Wang NYU-Poy EL53: Spatia Fitering

3 Smoothing Noise remova Probems Detai preserving image smoothing Sharpening Edge enhancement Detai ocusing Edge detection Yao Wang NYU-Poy EL53: Spatia Fitering 3

4 Approaches Spatia domain operation or itering the processed vaue or the current pie depends on both itse and surrounding pies Linear itering Non-inear itering Rank order itering incuding median Morphoogica itering Adaptive itering Yao Wang NYU-Poy EL53: Spatia Fitering 4

5 Noise Remova Image Smoothing An image may be dirty with dots speckesstains Noise remova: To remove speckes/dots on an image Dots can be modeed as impuses sat-and-pepper or specke or continuousy varying Gaussian noise Can be removed by taking mean or median vaues o neighboring pies e.g. 33 window Equivaent to ow-pass itering Probem with ow-pass itering May bur edges More advanced techniques: adaptive edge preserving Yao Wang NYU-Poy EL53: Spatia Fitering 5

6 Eampe Yao Wang NYU-Poy EL53: Spatia Fitering 6

7 Averaging Fiter: An Intuitive Approach Repace each pie by the average o pies in a square window surrounding this pie g m n m n m n m n 9 m n m n m n m n m n Trade-o between noise remova and detai preserving: m n Larger window -> can remove noise more eectivey but aso bur the detais/edges Yao Wang NYU-Poy EL53: Spatia Fitering 7

8 Eampe: 33 average Yao Wang NYU-Poy EL53: Spatia Fitering 8

9 Eampe Yao Wang NYU-Poy EL53: Spatia Fitering 9

10 Weighted Averaging Fiter Instead o averaging a the pie vaues in the window give the coser-by pies higher weighting and ar-away pies ower weighting. g m n k kk h k m k n m n * h m n This type o operation is in act -D inear convoution o mn by a iter hmn. Weighted average iter retains ow requency and suppresses high requency = ow-pass iter Yao Wang NYU-Poy EL53: Spatia Fitering

11 Eampe Weighting Mask A weights must sum to one Yao Wang NYU-Poy EL53: Spatia Fitering

12 Eampe: Weighted Average Yao Wang NYU-Poy EL53: Spatia Fitering

13 Eampe Origina image Average itered image Weighted Average itered image Yao Wang NYU-Poy EL53: Spatia Fitering 3

14 Symmetrica Fiter The smoothing iter is generay symmetrica in both horizonta and vertica directions as we as diagona. Simpiied cacuation. n m h n m h n m h n m h k Hh k= -W * n m h n m n k m k h n m g k k k W -W W = W n m n m m n m n h m n h m n g W k k W k W n k m n k m n k m n k m k h n m n m m n m n h Yao Wang NYU-Poy EL53: Spatia Fitering 4 k

15 Computation Cost Genera W+ by W+ iter W+ mutipications per processed pie W W g m n h k m k n k W W Symmetrica iter +W+W ~ =W+ mutipications Saving W = and 55 image Saving 5*5*5-3 = More savings or arger W Yao Wang NYU-Poy EL53: Spatia Fitering 5

16 Common Smoothing Fiters b. ; 4 6 ; with b b b b b b b Are any o these separabe? b Are any o these separabe? Criteria or designing a smoothing iter hk unction as averaging hk unction as averaging the mean vaue is preserved k k k k h Yao Wang NYU-Poy EL53: Spatia Fitering 6

17 Smoothing Fiter is a Low-pass Fiter The D Fourier Transorm o a smoothing iter Image signa usuay varies sowy and noise is usuay a wide band signa. Appying a ow pass iter removes the high-requency h part o the noise. Image edges and other detais are burred Yao Wang NYU-Poy EL53: Spatia Fitering 7

18 Interpretation in Freq Domain Fiter response Low-passed image spectrum Origina image spectrum Noise spectrum Yao Wang NYU-Poy EL53: Spatia Fitering 8

19 Spectrum o Eampe Low Pass Fiterrs H 9 H u v ; 9 cosu cosv H b b b b b ; / 9 b H u v b b b b cosu b cosv / b b= Yao Wang NYU-Poy EL53: Spatia Fitering 9

20 Directiona Smoothing Smoothing iter tend to bur detais Chaenging probem: detai preserving noise remova! D i the i-th directiona neighborhood Usuay 4 or 8 directions 3 core steps D 3 Find σ i the variance in D i Find i so that σ i σ i Fitering in i direction g m n N io m k n k D io D g m n m n* h i m n h i m n a ep{ mcos i nsin i } Yao Wang NYU-Poy EL53: Spatia Fitering

21 Noise Remova by Averaging Mutipe Images Yao Wang NYU-Poy EL53: Spatia Fitering

22 Noise Remova by Averaging Mutipe Images Images Assume y = + n where n ~ N σ. Observe one image the noise is n Observe N images: y i = + n i i= N where n i ~ N σ and n i Observe N images: y i n i i N where n i N σ and n i are independent. N N N n n y y i i i i i i n N n N y N y / ~ N N n where n y Averaging over N images reduces the noise variance by /N. Yao Wang NYU-Poy EL53: Spatia Fitering

23 What is an edge? Edge Detection Dierence between een edge and ine and point Yao Wang NYU-Poy EL53: Spatia Fitering 3

24 Characterization o Edges Idea step edge Rea edge has a sope First order derivative: Maimum at edge ocation Second order derivative: Zero crossing at edge ocation Yao Wang NYU-Poy EL53: Spatia Fitering 4

25 Edge Detection Based on First Order Derivatives Edge High-pass itering across edge Low-pass itering aong edge h Non-Edge No H g g >T Yes Edge What i we don t know edge direction? Yao Wang NYU-Poy EL53: Spatia Fitering 5

26 Edge Detection Based on Gradients in Two Orthogona Directions Combine resuts rom directiona edge detectors in two orthogona directions and determine the magnitude and direction o the edge. Non-Edge H g + y g >T Edge H - y tan / y theta y Yao Wang NYU-Poy EL53: Spatia Fitering 6

27 Directiona Edge Detector High-pass or band-pass in one direction simuating irst order derivative Low pass in the orthogona direction smooth noise Prewitt edge detector LP BP 3 3 ; 3 3 y H H Sobe edge detector 4 4 ; 4 4 y H H Yao Wang NYU-Poy EL53: Spatia Fitering 7 The sobe iter provides better smoothing aong the edge

28 Freq. Response o Sobe Fiter Edges : Fiter or Horizonta Sobe H DTFT R F 4 4 H h u j e e H h v j v j u j u j ] [ sin - ] [- Frequency Response DTFT : v e e H h v j v j y y cos 4 - ] [ 4 Hu Hyv Yao Wang NYU-Poy EL53: Spatia Fitering 8 /4 / u /4 / v Band-pass Low-pass

29 Spectrum o the Sobe Fiter H H y Low pass aong the edge band pass cross the edge Yao Wang NYU-Poy EL53: Spatia Fitering 9

30 Eampe o Sobe Edge Detector Origina image Fitered image by H Fitered image by H y Yao Wang NYU-Poy EL53: Spatia Fitering 3

31 Tria and error How to set threshod? According to edge magnitude distributiontion E.g assuming ony 5% pies shoud be edge pies then the threshod shoud be the 95% percentie o the edge magnitude Iustrate on board Yao Wang NYU-Poy EL53: Spatia Fitering 3

32 g gy gm Histogram o gm T= T=5 T= Yao Wang NYU-Poy EL53: Spatia Fitering 3

33 Probems o previous approach Cannot ocate edges precisey Ramp edges can ead to many edge pies detected depending on the threshod T T too high: may not detect weak edges T too sma: detected edges too think noisy points asey detected Remedy: Detecting oca maimum o g in the norma direction o the edge or try a possibe 8 direction in a 33 neighbor Ony consider pies with g > T Yao Wang NYU-Poy EL53: Spatia Fitering 33

34 Edge Detection with Many Directiona Fiters Instead o using two orthogona directions can design mutipe directiona iters See which h one gives the highest h response in the norma direction Yao Wang NYU-Poy EL53: Spatia Fitering 34

35 Edge Detection Based on Second Order Derivative Convove an image with a iter corresponding to taking second order derivative e.g. Lapacian or LoG operator Locate zero-crossing in the itered image Yao Wang NYU-Poy EL53: Spatia Fitering 35

36 Lapacian Operator y y y y y y y y y 4 y y y y y y y y y ; 4 ; 4 Yao Wang NYU-Poy EL53: Spatia Fitering 36

37 Fourier Transorm o Lapacian Operator More isotropic Yao Wang NYU-Poy EL53: Spatia Fitering 37

38 Isotropic Edge Detector Lapacian operator can detect changes in a directions Yao Wang NYU-Poy EL53: Spatia Fitering 38

39 Pros and Cons Can ocate edges more accuratey Can detect edges in various direction But more prone to noise Remedy: Smoothing beore appying ppy Lapacian Yao Wang NYU-Poy EL53: Spatia Fitering 39

40 Lapacian o Gaussian LoG To suppress noise smooth the signa using a Gaussian iter irst Fy* Gy G Then appy Lapacian iter Fy*Gy*Ly = Fy* Ly*Gy Equivaent iter: LoG Hy=Ly*Gy Yao Wang NYU-Poy EL53: Spatia Fitering 4

41 Derivation o LoG Fiter Anaog orm y y y y G G e y G 4 e y y G G y G Take sampes to create iter mask Si k > 6 dd Size o mask nn n>=6 odd En Yao Wang NYU-Poy EL53: Spatia Fitering 4

42 LoG Fiter Yao Wang NYU-Poy EL53: Spatia Fitering 4

43 Lapacian itered LOG itered Note that each strong edge in the origina image corresponds to a thin stripe with high intensity in one side and ow intensity in the other side. Yao Wang NYU-Poy EL53: Spatia Fitering 43

44 How to detect zero crossing? For each pie that has ow itered vaue check a 33 neighbor to see whether its two neighbors in opposite direction have opposite sign and their dierence eceeds a threshod Marr-Hidreth edge detection method Yao Wang NYU-Poy EL53: Spatia Fitering 44

45 Eampe Yao Wang NYU-Poy EL53: Spatia Fitering 45

46 Summary o Edge Detection Method First order gradient based: Using edge detectors in two orthogona directions For each direction: ow-pass aong edge high-pass across edge Using edge detectors in mutipe > directions Use threshod or detect oca maimum across the edge direction Second order gradient based Lapacian is noise-prone LoG is better Detect zero crossing Isotropic Yao Wang NYU-Poy EL53: Spatia Fitering 46

47 More on Edge Detection Methods discussed so ar generay cannot yied connected thin edge maps Need sophisticated post processing Thinning Connecting broken ines Noise can ead to many ase edge points Even with many years o research no perect edge detectors t eist! Canny edge detector: Gaussian smoothing aong edges high pass in each possibe edge direction For more on edge detection See Gonzaez Sec... Yao Wang NYU-Poy EL53: Spatia Fitering 47

48 Resuts using MATLAB edge unction SobeT=.4 LOG T=.5 cannyt=[.33.78] SobeT=. LOG T=. cannyt=[..5] Yao Wang NYU-Poy EL53: Spatia Fitering 48

49 Image Sharpening Sharpening : to enhance ine structures or other detais in an image Enhanced image = origina image + scaed version o the ine structures and edges in the image Line structures and edges can be obtained by appying a high pass iter on the image Combined operation is sti a weighted averaging operation but some weights can be negative. In requency domain the iter has the highemphasis character Yao Wang NYU-Poy EL53: Spatia Fitering 49

50 Interpretation in Freq Domain Fiter response high emphasis sharpened image spectrum Origina image spectrum Yao Wang NYU-Poy EL53: Spatia Fitering 5

51 Designing Sharpening Fiter Using High Pass Fiters The desired image is the origina pus an appropriatey scaed high-passed image Sharpening iter s h h s m n m n h m n h Yao Wang NYU-Poy EL53: Spatia Fitering 5

52 Interpretation in Freq Domain high emphasis=apass+highpass a pass high pass Yao Wang NYU-Poy EL53: Spatia Fitering 5

53 How to design high pass iter High pass iter tries to detect changes Dierence between a current pie and its neighbors Some coeicients shoud be negative Output shoud be zero i the image is at k k k h k More rigorous design woud start with desired requency response and take inverse FT. Yao Wang NYU-Poy EL53: Spatia Fitering 53

54 Using Lapacian Operator as Highpass Fiter More isotropic Yao Wang NYU-Poy EL53: Spatia Fitering 54

55 Eampe s =+ag g=*h h with H H s h with H H s h Yao Wang NYU-Poy EL53: Spatia Fitering

56 Eampe o Sharpening Using Lapacian Operator H h 4 8 Yao Wang NYU-Poy EL53: Spatia Fitering 56

57 Eampe o Sharpening Using Lapacian Operator 4 8 Yao Wang NYU-Poy EL53: Spatia Fitering 57

58 Designing Sharpening Fitering Using Low Pass Fiters Unsharp Masking The sharpened = origina a*owpass Sharpening iter h s s m n h m m n n Yao Wang NYU-Poy EL53: Spatia Fitering 58

59 Interpretation in Freq Domain high emphasis=apass- a*owpass a pass ow pass Yao Wang NYU-Poy EL53: Spatia Fitering 59

60 Eampe =*h s = with H H s Criteria or designing sharpening iters: There are both positive and negative Coeicients h > and hk = to keep the same mean vaue Yao Wang NYU-Poy EL53: Spatia Fitering 6 Coeicients h > and hk to keep the same mean vaue.

61 Eampe Origina image Sharpened image Yao Wang NYU-Poy EL53: Spatia Fitering 6

62 Chaenges o Noise Remova and Image Sharpening How to smooth the noise without burring the detais too much? How to enhance edges without ampiying noise? Sti a active research area Waveet domain processing Yao Wang NYU-Poy EL53: Spatia Fitering 6

63 Waveet-Domain Fitering Courtesy o Ivan Seesnick Yao Wang NYU-Poy EL53: Spatia Fitering 63

64 Feature Enhancement by Subtraction Taking an image without injecting a contrast agent irst. Then take the image again ater the organ is injected some specia contrast agent which go into the boodstreams ony. Then subtract the two images --- A popuar technique in medica imaging Yao Wang NYU-Poy EL53: Spatia Fitering 64

65 Summary Noise remova using ow-pass iters Averaging and weighted averaging iter Edge detection Using isotropic high-pass iters: eg. D Lapacian Using two orthogona high-pass iters: eg. Sobe Using many directiona high-pass iters Image sharpening by high emphasis iters High pass itering using Lapacian operators Sharpening using high pass iter ow pass+ a* high pass Sharpening using ow pass iter unsharp maskingoriginaa*owpass Given a iter you shoud be abe to te what does it do smoothing edge detection sharpening? by ooking at its coeicients and aso through DTFT Yao Wang NYU-Poy EL53: Spatia Fitering 65

66 Reading R. Gonzaez Digita Image Processing Section34~ Note that in Gonzaez book: Edge detection itering are reerred to sharpening. A. K. Jain Fundamentas o Digita Image Processing Section 7.4ecept median iter. Yao Wang NYU-Poy EL53: Spatia Fitering 66

67 Written Assignment. For each iter given beow compute its Fourier transorm and iustrate its magnitude response. Determine what is its unction smoothing edge enhancement or edge detection? based on the iter coeicients as we as its requency response. For each iter determine whether it is separabe? I yes compute the FT separatey and epain the unction o each whether it is separabe? I yes compute the FT separatey and epain the unction o each D iter. I not compute the FT directy. 4 6 H ; 8 H 5 H 3 Yao Wang NYU-Poy EL53: Spatia Fitering 67

68 Written Assignment cnt d. Reca that the Lapacian o Gaussian iter is the Lapacian o a Gaussian unction. Let the Gaussian unction be given by y G y e show that the LOG iter can be written as y L y e 4 3. Deine the a discrete Lapacian o Gaussian LoG iter o size 77 by samping the above continuous LoG unction assuming =. Determine its discrete space Fourier transorm. Iustrate its magnitude response. Epain its unction. You can use the reqz unction to determine and iustrate its DTFT. y Yao Wang NYU-Poy EL53: Spatia Fitering 68

69 Computer Assignment. Write a Matab or C-program to simuate noise remova. First create a noisy image by adding zero mean Gaussian random noise to your image In matab the noise can be generated using imnoise. You can speciy the noise variance in imnoise. Then appy an averaging iter to the noise added image. For a chosen variance o the added noise you need to try dierent window sizes rom 33 to 77 to see which one gives you the best trade-o between noise remova and burring. Hand in your program the origina noise-added images at two dierent noise eves. and. and the corresponding itered images with the best window sizes. Write down your observation. For the itering operation don t use the matab conv unction. Rather write your own unction or the averaging operation. Your program shoud aow the user to speciy the window size as an input parameter.. Write a Matab or C-program or impementing the oowing simpe edge detection agorithm: For every pie: i ind the horizonta and vertica gradients g and g y using the Sobe operator; ii cacuate the magnitude o the gradient using gm g g y ; and iii For a chosen threshod T consider the pie to be an edge pie i g m T. Save the resuting edge map Use a gray eve o 55 to indicate an edge pie and a gray eve o or a non-edge pie. Appy this program to your test image and observe the resuting edge map with dierent T vaues unti you ind a T that gives you a good resut. Hand in your program and the edge maps generated by two dierent threshod vaues. Write down your observation. Hint: one automatic way to determine T is by sorting the pies based on the gradient magnitudes and choose T such that a certain percentage say 5% o pies wi be edge pies. You can use either the matab conv unction or write your own code or the itering part. Yao Wang NYU-Poy EL53: Spatia Fitering 69

70 3. Using the matab edge unction evauate the edge detection resuts using the oowing methods on an image: sobe og and canny. For each method irst use the deaut threshod vaue. Then try to use a threshod that is higher and ower to see the eect. Comments on the advantages and disadvantages o each method. For probems and 3 try to use an image that has a ot o edges. A good set o test images can be ound under the matab directory./toobo/images/imdemos Yao Wang NYU-Poy EL53: Spatia Fitering 7