EECS 556 Image Processing W 09 Image enhancement Smoothing and noise removal Sharpening filters
What is image processing? Image processing is the application of 2D signal processing methods to images Image representation and modeling Image enhancement Image restoration/image filtering Compression (computer vision)
Systems for 2D signals Image processing systems enhanced version of f Continuous Zoom lenses optical color filters photographic film analog photocopiers
Systems for 2D signals Continuous Zoom lenses optical color filters photographic film analog photocopiers Discrete scanner / digitizer, contrast enhancer, denoiser, image compressor edge detector
Image enhancement Accentuate certain desired features for subsequent analysis or display. Contrast stretching / dynamic range adjustment Color histogram normalization Noise reduction Sharpening Image interpolation / motion estimation Edge detection
Contrast stretched image
Histogram equalization
Edge detection
Image enhancement Accentuate certain desired features for subsequent analysis or display. Contrast stretching / dynamic range adjustment Color histogram normalization Noise reduction Sharpening Image interpolation / motion estimation Edge detection Tools: Filters 2D Fourier
LSI (linear shift invariant) systems An LSI system is completely specified by its impulse response. Discrete linear convolution
Non linear operations Piecewise linear amplitude adjustment Histogram based transformations Edge detection methods
Image enhancement Notice: Objective goal is subjective and function of the human visual systems properties
Image enhancement in Matlab edge - Find edges in intensity image. histeq - Enhance contrast using histogram equalization. imadjust - Adjust image intensity values or colormap. imnoise - Add noise to an image. medfilt2 - Perform 2 D median filtering. ordfilt2 - Perform 2 D order statistic filtering. wiener2 - Perform 2 D adaptive noise removal filtering.
Point and local processing Operations on single pixel at a time Operations on groups of pixels (neighborhoods) Pixel 4 Neighbors 8 Neighbors
Simplest form of processing: Point Processing Pixel T r = f ( m, n) s = T (r) 255 S=T(r) No change Thresholding Image negative : s=l r 0 r 255 Black 7
Simplest form of processing: Point Processing Pixel T r = f ( m, n) s = T (r) 255 Common Examples: γ = 0. Dynamic Range Compression T ( r) = c log( + r) s Gamma Correction γ = 0 0 r 255 Black T ( r) = γ cr Narrow range of dark gets mapped to broad range of gray 8
Gamma Correction: To compensate for nonlinearities in I/O devices like photographic film, scanners, printers, monitors, and the eye 9
Histogram Processing Distribution of gray levels can be judged by measuring a Histogram 5 7 9 Histogram: 5 Graylevel 20
Example: 2
Transform histogram to desired histogram perceptually more pleasing 22
we will come back to this 23
Local processing Operations on groups of pixels (neighborhoods) Filtering Smoothing Noise smoothing Sharpening
Box (moving average) filter Kernel k with positive entries, that sum to. Notice: all weights are equal h[, ] smoothing Slide credit: David Lowe (UBC)
Box filter 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 30 30 30 20 0 0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20 0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30 0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30 0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30 0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20 0 0 0 0 0 0 0 0 0 0 0 20 30 30 30 30 20 0 0 0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Example: Smoothing with a box filter
Smoothing with a Gaussian Weight contributions of neighboring pixels by nearness 0.003 0.03 0.022 0.03 0.003 0.03 0.059 0.097 0.059 0.03 0.022 0.097 0.59 0.097 0.022 0.03 0.059 0.097 0.059 0.03 0.003 0.03 0.022 0.03 0.003 5 x 5, σ = Constant factor at front makes volume sum to (can be ignored, as we should normalize weights to sum to in any case). Slide credit: Christopher Rasmussen
Smoothing with a Gaussian Rule of thumb: set filter half width to about 3 σ
Smoothing with a Gaussian
Why are they appealing? Remove high frequency components from the image (low pass filter) Convolution with self is another Gaussian Convolving two times with Gaussian kernel of width σ is same as convolving once with kernel of width sqrt(2) σ Separable kernel Factors into product of two D Gaussians Useful: can convolve all rows, then all columns Source: K. Grauman
Separability of the Gaussian filter Source: D. Lowe
Noise smoothing Often the power spectrum of an image hasless energy at high spatial frequencies. In contrast, noise power spectra are often flat (e.g., white noise) or approximately flat. SNR is poor for high spatial frequencies. Common to use a low pass filter to remove poor SNR components and preserve the good SNR components at low spatial frequencies.
Gaussian noise Noise: sum of many independent factors, with zero mean modeled by a gaussian:
Noise smoothing by gaussian filtering Smoothing reduces pixel noise: Each row shows smoothing with Gaussians of different width; each column shows different amounts of Gaussian noise.
Noise smoothing by Median filtering A median filter operates over a window by selecting the median intensity in the window Is median filtering linear? Source: K. Grauman
Median filter What advantage does median filtering have over Gaussian filtering? Robustness to outliers Remove salt & pepper noise Source: K. Grauman
Filter salt & pepper noise Salt-and-pepper noise Median filtered MATLAB: medfilt2(image, [h w]) Source: K. Grauman
Median vs. Gaussian filtering 3x3 5x5 7x7 Gaussian Median
Sharpening sharpen an image by highpass filtering Extract fine details like edges
Sharpening
Sharpening & Highpass filtering Which filter can we use for sharpening? Does it work? No! We lose the DC component (uniform region)
Sharpening & Highpass filtering detail
= original smoothed (5x5) detail + α = original detail sharpened
Sharpening & Highpass filtering
Sharpening * 0 0 0 0 2 0 0 0 0 - = Original sharpened
More Linear Operations: Sharpening Filters They are useful for highlighting or enhancing details and high frequency information (e.g. edges) They can (and often are) based on derivativetype operations in the image Recall: f x f (x+ h) h f (x) f ( x + ) f ( x) Continuous Discrete
Derivative type Filters [ ]
Derivative type Filters High pass filter: noise is amplified!
Derivative type Filters In 2D: Common design choice: Low pass filter along y
Derivative type Filters
Derivative type Filters Directional Derivatives Of A Binary Image Original Image 2D Kernel 0 0 0 0 0 0 0 0 2D Kernel - 0-0 - 0 2D Kernel -2-0 2-2 - 0 2-2 - 0 2-2 - 0 2-2 - 0 2 2D Kernel -3-2 - 0 2 3-3 -2-0 2 3-3 -2-0 2 3-3 -2-0 2 3-3 -2-0 2 3-3 -2-0 2 3-3 -2-0 2 3
Derivative type Filters ), ( ), ( 2 ), ( y ), ( ), ( 2 ), ( x 2 2 2 2 + + + + y x f y x f y x f f y x f y x f y x f f [ ] 2 2 Laplacian: + = 2 2 2 2 2 y x f f f [ ] 2 = + 0 0 4 0 0 2
Variations of the Laplacian Filter Laplacian: + = 2 2 2 2 2 y x f f f [ ] 2 = + 0 0 4 0 0 2 Same response in row/column directions = + 8 0 0 4 0 0 0 0 4 0 0 Isotropic filter
59 Sharpening Using the Laplacian Filter ), ( ), ( ), ( 2 y x f y x f A y x g = + 8 A Boosting High Frequencies Courtesy of Peyman Milanfar
Courtesy of Peyman Milanfar