ECE 484 Digital Image Processing Lec 12 - Mid Term Review

Transcription

1 ECE 484 Digital Image Processing Lec 12 - Mid Term Review Zhu Li Dept of CSEE, UMKC Office: FH560E, lizhu@umkc.edu, Ph: x slides created with WPS Office Linux and EqualX equation editor Z. Li, ECE 484 Digital Image Processing, 2018 p.1

2 Outline About Mid-Term Review of the coverage so far Image Formation - Geometry and Color Point Operation Linear Filters Transform Domain Filters Non-Linear Filters Summary Z. Li, ECE 484 Digital Image Processing, 2018 p.2

3 DIP Mid-term Date: 10/25 (Thursday), in class Close book, no cell phone, but allow 1 A4/letter sized handwritten cheating sheet Mid-term coverage: Image Formation: Geometry constraint Image Formation: Color space and features Point operation: histogram equalization, quantization Linear Filtering Non-Linear Filtering: median, bilateral, cross bilateral, and guided filtering Freq domain filtering Image Resampling Image Restoration Green Cyan (120 o ) (180 o ) Blue (240 o ) White Black Val ue Yellow (60 o ) Red Magenta (0 o ) (300 o ) Hue Z. Li, ECE 484 Digital Image Processing, 2018 p.3

4 Image Formation - Geometry Projection Model Homography Z. Li, ECE 484 Digital Image Processing, 2018 p.4 ú ú ú ú û ù ê ê ê ê ë é ú ú ú û ù ê ê ê ë é ú ú ú û ù ê ê ê ë é = ú ú ú û ù ê ê ê ë é z y x t r r r t r r r t r r r v u v u w z y x b a

5 Image Wrapping VL_FEAT has an implementation: zero singular value V column Z. Li, Digital Image Processing, 2018 p.5

6 HSV Color Model Hue, Satuation and Value (brightness) color model A cone with inituitive appeal of painters' tint, shade and tone model pure red: H=0, S=1, V=1. tints: adding white pigments, decreasing saturation shades: adding black, decrease brightness tones: dereasing S and V Human can differentiate approx. 128 hues, and 130 levels of saturation The number of values (brightness) is color dependent, approx 16 for blue, and 23 for yellow Z. Li, Digital Image Processing, 2018 p.6

7 MPEG-7 Scalable Color Descriptor Scalable Color Descriptor: Scalable Color Descriptor (SCD) is in the form of a color histogram in the HSV color space encoded using a Haar transform. H is quantized to 16 bin and S and V are quantized to 4 bins each, total 256 bins. The pixel count for each bin is quantized to 4 bits, so at max 256x4=1024 bits for representing. The distance between two images are therefore hamming distance, Scalability thru Haar trans. Green (120 o ) Value Cyan (180 o ) Blue (240 o ) Yellow (60 o ) Magenta (300 o ) Red (0 o ) White Hue Black Saturation Z. Li, Digital Image Processing, Fall 2018 p.7

8 Gamma Correction - Adjust dynamic ranges Matching Display characteristics power-law response functions in practice CRT Intensity-to-voltage function has ¼ 1.8~2.5 Camera capturing distortion with c = Similar device curves in scanners, printers, power-law transformations are also useful for general purpose contrast manipulation Z. Li, ECE 484 Digital Image Processing, 2018 p.8

9 Histogram Equalization Objectives goal: map the each luminance level to a new value such that the output image has approximately uniform distribution of gray levels two desired properties monotonic (non-decreasing) function: no value reversals [0,1] [0.1] : the output range being the same as the input range pdf cdf 1 1 o 1 o 1 Z. Li, ECE 484 Digital Image Processing, 2018 p.9

10 Re-Cap: Linear Filters Convolution * 1/9 1/9 1 1/9 1 1/9 1 1/9 1 1/9 1 1/9 1 1/9 1 1/ = Padding Complexity: M*N*K 2 Z. Li, Digital Image Processing, 2018 p.10

11 Properties of Linear Filtering Convolution properties Shift-Invariant: f(m-k, n-j)*h = g(m-k, n-j), if f*h=g Associative: f*h 1 *h 2 = f*(h 1 *h 2 ) this can save a lot of complexity Distributive: f*h1 + f*h2 = f*(h1+h2) useful in SIFT s DoG filtering. Applications Scale space filtering with successive Gaussian DoG filtering with difference of Gaussian blurred images * = Z. Li, Digital Image Processing, 2018 p.11

12 Image Smoothing, Edge Detection Smoothing kernels from Gaussian im=imread('data/lenna.png'); im=rgb2gray(im); im=imresize(im, 0.5); sigma = 1.6.*sqrt([1:6]); m=15*ones(1,6); for k=1:length(sigma) h{k} = fspecial('gaussian', m(k), sigma(k)); if k>=2 h_dog{k-1} = h{k}-h{k-1}; im_dog{k-1} = imfilter(im, h{k}-h{k-1}); imagesc(im_dog{k-1}); axis off; title(sprintf('dog %s = %1.2f-%1.2f', '\sigma', sigma(k),sigma(k-1))); else imagesc(im); title('dog \sigma = 1'); axis off; end end fprintf('\n...end image smoothing'); Z. Li, Digital Image Processing, 2018 p.12

13 Sharpening Adding edge details back %sharpening for k=1:length(sigma) %sharpen by adding edge info im_sharp{k} = im + 0.5*im_dog{k}; figure(24); colormap('gray'); subplot(2,3,k); imagesc(im_sharp{k}); title(sprintf('sharpen: %s=%1.2f', '\sigma', sigma(k))); axis off; end Z. Li, Digital Image Processing, 2018 p.13

14 Bilateral Filter Filtering process * W c W s W s *W c input Output geometry kernelphotometry kernel bilateral kernel Z. Li, Digital Image Processing, 2018 p.14

15 Bilateral Filter No averaging across edges Z. Li, Digital Image Processing, 2018 p.15

16 BF Color Images Just need to adapt the range/photometry kernel works for hyperspectral images as well Z. Li, Digital Image Processing, 2018 p.16

17 Cross Bilateral/Guided Filters Cross Bilateral Filtering CBF: use a different image to derive range kernel Guided Filter solving local linear regression for a linear filter model from guided image smooth if input image and guide image are not correlated (a k is small) Edge preserving if a k is big. Z. Li, Digital Image Processing, 2018 p.17

18 Denoising with multiple exposure images Guided fitler for image denoising A = imread('toysnoflash.png'); G = imread('toysflash.png'); r = 3; s = 0.001*diff(getrangefromclass(G)).^2; B = imguidedfilter(a, G, 'NeighborhoodSize', r, 'DegreeOfSmoothing',s); Z. Li, Digital Image Processing, 2018 p.18

19 Freq Domain Filtering: 2-D FT illustrated FFT2 illustrated: real-valued real imag Z. Li, Digital Image Processing, 2018 p.19

20 notes about 2D-DFT Output of the Fourier transform is a complex number Decompose the complex number as the magnitude and phase components In Matlab: u = real(z), v = imag(z), r = abs(z), and theta = angle(z) real function Z. Li, Digital Image Processing, 2018 p.20

21 Filtering in Frequency Domain Why filtering in Freq Domain? faster convolution, if involves large kernels better denoising (notch, inverse and wiener filtering) Z. Li, Digital Image Processing, 2018 p.21

22 Sampling in Time Domain Computer needs a discrete representation of signals Many signals originate as continuous-time signals, e.g. conventional music or voice By sampling a continuous-time signal at isolated, equally-spaced points in time, we obtain a sequence of numbers s [ n] = s( ) n T s n {, -2, -1, 0, 1, 2, } T s is the sampling period. s ( t) = s( t) d ( t - n ) å sampled T s n= - impulse train s sampled Sampled analog waveform Z. Li, Digital Image Processing, 2018 p.22 T s ( t) T s t s(t)

23 Consequence in Freq Domain Multiplication with sampling train function, is convolving in freq domain Replicates spectrum of continuous-time signal At offsets that are integer multiples of sampling frequency Fourier series of impulse train where s = 2 f s ( ) = å dt s t d ( t - n Ts ) = + cos( s t) + cos(2 s t) +... n= - Ts Ts Ts 1 g( t) = f ( t) dt s ( t) = f ( t) + 2 f ( t)cos( s t) + 2 f ( t)cos(2 s t) T Example F( ) s ( +... ) Modulation by cos( s t) G( ) Modulation by cos(2 s t) -2 f max 2 f max - s - s s s gap if and only if 2 f max < 2 f s - 2 f max Û fs > 2 f max Z. Li, Digital Image Processing, 2018 p.23

24 Sampling in 2D Very similar Z. Li, Digital Image Processing, 2018 p.24

25 Filtering to combat Aliasing Pre-filtering to limit image bandwidth to fit in sampling rate Z. Li, Digital Image Processing, 2018 p.25

26 Resampling Interpolation 2D interpolation Z. Li, Digital Image Processing, 2018 p.26

27 Bilinear & DCTIF Interpolation Bilinear DCTIF Z. Li, Digital Image Processing, 2018 p.27

28 Image Restoration Image Restoration from Degradation Degradation sources: Noise - independent of (x,y) Point Spread Function (PSF) - a function of (x, y), and assuming linear Z. Li, ECE 484 Digital Image Processing, 2018 p.28

29 Noise Supression Spatial Filtering Linear: Mean, Gaussian, and Media Filters Non-Linear: Bilateral Filters/Guided Filters Freq Domain Filtering Low Pass Filters Band pass Filters Notch filters for repetive patterns Z. Li, ECE 484 Digital Image Processing, 2018 p.29

30 Inverse Filtering Degradation from PSF Z. Li, ECE 484 Digital Image Processing, 2018 p.30 ), ( ), ( ), ( ), ( ), ( ), ( ˆ v u H v u N v u F v u H v u G v u F + = =

31 Noise Magnifying Problem with Inverse filters G(u,v)=F(u,v)H(u,v)+N(u,v) Unknown noise => ˆ G( u, v) F ( u, v) = = F( u, v) + H ( u, v) N( u, v) H ( u, v) Estimate of original image Problem: 0 or small values Sol: limit the frequency around the origin Z. Li, ECE 484 Digital Image Processing, 2018 p.31

32 The Inverse Filtering - Cut offs and Pseudo Inv Inverse filter with cut-off: Pseudo-inverse filter: Can the filter take values between 1/H(u,v) and zero? Can we model noise directly? 32

33 Wiener filter goal: restoration with minimum mean-square error (MSE) optimal solution (nonlinear): restrict to linear space-invariant filter find optimal linear filter W(u,v) with min. MSE 33

34 Wiener filter Min MSE Fitlering: goal: restoration with minimum mean-square error (MSE) find optimal linear filter W(u,v) with min. MSE orthogonal condition correlation function wide-sense-stationary (WSS) signals Fourier Transform: from correlation to spectrum 34

35 observations about Wiener filter If no noise, S 0, it is a Pseudo Inv Filter: Pseudo inverse filter If no blur, H(u,v)=1 (Wiener smoothing filter) More suppression on noisier frequency bands 35

36 Wiener Filtering Wiener Filtering Solving for a MSE objective function, that has freq domain solution Basically inverse fitler but reflect the Signal to Noise ratios at freq locations Applications in debluring, and motion debluring Z. Li, ECE 484 Digital Image Processing, 2018 p.36

37 Summary Relax, mid-term is more for me to check on the coverage effectiveness, will adjust accordingly. Focus on your homework programming assignments, that is more useful Start thinking a course project that leads to short conf paper (4 page) submission that will give you 25% extra credit. Cheating sheet: sampling theorem, quantization,...etc. Z. Li, ECE 484 Digital Image Processing, 2018 p.37