Final Review Image Processing CSE 166 Lecture 18
Topics covered Basis vectors Matrix based transforms Wavelet transform Image compression Image watermarking Morphological image processing Segmentation CSE 166, Fall 2017 2
General inverse transform using basis vectors CSE 166, Fall 2017 3
Matrix based transforms Discrete Fourier transform (DFT) Discrete Hartley transform (DHT) Discrete cosine transform (DCT) Discrete sine transform (DST) Walsh Hadamard (WHT) Slant (SLT) Haar (HAAR) Daubechies (DB4) Biorthogonal B spline (BIOR3.1) CSE 166, Fall 2017 4
Basis vectors of matrix based 1D transforms Example: DFT of f(x) = sin(2πx), N = 8 real part + imaginary part CSE 166, Fall 2017 5
Basis vectors of matrix based 1D transforms N = 16 real part imaginary part Standard basis (for reference) CSE 166, Fall 2017 6
Basis vectors of matrix based 1D transforms N = 16 basis dual Standard basis (for reference) CSE 166, Fall 2017 7
Basis images of matrix based 2D transforms Standard basis images (for reference) 8 by 8 array of 8 by 8 basis images CSE 166, Fall 2017 8
Basis images of matrix based 2D transforms Discrete Fourier transform (DFT) basis images real part imaginary part CSE 166, Fall 2017 9
Basis images of matrix based 2D transforms Discrete Hartley transform (DHT) basis images CSE 166, Fall 2017 10
Basis images of matrix based 2D transforms Discrete cosine transform (DCT) basis images CSE 166, Fall 2017 11
Basis images of matrix based 2D transforms Discrete sine transform (DST) basis images CSE 166, Fall 2017 12
Basis images of matrix based 2D transforms Walsh Hadamard transform (WHT) basis images CSE 166, Fall 2017 13
Basis images of matrix based 2D transforms Slant transform (SLT) basis images CSE 166, Fall 2017 14
Basis images of matrix based 2D transforms Haar transform (HAAR) basis images CSE 166, Fall 2017 15
Wavelet transforms A scaling function is used to create a series of approximations of a function or image, each differing by a factor of 2 in resolution from its nearest neighboring approximations. Wavelet functions (wavelets) are then used to encode the differences between adjacent approximations. The discrete wavelet transform (DWT) uses those wavelets, together with a single scaling function, to represent a function or image as a linear combination of the wavelets and scaling function. CSE 166, Fall 2017 16
Scaling function, multiresolution analysis 1. The scaling function is orthogonal to its integer translates. 2. The function spaces spanned by the scaling function at low scales are nested within those spanned at higher scales. 3. The only function representable at every scale (all ) is f(x) = 0. 4. All measureable, square integrable functions can be represented as CSE 166, Fall 2017 17
Relationship between scaling and wavelet function spaces CSE 166, Fall 2017 18
2D discrete wavelet transform Decomposition CSE 166, Fall 2017 19
2D discrete wavelet transform 3 level wavelet decomposition CSE 166, Fall 2016 20
Wavelet based edge detection Zero lowest scale approximation Edges Zero horizontal details Vertical edges CSE 166, Fall 2016 21
Wavelet based noise removal Noisy image Threshold details Zero highest resolution details Zero details for all levels CSE 166, Fall 2016 22
Data redundancy in images Coding redundancy Spatial redundancy Irrelevant information Does not need all 8 bits Information is unnecessarily replicated Information is not useful CSE 166, Fall 2017 23
Fidelity criteria subjective (qualitative) CSE 166, Fall 2017 24
Approximations Objective (quantitative) quality rms error (in intensity levels) 5.17 15.67 14.17 Lower is better (a) (b) (c) Subjective (qualitative) quality, relative (a) is better than (b). (b) is better than (c) CSE 166, Fall 2017 25
Compression system CSE 166, Fall 2017 26
Compression methods Huffman coding Golomb coding Arithmetic coding Lempel Ziv Welch (LZW) coding Run length coding Symbol based coding Bit plane coding Block transform coding Predictive coding Wavelet coding CSE 166, Fall 2017 27
Symbol based coding (0,2) (3,10) CSE 166, Fall 2017 28
Block transform coding Encoder Decoder CSE 166, Fall 2017 29
Block transform coding 4x4 subimages (4x4 basis images) Walsh Hadamard transform Discrete cosine transform CSE 166, Fall 2017 30
8x8 subimages Block transform coding Fourier transform Walsh Hadamard transform cosine transform Retain 32 largest coefficients Error image rms error 2.32 1.78 1.13 Lower is better CSE 166, Fall 2017 31
Block transform coding Reconstruction error versus subimage size DCT subimage size: 2x2 4x4 8x8 CSE 166, Fall 2017 32
JPEG uses block DCT based coding Compression reconstruction Scaled error image Zoomed compression reconstruction 25:1 Compression ratio 52:1 CSE 166, Fall 2017 33
Predictive coding model Encoder Decoder CSE 166, Fall 2017 34
Predictive coding Example: previous pixel coding Input image Histograms Prediction error image CSE 166, Fall 2017 35
Wavelet coding Encoder Decoder CSE 166, Fall 2017 36
Wavelet coding Detail coefficients below 25 are truncated to zero CSE 166, Fall 2017 37
JPEG 2000 uses wavelet based coding Compression reconstruction Scaled error image Zoomed compression reconstruction 25:1 Compression ratio 52:1 CSE 166, Fall 2017 38
JPEG 2000 uses wavelet based coding Compression reconstruction Scaled error image Zoomed compression reconstruction 75:1 Compression ratio 105:1 CSE 166, Fall 2017 39
Visible watermark Watermark Watermarked image Original image minus watermark CSE 166, Fall 2017 40
Invisible image watermarking system Encoder Decoder CSE 166, Fall 2017 41
Invisible watermark Example: watermarking using two least significant bits Original image JPEG compressed Extracted watermark Two least significant bits Fragile invisible watermark CSE 166, Fall 2017 42
Invisible watermark Example: DCT based watermarking Watermarked images Extracted robust invisible watermark CSE 166, Fall 2017 43
Reflection and translation Reflection Translation CSE 166, Fall 2017 44
Sets of pixels: objects and structuring elements (SEs) Border of background pixels around objects Tight border around SE CSE 166, Fall 2017 45
Reflection about the origin Origin Don t care elements CSE 166, Fall 2017 46
Erosion Example: square SE CSE 166, Fall 2017 47
Erosion Example: elongated SE CSE 166, Fall 2017 48
Erosion Shrinks 11x11 15x15 45x45 CSE 166, Fall 2017 49
Dilation Examples Square SE Elongated SE CSE 166, Fall 2017 50
Dilation Expands CSE 166, Fall 2017 51
Opening Structuring element rolls along inner boundary CSE 166, Fall 2017 52
Closing Structuring element rolls along outer boundary CSE 166, Fall 2017 53
Opening and closing Erosion Opening Dilation Closing CSE 166, Fall 2017 54
Morphological image processing Noisy input Erosion Opening Dilation Closing Dilation Erosion CSE 166, Fall 2017 55
Boundary extraction Erosion Set difference CSE 166, Fall 2017 56
Boundary extraction CSE 166, Fall 2017 57
Hole filling Given point in hole CSE 166, Fall 2017 58
Hole filling Given points in holes All holes filled CSE 166, Fall 2017 59
Connected components Given point in A CSE 166, Fall 2017 60
Connected components X ray image 15 connected components CSE 166, Fall 2017 61
Image segmentation Input Edges Segmentation Edge based Region based CSE 166, Fall 2017 62
Image derivatives CSE 166, Fall 2017 63
Detection of isolated points Laplacian (second derivative) Threshold absolute value Input Segmentation CSE 166, Fall 2017 64
Line detection Double lines Input Laplacian (second derivative) Threshold absolute value Threshold value CSE 166, Fall 2017 65
Line detection, specific directions Spatial filters CSE 166, Fall 2017 66
Line detection, specific directions +45 Negative values set to zero Threshold CSE 166, Fall 2017 67
Edge models Step Ramp Roof edge CSE 166, Fall 2017 68
Edge models Step Ramp Roof edge CSE 166, Fall 2017 69
Ramp edge Two points First derivative Second derivative One point CSE 166, Fall 2017 70
Noise and image derivatives Input First derivative Second derivative Noise CSE 166, Fall 2017 71
Gradient and edge direction Gradient direction is orthogonal to edge direction CSE 166, Fall 2017 72
Gradient operators Forward difference CSE 166, Fall 2017 73
Gradients Input Magnitude of vertical gradient Magnitude of horizontal gradient Magnitude of gradient vector CSE 166, Fall 2017 74
Gradients Smooth image prior to computing gradients. Results in more selective edges Input Magnitude of vertical gradient Magnitude of horizontal gradient Magnitude of gradient vector CSE 166, Fall 2017 75
Edge detection Threshold magnitude of gradient vector Without smoothing With smoothing CSE 166, Fall 2017 76
Advanced edge detection Input Magnitude of gradient vector (with smoothing) Marr Hildreth Canny See textbook for algorithms CSE 166, Fall 2017 77
Thresholding Histograms Single threshold Dual threshold CSE 166, Fall 2017 78
Noise and thresholding Noise CSE 166, Fall 2017 79
Varying background and thresholding Input Intensity ramp Product of input and intensity ramp CSE 166, Fall 2017 80
Basic global thresholding Input Intensity ramp Threshold CSE 166, Fall 2017 81
Optimum global thresholding Input Histogram Basic global thresholding Optimum global thresholding using Otsu s method CSE 166, Fall 2017 82
Image smoothing to improve global thresholding Otsu s method Without smoothing With smoothing CSE 166, Fall 2017 83
Image smoothing does not always improve global thresholding Otsu s method Without smoothing With smoothing CSE 166, Fall 2017 84
Edges to improve global thresholding Input Masked input Mask image (thresholded gradient magnitude) Optimum global thresholding using Otsu s method CSE 166, Fall 2017 85
Edges to improve global thresholding Mask image (thresholded absolute Laplacian) Input Masked input Optimum global thresholding using Otsu s method CSE 166, Fall 2017 86
Variable thresholding Input Global thresholding Local standard deviations Local thresholding using standard deviations CSE 166, Fall 2016 87
Variable thresholding Input (spot shading) Global thresholding Local thresholding using moving averages CSE 166, Fall 2016 88
Variable thresholding Input (sinusoidal shading) Global thresholding Local thresholding using moving averages CSE 166, Fall 2016 89
Segmentation by region growing Input X ray image Initial seed image Final seed image Output image CSE 166, Fall 2017 90
Segmentation by region growing Difference image Difference image thresholded using dual thresholds Difference image thresholded with the smallest of the dual thresholds Segmentation by region growing CSE 166, Fall 2017 91
Advanced segmentation methods k means clustering Superpixels Graph cuts CSE 166, Fall 2017 92
Segmentation using k means clustering Input Segmentation using k means, k = 3 CSE 166, Fall 2017 93
Superpixels Input image of 480,000 pixels Image of 4,000 superpixels with boundaries Image of 4,000 superpixels CSE 166, Fall 2017 94
Superpixels 1,000 superpixels 500 superpixels 250 superpixels CSE 166, Fall 2017 95
Superpixels for image segmentation Input image of 301,678 pixels Segmentation using k means, k = 3 Superpixel image (100 superpixels) Segmentation using k means, k = 3 CSE 166, Fall 2017 96
Images as graphs Stronger (greater weight) edges are darker Simple graph with edges only between 4 connected neighbors CSE 166, Fall 2017 97
Graph cuts for image segmentation Cut the weak edges CSE 166, Fall 2017 98
Graph cuts for image segmentation Input Smoothed input Graph cut segmentation CSE 166, Fall 2017 99