Fall 2015 Dr. Michael J. Reale

Transcription

1 CS 49: Computer Vision MIDTERM REVIEW Fall 25 Dr. Michael J. Reale

2 Midterm Review The Midterm will cover: REVIEW slide decks (inclusive) Quizzes through 4 (inclusive)

3 REVIEW : INTRODUCTION

4 Basic Terms and Concepts Computer Vision Allow machine to see and understand real world Image Processing Helps emphasize important information for machines or humans Pattern Recognition Classifying and recognizing patterns in data (including image data) Computer Graphics Create or synthesize a virtual image Artificial Intelligence Emulating human intelligence

5 Computer Vision vs. Computer Graphics Computer vision Real world machine understanding Computer graphics Virtual world (machine) looks like real world

6 Computer Vision / Image Processing Evaluation Subjective Human thinks it looks good Objective Ground truth comparison E.g., mean square pixel error

7 Image Representation Image represented by f(x,y) f(x,y) two-dimensional light intensity function (x,y) spatial coordinates Value of f(x,y) intensity / gray level / gray scale Monochrome images single f(x,y) Color images multiple channels, each with their own f(x,y) or 3D function f(x,y,c) (with c = {,,2}) f(x,y) returns 3D vector {r,g,b}

8 Digital Image Representation Digital image Image f(x,y) 2D array with discrete coordinates Each element pixel Each pixel value discrete brightness levels Often [,255] one byte of storage

9 Digital Image Processing Operations Low-level: Primitive operations INPUTOUTPUT: Image Image Mid-level: Image segmentation, classification INPUTOUTPUT: Image Attributes (e.g., edges, objects) High-level: Making sense of ensemble of recognized objects; cognitive processing INPUTOUTPUT: Image Scene of objects

10 REVIEW 2: DIGITAL IMAGE BASICS

11 Human Vision System: Cones and Rods Two types of receptors in eye: Cones Brightness and color Photopic vision = bright-light vision Most in fovea true center of vision Rods Low-level light (no color) Scotopic vision = dim-light vision (# of rods) > (# of cones)

12 Image Perception and Formation: Eye vs. Camera Camera Lens moves but has fixed shape Eye Lens changes shape but doesn t move

13 Vision Properties: Brightness / Contrast Adaptation Humans dynamically adjust perceived brightness and contrast based on average local intensity Wide range of intensity levels human eye can adapt to Cannot adapt to entire range simultaneously

14 Vision Properties: Brightness and Spatial Discrimination Brightness discrimination Discriminate between different intensity levels Spatial discrimination Discriminate between different physical points on object

15 Image Acquisition: EM Spectrum Electromagnetic spectrum Varies in wavelength Visible light fraction of spectrum Higher frequency higher energy

16 Image Acquisition: Sensor Mechanics Illumination energy sensor voltage waveform

17 Image Acquisition: Image Digitizing Sampling = digitizing the coordinate values Ideally, sampling at Nyquist Rate: 2*F max F max = maximum frequency Quantization = digitizing the amplitude values Both may be uniform or non-uniform

18 Image Sensing: Types of Sensors Single sensor Must move sensor/object in at least two dimensions OR use mirrors Infinite possible resolution Line sensors Must move sensor/object in at least one dimension Sensor strips (scanners) and CT/MRI Finite resolution in one dimension; infinite in others Array of sensors 2D array CCD Finite resolution in all dimensions

19 Image Acquisition: How much space to represent a digital image? Given a digital image with: M rows (y coordinate) N columns (x coordinate) k bits per pixel Number of pixels total = M*N Number of possible gray levels per pixel = 2 k Total size = M*N*k

20 Image Acquisition: Coordinates and Values In image v = f(x,y): (x,y) IMAGE coordinates (NOT world coordinates) v INDEX to illumination value (NOT true illumination value)

21 Image Acquisition: Spatial Resolution Spatial resolution measured by: # of pixels per physical image size E.g. DPI = dots per inch # of line pairs per physical vertical image distance E.g., lp/mm = line pairs per millimeter

22 Image Acquisition: Aliasing Aliasing Jagged or staircase effect Caused by sampling/displaying lower than Nyquist rate (2*F max )

23 Image Transformations Size change: Zoom in low to high resolution Need to fill gaps: Pixel replication nearest neighbor per pixel Pixel interpolation weighted average of pixel and neighbors Super-resolution Map from new, high-res back to old, low-res Zoom out high to low resolution Pick pixels using above methods in reverse Map from new, low-res back to old, high-res Shape Change Also known as geometric transformation

24 Image Quality Assessment Subjective Human-verified Used often for image enhancement, visualization, and effects Objective Ground truth comparison Used often for encoding/decoding, classification, etc.

25 Connected Components: Neighbors N 4 (P) strong neighbors North, South, East, West 4-neighbors N D (P) weak neighbors Diagonals N 8 (P) = N 4 (P) + N D (P) 8-neighbors

26 Connected Components: Adjacency 4-adjacency q is in set N 4 (p) 8-adjacency q is in set N 8 (p) m-adjacency (mixed-adjacency) ) q is in set N 4 (p) OR 2) q is in set N D (p) AND no -valued pixels in N 4 (p) N 4 (q)

27 Connected Components: Definitions Path = p and q connected m path: path based on m-connected pixels closed path: starting p and ending q are connected Connected component Set of pixels which are connected connected set Region Connected set with closed-path boundary Edge Gray-level discontinuity at a point I.e., perpendicular to edge, intensity changes sharply Linked edge points edge segment

28 Connected Components: Distance Distance Types: Euclidean (D e ) circular, disk shape De(p,q) = sqrt[(xp xq)^(2) + (yp yq)^(2)] City-block or Manhattan (D 4 ) diamond shape D 4 (p,q) = (xp xq) + (yp yq) Chessboard (D 8 ) square shape D 8 (p,q) = max( (xp xq), (yp yq) ) Shortest m-path (D m )

29 Pixel Operations Pixel operations Point-wise operations Can operate on one or two images Treat image as an M x N matrix Can be linear or non-linear

30 REVIEW 3: INTENSITY TRANSFORMATIONS

31 Intensity (Gray-Level) Transformations Intensity (or Gray-Level) Transformations s = T(r)

32 Types of Intensity Transformations Basic Transformations Linear Original Negative Piece-wise linear Bit-plane slicing Log Power-Law (Gamma) Histogram equalization

33 Basic Transformation Functions

34 Linear Transforms Original s = r Negative s = L r

35 Linear Transforms Piece-wise linear Contrast stretching = increase range of intensity levels Intensity-level slicing = brighten/darken certain intensity range Zig-zag = when input range >> output range s s s r r r

36 Bit--Plane Slicing Bit Bit 7 Can decompose 8-bit image into bit planes., Bit-7 Bit-

37 Log Transformations s = c log( + r) When input range >> display range s n m a b r

38 Power-Law (Gamma) s = cr γ γ = Greek letter gamma More flexible than log transform Used in gamma correction Transforms image to display correctly on given device Needed because of non-linear voltage-to-intensity response

39 Histograms Histogram discrete probability function Bin for each possible value Bin contains either: Number of pixels with given gray value OR Probability value from. to. P ( r) r n r N where N = total # of pixels and n r = # of pixels with r intensity

40 Histogram Equalization Histogram Equalization Transform grayscale values so that histogram is more evenly distributed Use CDF (Cumulative Distribution Function) as transformation function T(r) s T( r) ( L ) r j P r ( j) ( L ) N r j n j r s

41 REVIEW 4: IMAGE FILTERING

42 Filtering Overview Filtering Accepting/rejecting certain frequency components in signal (image) Spatial Filtering Uses spatial filters, (masks, kernels, templates, windows) on each pixel and neighborhood to produce new output pixel

43 Types of Filtering Low-pass filtering mask Accepts (passes) lower frequencies Image blurring, smoothing, etc. Order-Statistic (Nonlinear) filtering E.g., median filter Get median value of neighbors High-pass filtering Accepts (passes) higher frequencies Sharpening, edge enhancement, etc.

44 Correlation and Convolution Correlation Procedure: Center mask M around each pixel Multiply each neighbor by corresponding mask pixel Divide by sum M Set output pixel to new value Can be used for matching section of image Convolution Same as correlation, but mask pre-rotated 8 Gives echo of mask when given unit impulse

45 Correlation Correlation vs. vs. Convolution Convolution w(x,y) Input image f(x,y) Rotated w(x,y) Correlation result Convolution result

46 Linear Filters Linear filters only require you to specify the values in the mask

47 Low Low-Pass Filters Pass Filters Box filter all coefficients equal Gaussian filter based on Gaussian function 9 M M

48 Order-Statistic (Non-Linear) Filters Order-statistic filter Order (rank) pixels inside filter s area Replace center pixel with value determined by ranking result Non-linear filter

49 Order-Statistic Filters: Median Filter Median filter Best known order-statistic filter Replaces center value with median of pixels Effective in presence of impulse (salt-andpepper) noise 5 th percentile

50 Order-Statistic Filters: Max and Min Filters Do not just have to pick the middle number Max filter use th percentile number Min filter use th percentile number

51 Sharpening Sharpening highlight transitions in intensity

52 Sharpening vs. Averaging Averaging analogous to integration Sharpening uses differentiation

53 Isotropic Filters Isotropic filters rotation-invariant I.e., get same result if you rotate filter

54 The The Laplacian Laplacian Filter Filter Uses second derivative Linear operator Without diagonal terms: With diagonal terms: y f x f f 4 8

55 Image Sharpening with the Laplacian Subtract Laplacian image from original image: g( x, y) f ( x, y) 2 f ( x, y) Note: if center of mask is positive (signs flipped), add Laplacian image

56 Unsharp Masking Follow these steps: ) Blur the original image 2) Subtract blurred image from original (resulting difference called a mask) 3) Add mask to original

57 Highboost Filtering If m(x,y) = mask and k = weight: g( x, y) f ( x, y) k * m( x, y) k = unsharp masking k > highboost filtering

58 Gradient: Introduction The gradient of f at (x,y) is a 2D column vector: f grad( f g ) g f f x y Points in direction of greatest rate of change of f at (x,y) x y

59 Gradient Magnitude Magnitude of gradient: M ( x, y) mag( f ) g 2 g 2 x y If mask only isotropic in 9 increments: M ( x, y) g g x y Gradient image = image with M(x,y) for each pixel value

60 Masks for g x and g y 2x2 Roberts cross-gradient operators Problem: even size awkward

61 Sobel Sobel Operators Operators 3x3 Sobel operators Effectively combining differentiation with Gaussian smoothing

62 REVIEW 5: IMAGE ENHANCEMENT

63 Introduction There are times we want to improve the quality of an image Subjective improvement ( looks better ) Emphasize important information Removing/weakening noise Some of the approaches: Histogram Equalization Image Filtering Other approaches: Basic Intensity Transformations Image Subtraction N-Images Averaging

64 Histogram Equalization Procedure: Compute histogram of image Normalize histogram (divide by total number of pixels) Get Cumulative Distribution Function (CDF) of histogram r ( L ) s T( r) CDF( r) n j N j Use CDF (multiplied by 255) to reassign pixel values s = T(r) = 255*CDF(r) Problem: If first value (r = ) dominates histogram image washed out after equalization! Histogram Equalization

65 Histogram Stretching To fix this, we need to stretch out the values Get the value of CDF[] and subtract it from every value Get value of CDF[255] and divide all values by CDF[255] Then, perform remapping

66 Resulting Image after Histogram Equalization and Stretching Histogram Equalization and Stretching

67 Image Filtering One can use image filters to: Blur an image (5x5 Gaussian) Sharpen an image (Laplacian) Enhance an image (7x7 Median)

68 Other Image Enhancement Approaches Intensity Transformations E.g., log transform Image Subtraction N-Images Averaging Get average of multiple images to filter out noise

69 REVIEW 6: IMAGE TRANSFORMATION: INTRODUCTION TO THE FOURIER TRANSFORM

70 Fourier Series Fourier Series Can express any periodic function as sum of sines/cosines Different frequencies and different coefficients f ( t) c n e 2n j t T n

71 Fourier Transform Fourier Transform Can express any function (with finite area) as integral of sines/cosines multiplied by weighing function Fourier Transform Pair: Forward: F( ) f ( t) e j2 t dt Inverse: f ( t) F( ) e j2 t d t is in the spatial domain, while μ is in the frequency domain

72 Fourier Domain and Back Again Can transform data to Fourier Domain and then back to the original domain without losing any information

73 FFT: Fast Fourier Transform Discrete Fourier Transform algorithm Developed in early 96s Allowed for much faster processing: O(N 2 ) O(N lg N)

74 Complex Numbers A complex number C is defined as: C R ji where: R and I = real numbers j = imaginary number equal to R real part I imaginary part

75 Complex Numbers as Coordinates One can think of complex numbers as being the coordinates on a 2D plane Imaginary I C R ji ( R, I ) R Real

76 Euler s Formula Euler s formula is given by the following: e j cos j sin Bonus: If you set θ = π, you get: e j

77 Impulse A unit impulse of a continuous variable t located at t = is defined as: ( t) constrained by: if if ( t) dt t t (t) t Basically, spike of infinite amplitude and zero duration, having unit area

78 Sifting Property Around : f ( t) ( t) dt f () Around arbitrary point t : f ( t) ( t t) dt f ( t)

79 Unit Discrete Impulse Unit discrete impulse with discrete variable x: ( x) if if x x which clearly hold to this constraint: x ( x)

80 Impulse Train Impulse train sum of infinitely many periodic impulses ΔT units apart: T n s ( t) ( t nt ) -3ΔT -2ΔT ΔT ΔT 2ΔT 3ΔT Can be continuous or discrete

81 Convolution Theorem Fourier Transform Pair: f ( t) h( t) H( ) F( ) f ( t) h( t) H( ) F( ) Double-arrow means: left left (Convolution of two functions in Spatial Domain) = (Product of two functions in Frequency Domain) right right

82 Modeling Sampling as an Impulse Train Multiply f(t) by impulse train s(t) to get samples ~ T n f ( t) f ( t) s ( t) f ( t) ( t nt ) f k f ( kt) Images from Gonzalez-Woods Digital Image Processing

83 Fourier Transform of Sampled Function ~ F( ) T n F n T Infinite, periodic sequence of copies of transform of original function Value of /ΔT determines separation between copies

84 Discrete Fourier Transform Pair Forward: F( u) Inverse: M x f ( x) M f M u ( x) e F( u) e j2ux/ M j2ux/ M u =,, 2,, M- x =,, 2,, M- Note: neither depend on ΔT, so this works on any finite, uniformlysampled discrete data!

85 Bringing in Euler s Formula Bring in Euler s Formula e j cos j sin Gives us a nice complex number to deal with: e 2ux cos M j sin j2ux/ M 2 ux M

86 REVIEW 7: IMAGE TRANSFORMATION: 2D FOURIER TRANSFORM AND SAMPLING THEORY

87 Computing Forward and Inverse Fourier Transform on Samples Use same number of frequency samples as spatial samples When computing a single sample in one domain use ALL samples in other domain

88 2D Impulses and Sifting Property ( t, z) f if t z otherwise ( t, z) dtdz ( t, z) ( t, z) dtdz f (,) f ( t, z) ( t t, z z ) dtdz f ( t, z )

89 Discrete 2D Impulses and Sifting Property ( x, y) x y f if x y otherwise ( x, y) ( x, y) f (,) x y f ( x, y) ( x x, y y) f ( x, y)

90 2D Continuous Fourier Transform Pair Forward: F(, ) f ( t, z) e j 2 ( t z ) dtdz Inverse: f ( t, z) F(, ) e j 2 ( t z ) dd (t,z) continuous spatial domain (μ,ν) continuous frequency domain

91 2D Discrete Fourier Transform Pair 2D Discrete Fourier Transform Pair Forward: where M = width, N = height Inverse: Same M and N from Forward Transform (x,y) discrete spatial domain (u,v) discrete frequency domain ) / / ( 2 ), ( ), ( M x N y N vy M ux j e y x f v u F ) / / ( 2 ), ( ), ( M u N v N vy M ux j e v u F MN y x f

92 Band-Limited Function Band-limited function F(μ) = outside of finite interval [-μ max, μ max ] Images from Gonzalez-Woods Digital Image Processing

93 Fourier Transform of Sampled Function Fourier Transform of Sampled Function Infinite, periodic sequence of copies of F(μ) /ΔT separation between copies To recover f(t), need clean copy of F(μ) Images from Gonzalez-Woods Digital Image Processing

94 The Sampling Theorem Sampling Theorem To completely recover a continuous, bandlimited function from a set of samples, you must sample at a rate greater than the Nyquist Rate (2μ max ) T 2 max μ max = maximum frequency ΔT = sampling distance

95 Over, Critically, and Under Sampled Over sampled Critically sampled Under sampled Images from Gonzalez-Woods Digital Image Processing

96 Aliasing Revisited Aliasing higher frequencies overlap with lower frequencies, corrupting the signal Caused by under-sampling sampling at less than the Nyquist rate Results in a variety of image artifacts (blockiness, jagged lines, false highlights, etc.)

97 How to Recover the Original Function Assuming periods separated enough: F ( ) H ( ) ~ F ( ) where: H( ) T max max otherwise Then you can use the recovered F(μ) in the inverse Fourier Transform: f ( t) F( ) e j2 t d

98 How to Recover the Original Function Images from Gonzalez-Woods Digital Image Processing

99 Aliasing in Image Data Aliasing in image data unavoidable Image data finite number of samples limited in duration NOT band-limited Limit duration of f(t) = multiplying by boxcar function = convolving with sinc() function No function of finite duration (spatial) can be band-limited (frequency) h( t) f ( t) H( ) F( ) Spatial Domain Frequency Domain

100 Reducing Aliasing Smooth f(t) BEFORE sampling Attenuates higher frequencies not as prominent in F(μ) Process known as anti-aliasing NOTE: In graphics, anti-aliasing is effectively blurring after sampling used to cover up artifacts

101 2D Sampling Theorem Must sample in 2D such that: T 2 max Z 2 max where: ΔT and ΔZ = separation between samples μ max and ν max = max. frequencies in each dimension

102 Two Types of Image Aliasing Spatial Aliasing Image aliasing effects E.g., blockiness, jagged lines, etc. Temporal Aliasing Frame rate (temporal sampling rate) too low E.g., wagon wheel effect

103 Moiré Patterns Moiré Patterns Can happen when sampling scenes with repeating (periodic) or nearly periodic components More general than sampling artifacts "Moiré grid" by Fibonacci (talk) - Own work. Based on Image:MoireGrid.png.. Licensed under Public domain via Wikimedia Commons - ile:moir%c3%a9_grid.svg

104 Translating Fourier Transform Translating Fourier Transform Shift by M/2 and N/2 ) ( ) / / ( 2 ) )(, ( ), ( ), ( y x N y v M x u j y x f e y x f v v u u F 4 back-to-back Periods meet here

105 Fourier Transform in Polar Form Fourier Transform in Polar Form Since 2D DFT is complex, can express in polar form: where: ), ( ), ( ), ( v u j e v u F v u F ), ( ), ( ), ( 2 2 v u I v u R v u F ), ( ), ( arctan ), ( v u R v u I v u Fourier (or frequency) spectrum (Magnitude) Phase Angle

106 Fourier Spectrum and Phase Angle Fourier Spectrum Insensitive to image translation! It is affected by rotation, however Phase Angle Gives little intuitive information Contains VERY important shape information!

107 DC Component DC component F(,) value at zero frequency (like DC current) Usually very large F(,) MN f ( x, y) Which is why we would want to use an intensity transform, like the log transform

108 Example: Simple Box Input Image Spectrum Magnitude Phase Angle

109 REVIEW 8: IMAGE TRANSFORMATION: HOUGH TRANSFORM

110 Problem Given edge image, find straight lines Brute force: n 3 in complexity

111 Hough Transform for Lines General idea: Define lines using parameters (slope and intercept, or angle and offset) Using predefined increments, create array of accumulator cells from parameters (one for each possible line) For each point, redefine in terms of parameters (convert to parameter space) Increment matching cells in accumulator array E.g., indicate that this point could be part of a given line Pick lines corresponding to cells with strongest response y i ax i b b x a i y i

112 Quick Recap Single line (variable x and y) Point (a,b) in parameter space All the lines going through a fixed (x,y) Single line in parameter space (variable a and b)

113 Problem a is the slope of the line approaches infinity as line becomes vertical Alternative: use polar coordinates! xcos y sin

114 Hough Transform for Lines Subdivide ρθ parameter space into accumulator cells -9 θ 9 -D ρ D θ inc = increment of θ ρ inc = increment of ρ where D = max distance between opposite corners of image Initially set all accumulator cells to zero For every non-background point (x k, y k ) in xy plane: Cycle through all values of θ (incrementing by θ inc ) Solve for ρ round to nearest ρ cell Increment accumulator cell (ρ, θ)

115 Computational Complexity Accuracy determined by size of increments for θ and ρ Complexity: linear in n (number of nonbackground points) Before: n 3

116 Hough Transform Extended Hough Transform applicable to any function g(v,c) = where: v = vector of coordinates c = vector of coefficients

117 Hough Transform on Circles ) 2 ( ) 2 2 ( x c y c c 3D accumulator cells (c, c 2, c 3 ) 2 3

118 REVIEW 9: IMAGE TRANSFORMATION: PRINCIPAL COMPONENT ANALYSIS

119 Image Vectors Training set of images Individual images concatenate rows of image to make D vector (n = # of pixels) X x x 2 x n

120 Mean and Covariance of the Images Mean vector: M EX X Covariance Matrix: C X E ( X M )( X Gives an idea of how the data varies in the set of samples (images) X M X ) T

121 Covariance Matrix: Correlation Positive correlation c xy > Negative correlation c xy < No correlation c xy =

122 Eigenvectors and Eigenvalues Given matrix M (real and symmetric): V = eigenvectors λ = eigenvalues such that: MV V For symmetric matrix, eigenvectors define an orthonormal basis: A set of orthogonal axes that can be used to find matrix M

123 Hotelling Transform Hotelling Transform Also called discrete Karhunen-Loève transform C = covariance matrix A = eigenvectors of C (one per row, sorted by largest eigenvalue) M = mean vector for all X s Map each vector X i (single image) using A into a vector Y i Y i A( X M ) i

124 Reconstructing X A A T Get X i vector back from A T, Y i, and the mean M: X i A T Y i M

125 Intuition Behind Reconstruction X i A T Y i M A = basis vectors (in columns) Axes in multidimensional space In simple example, we had two dimensions: x and y For the images, every PIXEL position is a dimension For n pixels, the image is an n-dimensional vector

126 Intuition Behind Reconstruction X i A T Y i M Y i = weights for each vector Note there is a Y i for each X i Determines how the basis vectors should be added together

127 Intuition Behind Reconstruction X i A T Y i M M = mean of entire data set Remember A contains eigenvectors of covariance matrix Thus, need to add back mean

128 Principal Component Analysis: Reconstructing with Fewer Values Principal Component Analysis Eigenvectors sorted in order of importance Higher eigenvalue more variation Take first k eigenvectors A k and use first k weights in each Y i vector ^ X i A Y M First k eigenvectors = principal components T k Keep in mind: reconstructed X an approximation of real X i

129 Choosing k Sum all eigenvalues s Percentage of variance g Procedure: g = From (largest to smallest λ) g = g + λ/s If(g > (desired percentage of variance)) Stop, and use the k vectors you have so far Common threshold: 8%, 9%, 95%, 99% Often, k is surprisingly small

130 Eigenfaces Get eigenfaces (eigenvectors from PCA on face images) and Y vectors Defines face space Project unknown face image into face space trying to define new face in terms of old faces If Y close to previous examples, image = face If Y close to particular example, image = face of particular person

131 REVIEW : MOMENTS

132 Central and Raw Moments Central and Raw Moments Central moment distribution about mean Raw moment distribution about zero ) ( ) ( ) ( L i i n i n r P m r r ) ( ) '( L i i n i n r P r r

133 Mean and Variance of Intensity Mean (average) intensity: Also first raw moment m L i r P( i r i ) Variance of intensity: 2 ( r) L i Measure of the spread of values about the mean Also second central moment (n = 2) ( r m) P( 2 i r i )

134 REVIEW : IMAGE SEGMENTATION: POINTS, LINES, AND EDGES

135 Image Segmentation Image Segmentation breaking up image into regions and/or objects Image Attributes Output depends on application One of the most difficult tasks in image processing

136 Types of Image Segmentation Edge-based segmentation Assumes boundaries of regions different enough from other regions and background Looking for discontinuity Region-based segmentation Insides of regions are similar according to predefined criteria Looking for similarity

137 Region Requirements ) All pixels in a region 2) Regions are connected (not split up) 3) Regions don t overlap 4) Regions meet some criteria Q 5) Two adjacent regions don t meet criteria in same way

138 Edge-Based Segmentation Detecting sharp, local changes in intensity E.g., points, lines, and edges Use approximation of st or 2 nd derivative

139 Transitions of Intensity Three kinds of transitions in intensity: ) Ramp edges More gradual change in intensity 2) Step edges Sudden change in intensity 3) Roof edges Refers to transitions across line objects like lines

140 Properties of st and 2 nd Derivatives st Derivatives Produce thicker edges value on ramps 2 nd Derivative For ramps and steps, only has responses at onset and end Produce thinner edges Double-edge effect! Opposite signs at onset and end Can use sign to determine dark-to-light or light-to-dark Stronger response to fine detail Isolated points Thin lines Noise

141 Point Detection Point Detection Use 2 nd derivative Laplacian mask Threshold response to get points 8 otherwise ), ( if ), ( T y x R y x g

142 Line Detection Can use 2 nd derivative Works best with thinner lines WARNING: Must handle double-line effect properly! Any direction use Laplacian mask (isotropic) Specific direction use mask with implicit direction

143 Edge Detection: Introduction Edge detection Most common way to segment images based on abrupt local changes in intensity Edge segment connected edge points Noise a serious problem should smooth image beforehand

144 Edge Detection: A Closer Looks at Derivatives Magnitude of st derivative Used to detect presence of edge Sign of 2 nd derivative Whether point lies on dark or light side of edge Zero-crossing point of 2 nd derivative Can find center of thick edges

145 Edge Detection: Overall Procedure ) Smooth image Reduces noise 2) Detect edge points Local operation Find all potential candidates for image edge points 3) Localize edges Select true candidates Often involves linking edges points together to form edge segments

146 Basic Edge Detection Smooth the image beforehand Compute gradient magnitude (also called edge map) Threshold the gradient image Can get rid of weak, spurious edges HOWEVER, can also break up edge segments Especially if blurring applied first

147 Gradient Masks Gradient Masks D Masks Horizontal or vertical only Roberts Cross-Gradient Horizontal, vertical, and diagonal Prewitt Can optionally add diagonal Sobel Better noise suppression Can optionally add diagonal

148 Diagonals Diagonals When using the preceding Prewitt and Sobel masks, can use simpler gradient magnitude function: ALTERNATIVELY, can add two additional masks to emphasize diagonal edges: g x g y y x M ), ( Prewitt: Sobel:

149 Gradient Direction and Angle Gradient direction points in direction that is: the greatest rate of change orthogonal to the edge itself Gradient vector edge normal Angle of gradient vector: a( x, y) tan a(x,y) can be treated as another image g g y x

150 Marr-Hildreth Edge Detector Combines Laplacian with 2D Gaussian Laplacian of a Gaussian (LoG) x y 2 x y 2 2 G( x, y) e Laplacian isotrophic do not need multiple masks σ = space operator allows tuning of size Operations linear, so can Gaussian smooth then apply Laplacian g( x, y) 2 G( x, y) f ( x, y) Edges zero crossings of g(x,y) Can implement as two D convolutions Can also be approximated with difference of Gaussians (DoG)

151 Marr-Hildreth Edge Detection Algorithm ) Filter image with NxN Gaussian filter 2) Compute Laplacian using 3x3 mask 3) Find zero crossings of image from step 2 3x3 filter, thus zero crossing signs of two opposing neighbor pixels must differ Need to test: Left/right Up/down Both diagonals Usually use threshold for difference between pixels

152 Canny Edge Detector Superior performance, but more complex Algorithm: ) Gaussian smoothing 2) Compute gradient magnitude and angle images 3) Apply nonmaxima suppression to gradient magnitude images Thin edges by looking along edge normal for points with larger gradient magnitudes 4) Use double thresholding and connectivity analysis to detect and link edges

153 Canny Edge Detector: Double Thresholding High and low thresholds create two images: g g NH NL ( x, ( x, Eliminate from g NL all nonzero pixels from g NH : g NL g NH = strong edge pixels g NL = weak edge pixels y) y) g g N N ( x, ( x, y) y) T T ( x, y) g ( x, y) g ( x, y) NL H L NH

154 Canny Edge Detector: Connectivity Analysis g NH strong edge pixels marked as edge points immediately Mark weak edge pixels in g NL connected to strong pixels in g NH Add g NH to marked pixels in g NL to get final edge image

155 Linking Edges in General Breaks in edges because of noise, illumination, etc. To link: Local processing Link similar nearby edge pixels (magnitude, direction) May be able to use scan lines and fill gaps Regional processing Assumes we roughly know region of interest Link boundaries by fitting to model (i.e., polynomial) Global Processing No a priori knowledge must use global properties Example: Hough Transform