Cameras Images Images Edges Talked about images being continuous (if you blur them, then you can compute derivatives and such). Two paths: Edges something useful Or Images something besides edges. Images something besides edges. May still want invariant properties, so find things that do not depend (too much) on brightness of the image, or which direction the light comes from. Goals for today: feature detection cross-correlation ssd (sum of squared differences) What is Texture? (or, a different kind of edge) Schools of thoughts on texture Texture: repeated elements, subject to randomization of their location, size, color orientation Julesz: Nth-order joint empirical densities of textons Bergen-Adelson,Malik, Tuner: Multi-scale filter banks, wavelets Non-parametric: Image prototypes Texture problems Representation Choice of model, Parametric vs. Non-parametric Method of measuring: Filter Banks, Local Statistics Similarity Invariant properties: (some feature all parts of the texture share, or some function that is constant over the texture). Choice of distance function Texture indexing Synthesis Choices of texture models Method of image generation, deterministic vs. stochastic Filter bank for texture representation
Gaussian Filter Image filtering: smoothing R(x,y) = I *g =sum_(u,v) I(x-u,y-v) g(u,v) Difference of Gaussian (DOG) Filter Banks Dot (DOG) filter response Odd symmetric filter outputs (derivatives)
Even symmetric filter ( bars) Filter Banks. (automatically constructed) Segmentation through filter response Simple filters can model complex shapes: (wavelets). Other computations on images. Image Type Conversions Many functions work on just one channel Run on each channel independently Convert from color grayscale weighting each channel by perceptual importance
Unix man page for ppmtopgm NAME ppmtopgm - convert a portable pixmap into a portable graymap SYNOPSIS ppmtopgm [ppmfile] DESCRIPTION Reads a portable pixmap as input. Produces a portable graymap as output. The output is a "black and white" rendering of the original image, as in a black and white photograph. The quantization formula used is.299 r +.587 g +.114 b. QUOTE Cold-hearted orb that rules the night Removes the colors from our sight Red is gray, and yellow white But we decide which is right And which is a quantization error. Thresholding Grayscale Binary: Choose threshold based on histogram of image intensities Image Histogramming Connected Components Uniquely label each n-connected region in binary image 4- and 8-connectedness Example: Connected Components Binary Operations Dilation, erosion Dilation: All 0 s next to a 1 1 (Enlarge foreground) Erosion: All 1 s next to a 0 0 (Enlarge background) courtesy of HIPR Original Eroded Dilated courtesy of Reindeer Graphics
Moments: Region Statistics Back to filter like things. Zeroth-order: Size/area First-order: Position (centroid) Second-order: Orientation Template matching Define a template a model of the object to be recognised Define a measure of similarity between template and similar sized image region Similarity Measure dissimilarity between image f[i,j] and template g[i,j] Place template on image and compare corresponding intensities Need a measure of dissimilarity Last is best... max f g [ i, j] R [ i, j] R f g SAD 2 ( f g) [ i, j] R SSD Geometric Image Comparison: SSD Given a template image I T and an image I, how to quantify the similarity between them? Vector difference: Sum of squared differences (SSD) Correlation for Template Matching Note that SSD formula can be written: When the last term is big, the mismatch is small the dot product measures correlation: By normalizing by the vectors lengths, we are measuring the angle between them
1 Normalized Cross-Correlation Input Template Output Shift template image over search image, measuring normalized correlation at each point Local maxima indicate template matches 100 80 60 40 20 from Jain, Kasturi, & Schunck 0 120 26 51 76 101 126 151 176 201 226 251 276 301 326 351 376 401 Non-Normalised Correlation 426 Position Matching Criteria Cross Correlation Handling illumination changes Why Normalized cross-correlation is used? The image to search (I 2 ) The window to search The maximum cross correlation score The cross correlation score I 1 I 2 Good and Bad Features. The aperture problem (cont )??? Hard to track vertically. Easy to track in both directions. Filter response is same if you move the filter up and down, (no local maxima)
Pattern Recognition Demo Watch the next slide very carefully it will only appear for a fraction of a second. Pattern Recognition Demo Pattern Recognition: Definition What did you see? Pattern Recognition: Definition Theories: 1. Template Matching Theory Ability to recognize and identify incoming stimuli Compares external stimuli to stored information about past experience Example: recognizing handwriting or type Other examples Compare stimuli to templates stored in memory When match found, stimulus is identified Template for every pattern Examples Bar code Numbers on check
Theories: 1. Template Matching Theory Problems: Variability (of form, style, size, orientation) Context effects Theories: 1. Template Matching Theory Problems: Different interpretations of the same stimulus Theories: 1. Template Matching Theory Problems: Different interpretations of the same stimulus (other examples) Theories: 2. Feature Detection based Recognition? Each pattern is stored as a set of features and a label The stimulus compared to these features and the best match is found Label stimulus as the one that matches the set of features that produces the greatest amount of feature overlap Next Time: Comparisons of whole images (and how you automatically define interesting filter banks).