PA2 Introduction to Tracking. Connected Components. Moving Object Detection. Pixel Grouping. After Pixel Grouping 2/19/17. Any questions?

Transcription

1 /19/17 PA Introduction to Tracking Any questions? Yes, its due Monday. CS 510 Lecture 1 February 15, 017 Moving Object Detection Assuming a still camera Two algorithms: Mixture of Gaussians (Stauffer Grimson) ViBE (Barnich Van Droogenbroeck) But these algorithms only label pixels Need to group foreground pixels into regions Need to connect regions across images To reason about moving objects Connected Components Goal: connect pixels into regions Pixel Grouping Connected components to group pixels Put a bounding box around component May chose to add additional requirements Minimum size Minimum width/height Merge regions that are close enough After Pixel Grouping Once you have found the pixel groups in an image... How do you connect objects across frames? 5 6 1

2 /19/17 New Goal: Find a small image within a larger one Pearson s Correlation ( A ( x, y) A ) ( B ( x, y) B ) x,y ( A ( x, y) A ) ( B ( x, y) B ) x,y The image above is a small piece of the image to the right. But from where? x,y This is a very important equation 7 Brute-Force Translation Invariance 8 Computing Cross-Correlation In cross-correlation, the mask is correlated repeatedly to image windows To find a small image in a large one, slide the small one across the large, computing Pearson s correlation at every possible position. zero-mean unit length the mask zero-mean unit length the image compute the sliding dot product This is almost convolving the image with the mask 9 10 The process of slide correlate is called cross-correlation Complexity is O(nm) N = of pixels in image (w h) M = of pixels in the template (w h) Highly parallel (every position can be computed independently) Still sensitive to Rotation in-plane out-of-plane Scale Perspective In Engineering, convolving a normalized mask with the source image is called correlation Is this exactly the same as Pearson s correlation? Why or why not? This is the most common definition of correlation in image processing texts Statistical Cross-Correlation Naming conventions 11 1

3 /19/17 Simple Tracking So how do you track an object from one frame to the next? Find the moving object in frame t Cross-correlation the attention window against frame t+1 The best-matching position is where the object went With a minimum threshold, in case the object left the FOV Can we do better? Is there something better we can compare to than the raw attention window? Do we have to search all of frame t+1? Can we limit the search? Predict where the object is headed? Describe the object s motion? Exploit foreground information in frame t+1? What if we don t find the target in frame t+1? Three Ways of Thinking About Images. 1. As D signals : f(x,y) For video, 3D signals : f(x,y,t) E.g. correlation, cross-correlation. As points in image N-dimensional image space E.g. geometry of correlation space To come: eigenvector analysis 3. As continuous surfaces Today s viewpoint Surface in 1D 1D cross-section of simple image surface Image from S gr1.jpg Image as Surface View the image as a 3D surface For every (x,y) pixel location, the intensity can be thought of as the z (height) value. Color images are 5D surfaces - too hard to think about. Color can also be thought of as 3 3D surfaces Pretend the surface is continuous Gradients on Surfaces Every point on the image surface has a direction of maximum change (remember your multivariate calculus?), and a magnitude of change in that direction

4 /19/17 Image Edges To find direction and magnitude of change, compute the mag. in any orthogonal directions and interpolate Again, this assumes a continuous surface WLOG choose the X Y directions: dx(x,y) = I(x,y) - I(x-1,y) dy(x,y) = I(x,y) - I(x,y-1) The edge magnitude and orientation is: Δ = dx + dy θ = cos 1 dy % Δ ( Estimating Edge Orientation Problem: images are not continuous surfaces Estimates of dx, dy based on grid sampling Here is the problem, do you agree:? I ( x, y) I ( x +1, y) = I ( x 1, y) I ( x, y) estimating derivatives from two values is highly error prone Accurate Edge Estimation We want to compute a real-valued function The partial derivatives dx dy All we have to work with are samples at equidistant points So model the function in terms of its Taylor series expansion: f ( x + h) = f ( x) + h1 1! ( ) + h f! x f!! x! ( ) + Accurate (II) Look at equations for f(x+h) and f(x-h): f ( x + h) = f ( x) + h f! ( x) + h f!! x f ( x h) = f ( x) h f!( x) + h f!! x subtract equation from 1 ( ) f ( x h) = h f!( x) + f x + h and solve for f! ( ) = f ( x + h) f ( x h) f! x h + ( ) + (1) ( ) () 1 Accurate (III) So the best ±1 mask is [-1,0,1], from d dx f ( x, y ) = f ( x +1, y ) f ( x 1, y) As an exercise, the best ± mask is [1,- 8,0,8,-1] ( ) = f ( x +, y ) +8 f ( x +1, y) 8 f ( x 1, y ) + f ( x, y) d dx f x, y 1 Stable Edges (III) Of course, pixels are still noisy and pixels are related to adjacent rows. The Sobel Edge Masks " Dx % " Dy % 3 CS 510, Image Computation, Ross Beveridge Bruce Draper 4 4

5 /19/17 Sobel Explanation " 1 1 % In any row (dx) or column (dy), this is a [1,0,1] mask to estimate the derivative [1,,1] weights approximate a σ=1 Gaussian. Over-constrained fit of a plane to 9 points Minimizes the sum-of-squared error Multiply result by 1/4 : range -55<x< 55 Multiply results by 1/8 : 8-bit response Sobel Edge Magnitude " % dx + dy And why edges Structure 3 Sources of edges Related Concept - Laplacian Surface discontinuity Second order derivatives Zero crossing idea not widely used today Where one surface ends and another begins 6 8 Illumination discontinuity Shadows Sudden change in surface orientation Remember your surface reflectance equations Specular reflections ditto Surface marking Change in material and/or surface color

6 /19/17 Say it with Code - Sobel Play with this Tutorial!