Computer and Machine Vision Lecture Week 5 Part-2 February 13, 2014 Sam Siewert
Outline of Week 5 Background on 2D and 3D Geometric Transformations Chapter 2 of CV Fundamentals of 2D Image Transformations and Convolutions (CV Chapters 3 & 4) Introduction to More Advanced 2D Transformations Bottom Up C without OpenCV Simple 2D Center-of-Mass Tracking Introduction to 2D Pattern Matching Next Time Sam Siewert 2
Thresholding Image Analysis for Segmentation and Tracking Sam Siewert 3
Simple Thresholds? Specular Reflection An Approximation to Segmentation Dividing a Scene into Constituent Objects Dark Foreground Objects on a Light Background Light Foreground Objects on a Dark Background (Laser Spot) How is Threshold Selected? This is the Challenge and Limitation Good Enough? Simple Machine Vision Problems E.g. Tracking Bright Objects Edge Detection Provides an Alternative Bounding Boxes and Approximate COM Filtering, Background Elimination, and PSF Enhancements Overall, an Unsolved Problem Even the Human Eye Can be Fooled http://en.wikipedia.org/wiki/file:tso_kiagar_lake_ladakh.jpg Bright Object Segmentation M.C. Escher http://en.wikipedia.org/wiki/file:escher_waterfall.jpg http://en.wikipedia.org/wiki/file:escher%27s_relativity.jpg Sam Siewert 4
Histogram Analysis For Simple Cases with Controlled Lighting, and Laser Emission, Intuition is Correct (Saturated Red Peak) Green and Blue Bands are Bi-Modal as is Luminosity (Gray) Histogram is a Good Guide for Threshold Value Selection Sam Siewert 5
Clutter Background Elimination Clutter Gives Us Multi-Modal Histograms No Obvious Threshold Function in Luminosity May be Ok with Laser in Red Band (Bright Enough) May Lose Spot Based on Local Background Sam Siewert 6
Does Image Enhancement Help? Filtering Not Much (Median) Sharpen and Background Elimination Works Well for Bright Target Tracking Can Be Used for Stereo Ranging E.g. 3D Analysis of Space (Active Laser Projection and Ranging) Sharpen Sam Siewert 7
Two Methods Adaptive Threshold Tile the Image and Use Threshold Based on Localized Histograms (Error Prone) Use Background Elimination (Best), Especially for Moving Targets Sam Siewert 8
Variance-Base Threshold What if the Histogram Bins are Expanded to Detect Threshold Classes? This is the Key Concept of Variance- Based Methods Variable Size Bins (Not Just New Bin Size) Probability of Pixel with Graylevel is p i =sum-at-level-i / N Partitioning of Histogram Optimized for Within-Class, Between, and Total Variance Compute Summation of Pixel Probability in Bin (π 0 ) or Not in Class (π 1 ) Use Normal Statistical Methods for Computation of Variance Between Classes Define Multiple Classes (Thresholds) Based on Maximizing Variance Sam Siewert 9
Entropy Threshold Metric for Re-binning the Histogram with a Range of Thresholds is Entropy (Flatness) Higher Entropy of Pixels in Each Sub-bin of Class are Evenly Distributed Lower Entropy of they Spike in the Class Find Thresholds Where Entropy in Each Class is Maximized Segments with Highest Order in the System Sam Siewert 10
Edge Detection Sobel Case Study Sam Siewert 11
OpenCV / GIMP Top-Down Approach Experiment with GIMP on Single Frames (Exemplars) Install OpenCV on Ubuntu Linux (Should be done!) Use Library Calls to OpenCV for Transformations in Video Embed and Optimize in Solution (Consider Methods to Port OpenCV to Embedded Systems C Implementations Bottom Up) Other Highly Useful Top-Down Tools/APIs Octave, MATLAB, Mathematica, ffmpeg and GIMP http://karytech.blogspot.com/2012/05/opencv-24-on-ubuntu-1204.html Sam Siewert 12
Bottom Up - Basic Edge Detection Examines Intensity Gradient True Edge vs. Noise Impulse Most Often Employs Convolution Mask (3x3 or Larger) Like PSF DG (Differential Gradient) Uses X and Y Direction Mask Nearest Neighbors or Pixels on 45 degree lines P4 P3 P2 P5 P0 P1 P6 P7 P8 P4 P3 P2 P5 P0 P1 P6 P7 P8 TM (Template Matching) Uses Many Directions Sam Siewert 13
Intensity Differential Gradient Use Mask Intensity Change Edge Function Step Slanted Step Smooth Step δi = sqrt(δi 2 x + δi y2 ) Edge Magnitude From Intensity P(x) to P(y) Approximate as δi x + δi y or max( δi x, δi y ) http://store.elsevier.com/computer-and-machine-vision/e_-r_-davies/isbn-9780123869081/ Sam Siewert 14
Application of Sobel DG Operator Compute X and Y Gradients, Where Image is A Compute (Approximate) Gradient Magnitude G {Q} = {G} from {A} (Gradient Magnitude Image) http://en.wikipedia.org/wiki/sobel_operator Sam Siewert 15
More on Edges Read CV Page 210 to 219 Sobel DG Edge Finder Why is Canny Better? Detailed Treatment of Sobel and Canny in E.R. Davies Computer and Machine Vision, 4 th Edition, Pages 111 to 131 You are Responsible for Concepts Only and Use of DG Edge Operators with OpenCV and GIMP Sam Siewert 16
Template Matching Up to 8 Masks With Angles Differing by 45 degrees Experiment in GIMP (Prewitt for Example) Sam Siewert 17
Differential Edge Operators E.g. Sobel Edge Operator Fast Enough for Real-Time Simpler than Canny Sam Siewert 18
Lab #3 Bottom Up - Threshold Function Design for Maximum Edge Extent (Bounding Box) Sam Siewert 19
Lab Goals Do Not Use OpenCV in Solution Do Use OpenCV to Explore Options Implement Tracker in C/C++ Using Threhshold Analysis and Simple Center of Mass (Centroid) for Bright Spot Sam Siewert 20
Simple Graymap Threshold Raster Step-1: Conversion to Graymap - Convert PPM (RGB) to PGM by Taking G band Only from Each Frame (Ok to Use OpenCV, but Try Writing your own code) Step-2: Background Elimination Step-3: Noise Filtering - Apply Sharpen PSF and/or Median Filter to Remove Any Residual Noise Step-4: Threshold Function Design - Perform Threshold Histogram Analysis and Determine Threshold Function for Laser Spot Edge Transition Step-5: Find Spot Bounding Box - Raster Image to find Maximum distance between Left and Right Edge Transition on X, First edge transition on Y, and last edge transition on Y Step-6: Compute COM - X-bar and Y-bar Step-7: Mark COM with Saturated (White=255) Crosshairs Sam Siewert 21
0,0 Bounding Extents Y bar = (Y bottom Y top ) / 2 + Y top X bar = (X right X left ) / 2 + X left Slightly Different than True Center of Mass, but Approximate for Radially Symmetric Object X left,y top X right,y bottom n,m (639, 479) Sam Siewert 22
Pattern Generation / Recognition Concepts Sam Siewert 23
Complex Patterns in Nature Fibonacci Golden Ratio Fermat s spiral Ulam s Spiral Pascal striangle http://en.wikipedia.org/wiki/file:aeonium_tabuliforme.jpg Sam Siewert http://en.wikipedia.org/wiki/file:sunflower_spiral.png 24
Basic Patterns For Next Week Lines Hough Linear Circle / Elipse Hough Circular Patterns Generalized Hough, SIFT/SURF Sam Siewert 25