Computer and Machine Vision Lecture Week 4 Part-2 February 5, 2014 Sam Siewert
Outline of Week 4 Practical Methods for Dealing with Camera Streams, Frame by Frame and De-coding/Re-encoding for Analysis (And Lab Reports!!) Deeper Dive on Color, Human Vision Characteristics Wrap-up On Convolution and Transformation Finish Reading Through Chapter 3 in CV (Image Processing and Transforms) Finish Reading in OpenCV through Chapter 6 Start asking Questions about Example Code as We go Introduction to Segmentation and Recognition Problem and Approaches Next Step is Histograms and Thresholds Goal is Recognition Then 3D Sam Siewert 2
FFMPEG FAQ Read it! Understand it! Use it! http://ffmpeg.org/faq.html Sam Siewert 3
Looking Beyond Images and Image Processing How to Get to Scene Segmentation and Recognition Sam Siewert 4
3D Cues in Human Vision 9 to 15 Recognized Human Cues James E. Cutting and Peter M. Vishton 1. Occlusion 2. Relative Size 3. Relative Density 4. Height in Visual Field (Relative) 5. Aerial Perspective (Sky blue hues) 6. Motion Perspective (Image Stabilization) 7. Convergence (Eye Orientation) 8. Accommodation (Eye Shape) 9. Binocular Disparity (Stereopsis - Depth, Diplopia Double Vision) 10. Textures 11. Linear Perspective (Vanishing points) 12. Brightness and shading 13. Kinetic depth, occlusion (structure revealed by rotation/motion) 14. Gravity (free falling objects) Sam Siewert 5
Outline for Rest of Semester Image Capture and Encode Pixel Encoding (YCrCb, srgb) Resolution (Down conversion) Frame Rate (Decimation) A/D Calibration (Flat-field Correction, Saturating with Known Stimulator) to Remove Artifacts of Detector Caused by Variations Pixel to Pixel ( Christmas Lights and Dark Currents) Post Capture ImageTransformations Enhancement with Pixel Convolution (e.g. Sharpen, Gamma Correction, Filtering) Background Elimination (Focus on Changing Pixels Only) Adjustment for Lighting Conditions and Color Encoding Image Parsing - Edge and Feature Detection, Segmentation Basic Raster with Threshold Detection Detection of Closed Areas/Regions (Segmentation) Image Understanding - Object Recognition and Tracking Based on Invariants (Shape, Color, Aspect Ratio,, Principal Components) Based on Models of Objects (Behavior and Signature) e.g. Eye Saccades, PCA Image Perception E.g. 3D Depth Maps, Registration, Hough Transform, SIFT/SURF Reasoning about a Scene E.g. Stereo Ranging with Multiple Cameras or Laser+Camera Depth Perception Proprioception Robotic or Human Knowledge of Effector Part Locations in Space Relative to Objects Sam Siewert 6
Image Transformations Transforms for Users Vs. Transforms for Machine Vision Sam Siewert 7
Basic Image Operations Graymaps - Nonlinear Designed for the Eye E.g. Gamma Correction Tri-stimulus Eye Follows gamma power function Designed for Machine Vision E.g. Background Elimination Filtering Edge Enhancement Thresholding http://en.wikipedia.org/wiki/file:gammacorrection_demo.jpg Sam Siewert 8
Image Parsing Edge and Feature Detection Segmentation Methods Sam Siewert 9
Edge and Feature Detection Edge Detection Sobel and Canny Feature Boundary Registration Common Feature Left/Right Eye Sam Siewert 10
Object Recognition Shapes Generalized Hough Transform Patterns SIFT (Scale Invariant Feature Transform) and SURF (Speeded Up Robust Features) http://docs.opencv.org/modules/nonfree/doc/nonfree.html Sam Siewert 11
Image Perception Tracking, Ranging, Estimation of State, and Understanding Sam Siewert 12
Image Perception Invariants and Perspective Ranging, Tracking, and State Estimation Application In Vehicle Vision Systems Other Applications of Interest Sorting - http://www.bestsorting.com/ Celestial Tracking, Attitude Estimation and Peak-up Optical Navigation and Robotics Sam Siewert 13
Behavior - Perception Gate of an Animal (Skeletonization) Behavior is a Cue for Recognition At Some Point, Intelligent Systems Machine Learning is Required Generalization Adaptation Sam Siewert 14
Simple Edge Detection First Step Parsing Text, Segmentation, Object Outlines Sam Siewert 15
Finding Shapes in Images Hough Transform First CV Algorithms Used in Robotics Where are all the Lines, Circles, Ellipses, Arbitrary Shapes? Sam Siewert 16
Skeletal Transform Behavioral Scene Perception 1) Camera Frame 3) Threshold 2) Graymap 4) Medial Thinning Sam Siewert 17