Computer Vision Prof. Sung-Hyuk Cha Fall of 2002 School of Computer Science & Information Systems Artificial Intelligence 1
Perception Lena & Computer vision 2
Machine Vision Pattern Recognition Applications 3
Iris authentication 4
Face Recognition System Each person has different faces. Face Recognition System Query? Face DB 5
Head Pose Recognition left strt rght up Complex Pattern Recognition Applications Sargur N. Srihari 520 Lee Entrance STE 202 f 5 primary number f 6 street name f 2 state abbr. f 7 secondary designator abbr. f 3 5-digit ZIP Code f 8 secondary number Amherst NY 14228-2583 f 1 city name Delivery point: 142282583 f 4 4-digit ZIP+4 add-on 6
Speech Recognition System ( 1 2 x x x x x x x x x x x x x x x x x x f, f,..., f ) f, f,..., f ) f, f,..., f ) f, f,..., f ) f, f,..., f ) f, f,..., f ) d ( 1 2 d ( 1 2 Applications d ( 1 2 d ( 1 2 d ( 1 2 d LCD Pen tablet Microphone Digital Camera biomouse Fingerprint scanner Vital Sign monitor 7
Measurements brightness, length Salmon1 = ( 12, 16 ) Salmon2 = ( 11, 20 ) Salmon Bass1 = ( 7, 6 ) Bass2 = ( 3, 4 ) Truth features Bass 8
9
Decision theory (cost) 10
Distributions and Errors Salmon Decision boundary Bass identified as salmon salmon identified as bass Bass Parametric Univariate Dichotomizer (a) length (b) lightness (c) width (a) (b) (c) Type I 9 % 7 % 5 % Type II 39 % 27 % 26 % 11
Multivariate Analysis Nearest Neighbor Classifier bright ness? = salmon Salmon Bass length 12
Nearest Neighbor Classifier reference set testing set R = r1 = 5, 7 r2 = 3, 8 r3 = 10, 16 r4 = 12, 14 Bass Bass Salmon Salmon q = 4, 6 T = t1 = 4, 6 t2 = 2, 5 t3 = 11, 17 t4 = 14, 12 Bass Bass Salmon Salmon Bass Bass Salmon Bass rn = 14, 15 Salmon tn = 14, 17 Salmon Bass too slow for users to wait for the output. Performance is evaluated by using a testing set. Machine Learning (Linear function) bright ness? = salmon Salmon Bass Y > ax + b length 13
Artificial Neural Network synapse x 1 (t) nucleus axon x 2 (t) w 2 w 1 Σ a(t) y y=f(a) a O(t+1) w 0 dendrites x n (t) w n the biological neuron the artificial neuron Machine Learning (Linear function) reference training set set testing set R = r1 = 5, 7 r2 = 3, 8 r3 = 10, 16 r4 = 12, 14 Bass Bass Salmon Salmon Y > ax + b T = t1 = 4, 6 t2 = 2, 5 t3 = 11, 17 t4 = 14, 12 Bass Bass Salmon Salmon Bass Bass Salmon Bass rn = 14, 15 Salmon tn = 14, 17 Salmon Bass extremely fast. No need to load the training data during the classification. Performance is evaluated by using a testing set. Performance is not as good as the NN classifier s. 14
Non-Linear case Salmon bright ness Bass Y > ax + b length Non-Linear case Salmon bright ness Bass length NN is better. will learn artificial neural network which is non-linear function. 15
Human Brain Artificial Neural Network f 1 f 2 f 3 f 4 Class f 5 f 6 f 7 Fully Connected, feed forward, back-propagation multi-layer Artificial neural network (11-6-1) (ANN). 16
17
Purpose of Pattern Recognition Predict unseen future instance. Generalization. Inductive step. width Generalizability (statistical inferece) width training set universe length length width validating set length 18
Inferential Statistics 1. Inferential Statistics is inferring a conclusion about population of interest from a sample. - need a procedure for sampling the population. - need a measure of reliability for the inference. 2. If error rate in a random sample set is the same as in universe, then the procedure is a sound inferential statistical procedure. 3. If error rate in one random sample set is the same as in another random sample set, then the procedure is sound. Generalization δf2 δf1 Universe 19
Sampling & learning δf2 δf1 Sample 1 Testing on another sample δf2 δf1 Sample 2 20
Generalization δf2 δf1 Universe Multiple classification f2 class 1 class 3 class 2 f1 Classes = {class 1, 2, 3} 21
22
Template for PR Applications 1. Data acquisition: a. Recruit subjects. b. Modality interface (Scanning, picturing, recording, etc). 2. Feature Extraction: a. Raw data to feature vectors. b. Involves image/ voice/ signal processing techniques. 3. Training a classifier: a. Design a classifier (e.g., ANN). b. Enter the training (& validating) feature vector set(s). 4. Classification system: a. embed the ANN engine to your actual program (Java/C) b. User interface for the Final Product. Further Pattern Recognition http://www.csis.pace.edu/~scha/pr Fast Nearest Neighbor Search Algorithms Decision Tree Statistical Pattern Recognition. Artificial Neural Network. Clustering etc. 23
Decision Tree outlook temperature humidity windy play sunny hot high false no sunny hot high true no overcast hot high false yes rainy mild high false yes rainy cool normal false yes rainy cool normal true no overcast cool normal true yes sunny mild high false no sunny cool normal false yes rainy mild normal false yes sunny mild normal true yes overcast mild high true yes overcast hot normal false yes rainy mild high true no Clustering (a) d a k g e j h f i c b (b) d a k g e j c h i f b (c) 1 2 3 (d) a b c... 0.4 0.1 0.5 0.1 0.8 0.1 0.3 0.3 0.4 g a c i e d k b j f h 24
Terminology Classification: The process of assigning one of a limited set of alternative interpretations to (the generator of) a set of data. Often requires the steps of the computation of relative probabilities (or a quantity related to them) followed by the application of a decision rule. All classification processes can be evaluated in terms of "detection" and "misclassification" rates. Terminology Computer Vision: Compter Vision is the subject area which deals with the automatic analysis of images for the purposes of quantification or system control (often mimicking tasks which humans find trivial). It is to be distinguised from "Image Processing" which deals only with the computational processes applied to images, including enhancement and compression, but does not deal with abstract representation for the purposes of reasoning and interpretation. Compter Vision can be seen as the inverse of Computer Graphics, though generally the representations and methods of this area are not of use in Computer Vision due to the incomplete and therefore ambiguous nature of images. This requires prior knowledge to be used in order to obtain robust scene interpretation. 25
Terminology Machine Vision: Like "computer vision" but generally more closely associated with its use in robotics. Pattern Recognition Pattern recognition is the process of assigning a pattern classification to a particular set of measurements, normally represented as a high dimensional vector. This is normally done within the context of "probability theory", whereby a particular set of assumptions regarding the expected statistical distribution of measurements is used to compute classification probabilities which can be used as the basis for a decision such as the "Bayes decision rule". There are several popular forms of classifier including "k-nearest neighbour", "parzen windows", "mixture methods" and more recently "artificial neural networks". Terminology Images: An image is two dimensional spatial representation of a group of "objects" (or "scene") which exists in two or more dimensions. It is an intuitive way of presenting data for computer interfaces in the area of graphics, but in machine vision it may be defined as a continuous function of two variables defined within a bounded (generally rectangular) region. Histograms A histogram is an array of non negative integer counts from a set of data, which represents the frequency of occurance of values within a set of non-overlapping regions. 26
Features & Class Features class dark blob hole slant width skew ht pixel hslope nslope pslope vslope int int int real int real int int int int int int.95.49.70.71.50.10.51.92.13.47.32.21.94.49.75.70.50.11.53.84.26.54.35.18.94.49.67.74.50.10.45.85.23.48.32.22.93.72.33.47.50.21.28.30.66.60.42.10.93.74.33.48.50.22.26.30.60.59.45.10.93.79.36.54.50.18.27.32.60.59.52.09 S S S S S S.92.30.61.66.60.11.35.49.70.71.57.10.94.42.72.66.60.11.32.49.67.74.53.10.94.40.75.67.60.12.34.49.75.70.54.11.96.30.60.59.50.10.21.30.66.60.36.10.95.32.60.59.50.09.22.30.60.59.39.10.95.30.66.60.50.10.21.32.60.59.34.09 B B B B B B Representation length a = (12,6,-5) b = (16,9,10) c = (19,7,-10) 40 35 30 25 20 15 10 5 0-5 -10 10 12 width a 14 16 18 20 22 b c 24 lightness 5 6 7 8 9 10 27
Image Classification??? Image Indexing & Retrieval 28
Query by Image Content? Acute myeloid leukemia Acute myeloid leukemia Acute myeloid leukemia Dissimilarity (distance) / Similarity D( ) =?, S( ) =?, 29
Overview Image processing vs. computer vision Human vision & illusion. Basic Image Processing Machine Vision Applications. Histogram based Image Indexing & Retrieval. Digital Image Processing vs. Computer Vision There are no clear distinction Image processing Applications where humans are in the loop. Humans supply the intelligence Image Analysis - extracting quantitative info. Size of a tumor distance between objects facial expression Image restoration. Try to undo damage needs a model of how the damage was made Image enhancement. Try to improve the quality of an image Image compression. How to convey the most amount of information with the least amount of data 30
Digital Image Processing vs. Computer Vision Computer Vision Take the human out of the loop The computer supplies the intelligence Where does the computer get it s intelligence? Human Vision 31
Cerebral Cortex Human Vision Monocular Visual Field: 160 deg (w) X 175 deg (h) Binocular Visual Field: 200 deg (w) X 135 deg (h) 32
The figure-ground Problem The Bunny/Duck illusion Mouth Mouth 33
More illusions Squares or lines? More illusions: How many colors? 34
More illusions: parallel line More illusions 35
More illusions More illusions 36
More illusions Photometry Concerned with mechanisms for converting light energy into electrical energy. World Optics Sensor Signal Digitizer Digital Representation 37
. Binary image i 1 2 3 4 5 6 7 j 1 2 3 4 5 6 7 Grey image E(x,y) : Electrical video signal Optics Video Camera Image Plane Image L(x,y) A/D Converter and Sampler I(i,j) Digital Image 22 34 22 0 18 Grayscale Image Data Computer Memory 38
. Color image E(x,y) : Electrical video signal Optics Video Camera Image Plane Image L(x,y) B(i,j) G(i,j) R(i,j) Blue Channel Green A/D Channel Converter Red A/D Channel Converter A/D Converter 22 3422 0 18 Digital Image Computer Memory HSL Color Space Lightness Hue (color) Saturation (white) 39
Color Contrast Stretching 40
Linear Stretching 255 OUTPUT 0 INPUT 255 Histogram Equalization Adjust peaks and plains 41
False Color Warping http://www.doctorwarp.com/index.php?id=23&flx=world 42
Compression Mosaics 43
Stereo Stereo vision 44
Noise Removal salt pepper Zooming Important for size invariance 45
Rotation Important for rotation invariance Subtraction 46
Connected Components/ Image Labeling Goal: To find clusters of pixels that are similar and connected to each other How it works: Assign a value to each pixel Define what similar values mean e.g., 10 +/- 2 Determine if like pixels are connected 4- connected 8-connected Connected Components/ Image Labeling 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 1 1 1 2 2 0 0 1 1 2 2 0 0 1 A A A A A A A B B A A A A A A C A A A D D C C A A D D C C A 47
Connected Components/ Image Labeling 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 1 1 1 2 2 0 0 1 1 2 2 0 0 1 A A A A A A A B B A A A A A A B A A A C C B B A A C C B B A Segmentation 48
Segmentation Edge Detection convolution B = 1 1 1 mask 1 + 8 1 1 1 1 49
Handwriting Each person writes differently. Handwriting Analysis Taxonomy Analysis of Handwriting Recognition Examination Personality identification (Graphology) On-line Off-line Writer Identification Writer Verification Natural Writing Forgery Disguised Writing 50
Pattern Recognition The End See U all next week. 51