Image processing & Computer vision Xử lí ảnh và thị giác máy tính

Transcription

1 Image processing & Computer vision Xử lí ảnh và thị giác máy tính Detection and Recognition 2D et 3D Alain Boucher - IFI

2 Introduction In this chapter, we introduce some techniques for pattern detection and recognition It is not a complete course in pattern recognition, but more focused parts for computer vision Object detection or recognition based on features models global appearance local apparence... 2

3 Introduction Two types of recognition Generic recognition example: search for a human in the image Specific recognition example: search for Alain Boucher in the image 3

4 Some generalities for pattern recognition in images 4

5 Global appareance Classical methods for object recognition are based on the global appearance measured using features Classical approach (1) Image segmentation (2) Feature extraction from segmented objects (3) Object recognition 5

6 Example of recognition What are the objects shown in this image? Is there a ball? What is a ball? Round??? Source : Patrick Hébert, Vision numérique, Université Laval (Québec, Canada). 6

7 Segmentation / recognition We must do segmentation before recognition; But we need recognition to do segmentation! The fish here is impossible to segment automatically if we ignore first what we are looking for! Source : Patrick Hébert, Vision numérique, Université Laval (Québec, Canada). 7

8 How to describe a fish? Source : Patrick Hébert, Vision numérique, Université Laval (Québec, Canada). 8

9 What is a chair? 9

10 Feature representation Measures of properties: color, texture geometry: length, height, ratios, etc. build a vector ---> statistical approaches Elementary primitives: line segments, cercle archs perceptual grouping build a graph ---> structural approaches 10

11 Pattern features (attributes) Center of gravity Cx and Cy Surface(area) [S] (in pixels) Perimeter [P]: measured as the sum of distances between pixels (optional : sum of 1 and 2 ) Mean of gray levels (or colors) Variance of gray levels (or colors) Advice: using another color space that RGB (HSV or Lab for examples) can give better results Shape factor: 4 πs Ff = 2 P Line : Ff = 0 Circle : Ff = 1 11

12 Examples of features Axes of inertia Angular profile Concavities 12

13 Classification methods There exist many classification methods: Nearest neighbors Decision trees Neural networks Support vector machines Bayesian methods See a pattern recognition course for more 13

14 Class discrimination p zebra image p non zebra image Frontier for decision Zebra Non-zebra Source : Jana Kosecka, CS 682 Computer Vision, George Mason University (USA) 14

15 Bayesian classification Classifier based on the Bayes law: P classi features = P features class i P class i P features P classi features P features class i P class i Knowing the features of an object, what is the probability that it belongs to a class i [P(classi features)]? For this, we must know the probability function of the features for the class i [P(features classi)] the probability for the class i [P(classei)] : allow to give a different probability depending on classes (otherwise default value for all) the probability for the feature vector [P(features)] is the same for all classes : we can take it out here 15

16 Gaussian probability function Probability function for features = Gaussian 1 x m j 2 /2σ 2j P classi features = e P class i 2 πσ j We define the means, such as m1 and m2, and the standard deviations, such asl σ1 and σ2, for each class Source : Gonzalez and Woods. Digital Image Processing. chap. 12, Prentice-Hall,

17 Statistic classifier - Bayes Decision frontier 17

18 Classification risk Decision frontier Decision frontier Bayesian risk All the existing classification methods must make a choice between two or more object classes Object classification in a class is often done following the probability to belong in each class This creates a decision frontier between classes and leads to a classification error Source : Gonzalez and Woods. Digital Image Processing. chap. 12, Prentice-Hall,

19 Example: Iris recognition Example: 3 types of Iris are classified using their petal lengths and widths virginica versicolor Source : Jacques-André Landry, Vision robotique, ETS (Montréal, Canada) setosa 19

20 Pattern recognition We have three classes ω1, ω2 and ω3 Each flower is described using two features Iris virginica, Iris versicolor, Iris setosa Petal length, petal width [] x1 X= x2 There are some differences of petal length and width between all classes There are also some variability within each class Source : Jacques-André Landry, Vision robotique, ETS (Montréal, Canada) 20

21 Pattern recognition Source : Gonzalez and Woods. Digital Image Processing. chap. 12, Prentice-Hall,

22 Pattern recognition The setosa type is well differentiate from the two others It is difficult to differentiate the two other types without error It is mainly a problem with the selection of good features It is important to select discriminative features! Source : Jacques-André Landry, Vision robotique, ETS (Montréal, Canada) 22

23 Minimal distance classification Computing vector distance between an unknown object and a reference object A possible distance is the Euclidean distance What is the shape vector for the reference object? May be a shape vector from a unique reference object May be a mean vector for a set of objects m j = 1 N j x ω x j j = 1, 2,...,W j Nj = number of shape vectors for the class ωj 23

24 Minimal distance classification For a new object to recognize, we search for the class with the minimal distance with that object? d1 m1 = (4.3, 1.3)T d2 m2 = (1.5, 0.3)T Source : Gonzalez and Woods. Digital Image Processing. chap. 12, Prentice-Hall,

25 Minimal distance classification We can also define a decision frontier between classes ω1 m1 = (4.3, 1.3)T m2 = (1.5, 0.3)T ω2 Problem with axis dispersion Source : Gonzalez and Woods. Digital Image Processing. chap. 12, Prentice-Hall,

26 Mahalanobis distance Better than the Euclidean distance, we can use the Mahalanobis distance: DMahalanobis = ( x x) Ψ ( x x) T Takes into account the variance for each feature and the variance between features Ψ is the feature covariance matrix 1 N T Ψ= ( x x )( x x ) k k N i 1 k =1 The covariance matrix is very important here it contains all data dependencies between classes 26

27 Mahalanobis distance The distance is the same along each elliptic line Source : Patrick Hébert, Vision numérique, Université Laval (Québec, Canada). 27

28 Bayesian classifier with covariance Bayesian classifier Gaussian distribution P classi features P features class i P class i P classi = Ni N Number of examples for the class i Total number of examples P features class i = µ=x [ 1 1 T 1 exp x μ Ψ x μ n /2 2 2π det Ψ ] Multi-dimension Gaussian n dimensions 1 N T Ψ= ( x x )( x x ) k k N i 1 k =1 Covariance matrix 28

29 Choice for features The main problem is to choose the good features allowing to differentiate classes see the example with Iris flower The features identified by a human expert of the domain (e.g. botanist) are not always the best one for a computer algorithm A known method to choose important features is Principal Component Analysis (PCA) Also known as Karnuhen-Loeve transform or other names 29

30 Examples of correlation based on correlation 30

31 Classification by correlation Correlation between a sub-image w(x,y) and an image f(x,y) w(x,y) is of size J x K f(x,y) is of size M x N J M and K N The correlation between f(x,y) and w(x,y) is: c(x,y)= s f (s,t) w (x + s, y + t) t 31

32 Classification by correlation c(x,y)= s f (s,t) w (x + s, y + t) t Source : Gonzalez and Woods. Digital Image Processing. Prentice-Hall,

33 Classification by correlation We look for the maximum correlation in the image Source : Gonzalez and Woods. Digital Image Processing. chap. 12, Prentice-Hall,

34 Examples of recognition appearence-based 34

35 Appearance-based recognition The value of each pixel can be considered as a feature in a vector For an image of size NxM : X = { img(0,0), img(0,1), img(0,m), img(1,0), img(n,m) } Source : David Kriegman, CSE152 Introduction to Computer Vision, UC San Diego (USA),

36 Pixel vectors (comparison) For a new image, we compare it to other existing images Example with face recognition Source : David Kriegman, CSE152 Introduction to Computer Vision, UC San Diego (USA),

37 Other example for face recognition Recognition of face expression (Martinez 2002) 37

38 Different example for skin detection Skin can be modeled: with its color mean or histogram by choosing a right colorspace with its texture (few) IF p(pixels x) > Threshold THEN skin IF p(pixels x) < Threshold THEN no IF p(pixels x) = Threshold THEN?? Source : Statistical color models with application to skin detection,, M.J. Jones et J. Rehg, Proc. Computer Vision and Pattern Recognition,

39 Specific vision problems with object recognition 39

40 Problems with recognition (1) Position of the camera Translation, rotation, scaling, streching Depth, orientation (2) Variation in illumination, in colors, in shadows Different light or different reflects (3) Occlusions One part of the object may not be visible (4) Variations inside a class Faces, flowers, all living beings (5) Non-rigid motion Human body, hands, ++ Object recognition is most of all based on appearance, and not on semantic! 40

41 (1) Problème of orientation Michel-Angelo Source : Jana Kosecka, CS 682 Computer Vision, George Mason University (USA) 41

42 (2) Problem of illumination Source : Jana Kosecka, CS 682 Computer Vision, George Mason University (USA) 42 slide credit: S. Ullman

43 (3) Problem with occlusion Magritte, 1957 Source : Jana Kosecka, CS 682 Computer Vision, George Mason University (USA) 43

44 (4) Variability inside a class Source : Jana Kosecka, CS 682 Computer Vision, George Mason University (USA) 44

45 (5) Problem with non-rigid bodies Xu, Beihong 1943 Source : Jana Kosecka, CS 682 Computer Vision, George Mason University (USA) 45

46 Industrial vision In industries, we can find some vision system using recognition performing very well Big constraints on light type of objects position of objects It is in the general case that gives problems (and a lot of research) Source : George Bebis, Computer Vision Review, CS491E/791E: Computer Vision (Spring 2004), University of Nevada (USA). 46

47 References (see also course web page) Jana Kosecka, CS 682 Computer Vision, George Mason University (USA). David Kriegman, Appearance-based recognition, CSE152 Introduction to Computer Vision, University of California, San Diego (USA), Hoang Thanh Lam. Indexation et recherche d'images par utilisation de mots visuels. Travail Pratique Encadré (TPE), IFI promo 13, George Bebis, CS491E/791E: Computer Vision (Spring 2004), University of Nevada (USA). Computer Vision Review: 47