PATTERN CLASSIFICATION BY DISTANCE FUNCTIONS

Transcription

1 PATTERN CLASSIFICATION BY DISTANCE FUNCTIONS Dr. K.Vijayarekha Associate Dean School of Electrical and Electronics Engineering SASTRA University, Thanjavur Joint Initiative of IITs and IISc Funded by MHRD Page 1 of 8

2 Table of Contents 1. PATTERN CLASSIFICATION BY DISTANCE FUNCTIONS Minimum distance classification Single prototypes Multi-prototypes Algorithm... 6 Joint Initiative of IITs and IISc Funded by MHRD Page 2 of 8

3 1. PATTERN CLASSIFICATION BY DISTANCE FUNCTIONS 1.1 Minimum distance classification In this lesson we shall try to learn the minimum distance classification. The closeness of an incoming pattern to patterns of the possible pattern classes provides a measure in determining the pattern class of the pattern under consideration. As the classification is based on the minimum distance calculation, this method is called minimum distance classification procedure. Minimum distance procedure performs efficiently if pattern classes can be represented by a single prototype or by several prototypes around which the patterns form clusters. The simplest type of minimum distance classifier is the one wherein the patterns of all classes are very close to each other and each class can be represented by a single prototype. 1.2 Single prototypes Let there be m pattern classes in Rn denoted by C1, C2, Cm which are represented by the single prototype vectorsy1, y2,, ym. The distance between an incoming pattern x and the prototype vectors are The minimum distance classifier will classify x at Cj for which Dj is minimum. If the calculated distance is minimum for more j s the sample x is classified as belonging to the first C j for which a minimum was found or it is not classified at all. Minimizing D i 2 we get D i 2 = (x-y i ) T (x-y i ) = x T x 2x T y i +y T y i Joint Initiative of IITs and IISc Funded by MHRD Page 3 of 8

4 And as xx T can be removed, we minimize 2x T y i +y T y i This is equivalent to maximize 2x T y i - y T y i from which we can formulate the decision function as d i (x) = x T y i ½ y i T y i The classifier will classify the sample x as belonging to C i if d i (x) > d j (x) The decision functions will be linear d i (x) = w T x where x is given by (x1, x2, x3, ) T and w i = (w i1,w i2, w in, w i n+1 ) T and is determined by y i =( y i1, y i2, y in ) T where x is given by and is determined by w ij = y ij w i,n+1 = ½ y i T y i If there are two pattern classes, the decision boundary using minimum distance classification is d 12 (x) = d 1 (x) d 2 (x) = x T (y 1 -y 2 )-1/2 y 1 T y 1 +1/2 y 2 T y 2 = 0 Joint Initiative of IITs and IISc Funded by MHRD Page 4 of 8

5 This describes a hyperplane in normal direction to the vector y 1 -y 2. Substituting x = (y 1 +y 2 )/2 We get d 12 (x) = ½(y 1 +y 2 ) T (y 1 -y 2 )-1/2y 1 T y 1 +1/2y 2 T y 2 The decision boundary described by the previous equation is a hyperplane which is perpendicular to the vector connecting the two prototypes and bisects If more than two classes exist, the decision boundaries are piecewise linear. Let us consider a single prototype case wherein the number of pattern classes is three and are represented by the prototypes y1, y2, y3 The decision boundaries for the three prototypes y1, y2, y3 are the piecewise linear curves AOE, AOD and EOD respectively. A pattern sample can be classified successfully to any of the pattern classes provided it does not fall on the decision boundary Joint Initiative of IITs and IISc Funded by MHRD Page 5 of 8

6 1.3 Multi-prototypes Till now we saw the cases where the pattern classes were represented by single prototypes. Now we shall discuss about pattern classes consist of many clusters. Each cluster is represented by a single prototype which is the cluster center. Hence we can say that each pattern class is represented by a finite number of prototypes. The following figure represents two pattern classes with more than a single prototype representation. As previously stated in Example 3.2.1, the minimum distance classifier using single or several prototypes, classifies every incoming pattern in one of the existing classes, i.e. there are no indeterminate regions. This is an immediate consequence of the specific linear boundaries imposed by the minimum-distance approach. 1.4 Algorithm Consider a three-class problem in follows where each class is represented by its prototypes as Given the incoming pattern z = (1,-1) we get and the decision functions are Joint Initiative of IITs and IISc Funded by MHRD Page 6 of 8

7 And the decision boundaries are Since we classify (Fig ) A minimum distance classifier MC-MP Given a set of classes and prototypes in R n this algorithm uniquely classifies an arbitrary incoming pattern using the minimum distance approach with Eucidean norms Input : N the problem s dimension m- the number of classes. N i - number of prototypes for the i-th class for 1 [ the prototypes of the ith class for x- an income pattern Joint Initiative of IITs and IISc Funded by MHRD Page 7 of 8

8 Output : K-the number of class into which x is classified. Step 1 : For I 1,2,..m and find j(i,x) which yields Step 2 : Find k which satisfies Joint Initiative of IITs and IISc Funded by MHRD Page 8 of 8