Fuzzy Logic & Data Processing

Transcription

1 Fuzzy Logic & Data Processing Lecture notes for Modern Method of Data Processing (CCOD) in 2014 Akira Imada Brest State Technical University, Belarus (last modified on) December 17, 2014 (still under construction) 1

2 (Contemporary Method of Data Processing) 2 1 Fuzzy Set Theory 1.1 Fuzzy set vs. Crisp set Examples of crisp set 0 < x < 10 x=12 Examples of fuzzy set {x is much smaller than 10} {x is close to 12} PART I Fuzzy Set Arithmetics Beer is either of {very-cold, cold, not-so-cold, warm} Membership function How x is likely to be A is expressed by a function called membership function. Usually it is described as µ A (x). For example, a possible membership function for a fuzzy expression {x is close to 12} will be 1 µ(x) = (1) 1 + (x 12) 2 See Figure AND and OR in Fuzzy Logic In the logic of crisp set A and B and A or B are defined as in Figure 3. In Fuzzy Logic, on the other hand, the membership function of A and B and A or B are specified in various way, but most popular ones are: and respectively. µ A B (x) = min{µ A (x), µ B (x)} (2) µ A B (x) = max{µ A (x), µ B (x)}, (3)

3 (Contemporary Method of Data Processing) 3 Figure 1: Examples of membership function {x is much smaller than 10} (right) and {x is close to 12} (left). Figure 2: AND and OR in crisp set. Figure 3: AND and OR in fuzzy set.

4 (Contemporary Method of Data Processing) 4 To be more concrete the membership function of x is closer to 4 AND/OR x is closer to 5 is like a Figure 4. Figure 4: Membership function of x is closer to 4 OR x is closer to 5 Very cold or pretty cold beer. (µ(x) is defined on temperature). Assume we like very cold beer or pretty cold beer and now we have a beer the temperature of which is 3 degree. Then how is the beer likely to be our prefered one? Note that this operation of OR was possible because both of the two membership function is defined on the same domain temperature. Then what if two membership functions are defined on different domains, such as age and height? Young and tall. For example, We cannot draw the membership function of young and tall on the 2-dimensional coordinate any more. (1) 3-D graphic (z = µ is defined on x = age and y = height) (2) Matrix representation koko

5 (Contemporary Method of Data Processing) 5

6 (Contemporary Method of Data Processing) 6 height \ age IF-Then rule in Fuzzy Logic In Fuzzy Logic, the membership of IF A Then B is specified also in many way. Here, let s take it as Mamdani s proposal Larsen s proposal If he is young then my love to him is strong. µ A B (x) = min{µ A (x), µ B (x)} (4) µ A B (x) = µ A (x) µ B (x) (5) If he is young and tall then my love to him is very strong. 1.2 How to express multidimensional membershipfunction

7 (Contemporary Method of Data Processing) 7 PART II Fuzzy Controller 2 Fuzzy Controller Let s construct a virtual metro and control trains by fuzzy controller. A goal We now assume x is speed of my car, y is distance to the car in front, and z is how strongly we push brake-pedal. Then let s controll my car with a set of rules, like IF x is high AND y is short THEN z should be strong IF x is medium AND y is long THEN z should be medium IF x is medium low or x is medium AND y is long THEN z should be weak IF x is low or x is medium low AND y is short or y is medium hort THEN z should be medium weak etc. Then the results will be plotted like in the Figure below. Figure 5: An example of the goal of Fuzzy controller

8 (Contemporary Method of Data Processing) Virtual metro system with two trains in a loop line We study Fuzzy Controllar via a simulation of virutal metro with one loop line on which two Train A and B run. To simplify we don t assume stations. That is, both trains always run. The speed of these trains are denoted as x A and X B. The distance from train A to train B is denoted as y A and from train B to train A is y B. Note that x A + y B is constant (length of the loop line). Speed will be controlled by the distance to the train in front via its break. The shorter the distance, the storonger the break in order to avoid a collision. 3 Vertual metro Let s create a virtual metro system with 2 cars on a loop line with 1000 pixels in which 4 stations 1, 2, 3 and 4 at pixel number 0, 250, 500 and 750, respectively. Figure 6: To de-fuzzify strength of break Exercise 1 Create your own simulation of metro with one loop on which two trains A and B run, using graphics. 6 parameters x A, x B, y A, y B, z A, z B, should also be desplayed on the screen. The simulation might be started with x A = x B, y A = y B, z A = z B = Let s design a set of rules for driving a train membership function E.g., when we say speed is 13, it is likely to be fast with 75% certain and medium with 25%.

9 (Contemporary Method of Data Processing) 9 Figure 7: An example of 5 membership function for speed. Figure 8: An example of 5 membership function for speed. Figure 9: An example of 5 membership function for speed.

10 (Contemporary Method of Data Processing) One point of break Figure 10: An example of 5 membership function for speed One plaine of break Figure 11: An example of 5 membership function for speed D control of break Figure 12: An example of 5 membership function for speed.

11 (Contemporary Method of Data Processing) To calculate one point of break value for fixed speed and distance under one rule We now assume our rule is Then IF speed is fast AND distance is short THEN break is strong speed µ distance µ break µ total µ Figure 13: An example of 5 membership function for speed. We now know the center of gravity of break is at at 6. So the point is (13, 125, 6).

12 (Contemporary Method of Data Processing) 12 Figure 14: An example of 5 membership function for speed.

13 (Contemporary Method of Data Processing) To calculate one point of break value for fixed speed and distance under two rules Rule 1 Rule 2 Total x µ y µ z µ µ1 x µ y µ z µ µ2 µ-final

14 (Contemporary Method of Data Processing) To calculate one point of break value for fixed speed and distance under ten rules Rule 1 Rule 2 Rule 10 Total x y z µ x y z µ x y z µ µ

15 (Contemporary Method of Data Processing) 15 Figure 15: An example of 5 membership function for speed.

16 (Contemporary Method of Data Processing) 16 PART III Fuzzy Classification 4 Classify data by a rule set Assume we classiy M data to be classified by using N features. x 1, x 2, x 3,, x N. A rule such as If x 1 = A 1 and x 2 = A 2, and, and x N = A N then class = ω p. classifies the data to one class ω p. A i is called attribute. For instance, (i) IF x i = 30, (ii) IF 15 < x i < 20, (iii) IF x i is Large, or (iv) IF x i is Female, etc. The first two are called crisp, second is fuzzy, and fourth is called categorical. Let s take an example. IFx 1 = 20g AND 10cm < x 2 < 20cm AND x 3 = Green, AND x 4 = F ruits THEN this is apple.

17 (Contemporary Method of Data Processing) 17 5 A benchmark Iris database As an example target here, we classify Iris flowers. Iris flower dataset 1 is made up of 150 samples consists of three species of iris flower, that is, setosa, versicolor and virginica. Each of these three families includes 50 samples. Each sample is a four-dimensional vector representing four attributes of the iris flower, that is, sepal-length, sepal-width, petal-length, and petal-width. All data are given as crisp as below. x 1 x 2 x 3 x 4 class (Setosa) (Setosa) (Setosa) (Versicolor) (Versicolor) (Versicolor) (Virginica) (Virginica) (Virginica) 1 University of California Urvine Machine Learning Repository. ics.uci.edu: pub/machine-learning-databases.

18 (Contemporary Method of Data Processing) Let s visualize data Each data is a point in 4-dimensional space, i.e., x 1, x 2, x 3 and x distribution of each of 4 attributes First, let s see how each x i (i = 1, 2, 3, 4) of these dataset distributed What if we had only two attributes? Demension reduction from 4 to 2 by Samon mapping Figure 16: Sammon mapping of iris data.

19 (Contemporary Method of Data Processing) Non-fuzzy approach If a 1 1 x 1 < b 1 1 and a 1 2 x 2 < b 1 2 and a 1 3 x 3 < b 1 3 and a 1 4 x 4 < b 1 4 then class = 1. Else if a 2 1 x 1 < b 2 1 and a 2 2 x 2 < b 2 2 and a 2 3 x 3 < b 2 3 and a 2 4 x 4 < b 2 4 then class = 2. Else clase = 3.

20 (Contemporary Method of Data Processing) 20 6 TS-Fuzzy Formula 6.1 Singleton Consequence R i : If x 1 is A i 1 and x 2 is A i 2 and and x n is A i n then y is g i. 6.2 Linear Regression Consequence R i : If x 1 is A i 1 and x 2 is A i 2 and and x n is A i n then y = a 1 1 x 1 + a 1 2 x a i nx n + b i. 6.3 Triangle vs. Gaussian membership function (x c)2 µ(x) = exp{ } w Example - Iris data classification Singleton Consequence with triangle membership Apply the TS-fuzzy formula above to the iris flawer database, assumming the folloing p = 3 rules and membership functions 2. R 1 : IF x 1 is short AND x 2 is long AND x 3 is short AND x 4 is short THEN y = R 2 : IF x 1 is medium AND x 2 is small AND x 3 is medium AND x 4 is medium THEN y = R 3 : IF x 1 is long AND x 2 is medium AND x 3 is long AND x 4 is long THEN y=2.95. where each of the membership functions are adjusted as Figure 1 below. Apply TS-formula above and then estimated class is: 1... if ŷ < 1.5 y = 2... if 1.5 ŷ < if 3.0 ŷ Exercise 2 How many y out of 150 data are collect? 2 Taken frome the draft of H. Roubos et al. (2001) IEEE Transactions on Fuzzy Systems, Vol. 9, No. 4, pp

21 (Contemporary Method of Data Processing) Lenear Regression Consequence with Gaussian membership Apply the TS-fuzzy formula above to the iris flawer database, assumming the folloing p = 3 rules and membership functions 3. a 1 1 a 1 2 a 1 3 a 1 3 a 2 1 a 2 2 a 2 3 a 1 3 a 3 1 a 3 2 a 3 3 a 1 3 = and b 1 b 2 b 3 = Gaussian membership function is defiend here as: (x c)2 µ(x) = exp{ } w 2 where c and w represent center and width of distribution, respectively. 6.5 Neuro Fuzzy approach Fuzzy Neural Network Implementation. The procedure described in the previous sub-subsection can be realized when we assume a neural network architecture such as depicted in Fig. 2. The 1st layer is made up of n input neurons. The 2nd layer is made up of H groups of a neuronal structure each contains n neurons where the i-th neuron of the k-th group has a connection to the i-th neuron in the 1st layer with a synaptic connection which has a pair of weights (w ik, σ ik ). Then k-th group in the second layer calculates the value µ k (x) from the values which are received from each of the n neurons in the first layer. The 3rd layer is made up of m neurons each of which collects the H values from the output of the second layer, that is j-th neuron of the 3rd layer receives the value from k-th output in the second layer with the synapse which has the weight ν kj How it learns? Castellano et al. [?] used (i) a competitive learning to determine how many rules are needed under initial weights created at random. Then, in order to optimize the initial random 3 Taken frome M.H. Kim et al. (2004) A novel appreoach to design of Takagi-Sugeno fuzzy classifier. Joint International Conference on Soft Computing and Intelligent Systems and International Symposium on Advanced Intelligent Systems.

22 (Contemporary Method of Data Processing) 22 weight configuration, they use (ii) a gradient method performing the steepest descent on a surface in the weight space employing the same training data, that is, supervised learning. Here, on the other hand, we use a simple genetic algorithm, since our target space is specific enough to know the network structure in advance, i.e., only unique rule is necessary. Our concern, therefore, is just obtaining the solution of weight configuration of the network. That is to say, all we want to know is a set of parameters w ik, σ ik and ν kj (i = 1, n), (k = 1, H), (j = 1, m) where n is the dimension of data, H is the number of rules, and m is the number of outputs. Hence our chromosome has those n H m genes. Starting with a population of chromosomes whose genes are randomly created, they evolve under simple truncate selection where higher fitness chromosome are chosen, with uniform crossover and occasional mutation by replacing some of a few genes with randomly created other parameters, expecting higher fitness chromosomes will be emerged. These settings are determined by trials and errors experimentally. 6.6 Multi input multi output

23 (Contemporary Method of Data Processing) 23 Figure 17: TS-model singleton conswquence. How to estimate class of a data.

24 (Contemporary Method of Data Processing) 24 Figure 18: Triangle membership functions representing small, medium and large for x 1 (up left), x 2 (Up right), x 3 (bottom left) and x 4 (bottom left). Figure 19: Data for 12 triangle membership function above, indicating (start peak end) of each triangle for small, medium and large for each of x 1, x 2, x 3 and x 4.

25 (Contemporary Method of Data Processing) 25 Figure 20: Gaussian membership functions representing small, medium and large for x 1 (up left), x 2 (Up right), x 3 (bottom left) and x 4 (bottom left).

26 (Contemporary Method of Data Processing) 26 Figure 21: Architecture of the proposed fuzzy neural network which infers how an input x = (x 1, x n ) is likely to belong to the j-th class by generating outputs y j each of which reflect the degree of the likeliness. In this example, a 20-dimension data input will be inferred to which of the 3 classes the input belongs by using 2 rules.

27 (Contemporary Method of Data Processing) 27 PART IV Time series data forecasting with TS-Fuzzy formula 7 Two different formulae 7.1 Forcasting a value from its history Assume y(t) is a value of a variable y at time t such as maximum price of a stock during a day. Then T-S formula for singleton consequeance is as follows 4. R i : If y(t 1) is A i 1 and y(t 2) is A i 2 and and y(t n + 1) is A i n then y(t) is g i. 7.2 Forcasting a value from its history R i : If x 1 (t) is A i 1 and x 2 (t) is A i 2 and and x n (t) is A i n then y(t) is g i. 4 Taken from Sheta, A. F. () Forecasting the Nile river flow using fuzzy logic model.

28 (Contemporary Method of Data Processing) 28 PART V Fuzzy Relation 8 Relation between two sets In this section we study fuzzy expressions such as at least middle-aged, brighter than average, more or less expensive and younger than about 20. First, let s recall Cartesian product X Y in which both X and Y is a set. Let me take an example. Assume now X = {1, 2} and Y = {a, b, c} then X Y = {(1, a), (1, b), (1, c), (2, a), (2, b), (2, c)}. Then relation is defiened over Cartesian product X Y, that is, a subset of X Y. In other words relation is a set of ordered pair in which order is important. Generally it is defined over multipel set, like X 1 X 2 X n, but here we think of only product of two set, and call it binary relation. To visualize we can plot µ R (X, Y ) 3-D Cartesian space. Example 1... X = {1, 2}, Y = {2, 3, 4}, R : X < Y Let s think of it as a crisp logic, that is, the value is 1 (yes) or 0 (no). Then membership function of this relation will be: X \ Y Then what about the relation R : x y. Let s think of this example with fuzzy logic. Example 2... X = {1, 2}, Y = {2, 3, 4}, R : X Y X \ Y /3 1/ /3 1/3 3 2/3 1 2/3

29 (Contemporary Method of Data Processing) 29 The values are just examples. Further more we think of X and Y as a continuous values instead of integer. Then membership function is a surface instead of just 9 points, over X Y coordinate. We now proceed to examples where we use fuzzy linguistic expression instead of numbers. First, these matrices are not necessarily rectangular. For example: Example 3... X = {Brest, London, BuenosAires} Y=Tokyo, NewYork, Minsk, Johanesburg R: very far. X \ Y Tokio New York Minsk Johanesburg Brest London Buenos Aires Try to fill those blancs by yourself. Example 4... X = {green, yellow, red}, Y = {unripe, semiripe, ripe}. Imagine an apple. First, with a crisp logic. A red apple is usually ripe but a green apple is unripe. Thus: X \ Y unripe semiripe ripe green yellow red Now, secondly, with a fuzzy logic. A red apple is provably ripe, but a green apple is most likely, and so on. Thus, for example: X \ Y unripe semiripe ripe green yellow red

30 (Contemporary Method of Data Processing) Combine two fuzzy relations We now return to the previous example of tomato. X \ Y unripe semiripe ripe green yellow red This is the relation of two sets: X = {green, yellow, red} and Y = {unripe, semiripe, ripe} Let s call this relation R 1. Then we think a similar but new Relation. and Let s call this relation R 2. Y = {unripe, semiripe, ripe} Z = {sour, sour sweet, sweet} X \ Y sour sour-sweet sweet unripen semiripe ripe If we combine these two relations R 1 and R 2 by the formula µ R (x, z) max y X {min{µ R(x, y), µ R (y, z)}}, the result is: This relation could be expressed by our daily language like If tomato is red then it s most likely sweet, possibly sour-sweet, and unlikely sour.

31 (Contemporary Method of Data Processing) 31 X \ Y sour sour-sweet sweet red yellow green If tomato is yellow then probably it s sour-sweet, possibly sour, maybe sweet. If tomato is green then almost always sour, less likely sour-sweet, unlikely sweet. Or, we could say: Now tomato is more or less red, then what is taste like? 9 Clustering by fuzzy relation In the previous section, we studied Fuzzy relation between two sets X and Y R(X, Y. Here, in this section, we extended it to fuzzy relation between X and X, in which our goal is to cluster the elements of X into n clusters such that n X = We start with an intuitive proximity relation which should follow only two condition (i) Reflectivity i=1 C i and (ii) Simmetry µ R (x, x) = 1... x X. µ R (x, y) = µ R (y, x)... x, y X. Then we use an algorithm 5 to obtain similarity relation, which also follows (iii) Transivity The µ R (x, y) max z X {min{µ R(x, z), µ R (z, y)}}... x, y X. 5 Firstly by Tamura, S. et al. (1971) Pattern classification based on fuzzy relations. IEEE Transactions on Systems, Man, and Cybernetics, Vol.SMC-1, No. 1. and then extended by Yang, M-S. et al. (1999) Cluster analusis based on fuzzy relations. Fuzz Sets and Systems, Elsevier, Vol. 120 pp

32 (Contemporary Method of Data Processing) 32 Algorithm 1 1. Calculate a max-min similarity-relation R = [a ij ] 2. Set a ij = 0 for all a ij < α and i = j 3. Select s and t so that a ij = max{a ij i < jand i, j I}. When tie, select one of these pairs at random WHILE a st 0 DO put s and t into the same cluster C = {s, t} ELSE [4.] ELSE all indices I into separated clusters and STOP 4. Choose u I \ C so that i C a iu = max { a ij a ij 0} j I\C i C When a tie, select one such u at random. WHILE such a u exists, put u into C = {s, t, u} and REPEAT [4.] 5. Let I = I \ C and GOTO [3.]

33 (Contemporary Method of Data Processing) 33 Example Exercise 3 Starting from the following proximity-relation R ( 0), apply the algorithm above. Assume now α = R (0) = An example solution By repeating R (n+1) = R (n) R (n) till R (n) = R (n+1). In this way, similarity-relation R (n) will be calculated as: R (n) = Now apply [1.] and [2.]

34 (Contemporary Method of Data Processing) 34 First, set I = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10} and C = { }. Then 3. Now a 18 = a 26 = a 5 10 = 0.9 are maximum and a 18 is randomly selected. Then C = {1, 8}. 4. a 12 + a 82 = a 14 + a 84 = 1.5 are maximum and j = 4 is randomly selected. Then C = {1, 8, 4}. 4. a 12 + a 42 + a 82 = 2.1 is maximum, then C = {1, 8, 4, 2}. 4. There are no j such that a 1j +a 2j +a 4j +a 8j is maximum. Then final C = {1, 8, 4, 2}. a 16 + a 26 + a 46 + a 86 = = 2.2 seems maximum but actually not because a 46 = 0 Note that i C a iu = max j I\C { i C a ij a ij 0} 5. Let I = {3, 5, 6, 7, 9, 10} 3. a 5 10 = 0.9 is maximum. Then renew C as {5, 10}. 4. a 59 + a 10 9 = 1.5 is maximum. Then C = {5, 10, 9}. 4. There are no j in {3, 6, 9} such that a 5j + a 9j + a 10j is maximum. Then final C = {5, 10, 9}. 5. Let I = {3, 6, 7}. 3. Now a 36 = a 37 = a 67 = 0. Then {3}, {6}and {7}are three separated clusters. In fact, a 33 a 36 a a 63 a 66 a 67 = a 73 a 76 a So i {3,6,7} a iu = max j {3,6,7} { i C a ij a ij 0} does not exit any more. In this way, when α = 0.55, we have 5 clasters {1, 8, 4, 2},{5, 10, 9}, {3}, {6} and {7} are obtained.

35 (Contemporary Method of Data Processing) Other formula for combination Other than (i) max-min formula µ R (x, y) = max z X {min{µ R(x, z), µ R (z, y)}} we have a cuple of other formulae: (ii) max-prod (iii) max-avg (iv) max- µ R (x, y) = max z X {µ R(x, z) R (z, y)}} µ R (x, y) = max z X {(µ R(x, z) + µ R (z, y))/2} µ R (x, y) = max z X {max{0, µ R(x, z) + µ R (z, y) 1}}