Machine Learning: Basic Principles

Transcription

1 Machine Learning: Basic Principles Teaching demonstration Kalle Palomäki Department of Signal Processing and Acoustics Aalto University

2 Content 1. Goal 2. Machine learning: definition 3. Classification an important machine learning approach 4. A machine learning problem Hands on problem solving Demonstration 5. Summary

3 Goal Part of introductory sessions adjusted to 20 minutes 4 th year students with no background in machine learning Start building understanding of machine learning by Concrete examples Solving simple hands on problems

4 Machine learning - definition Wikipedia: Machine learning deals with the construction and study of systems that can learn from data, rather than follow only explicitly programmed instructions

5 Common sense definition: machines that learn a little like the brains

6 Internet and machine learning - far beyond the single brains capacity

7 Machine learning categories Supervised learning Classification Unsupervised learning Clustering Reinforcement learning

8 Classifier

9 Classifier

10 Problem Lisa is a tailor

11 Lisa makes uniforms Salvation army uniforms: men have trousers, women skirts

12 Sometimes she makes mistakes These should be skirts.

13 Once she made a skirt for prince Charles!

14 Hip Waist Hip Waist

15 Here is Lisa s data waist (cm) hip (cm) gender Female Female ??? Male ??? Male ???

16 Female samples: Red Missing gender information: * * * Male samples : Blue

17 Some help to Lisa? Discuss in pairs 2 min: How would you approach this problem? What kind of algorithm would you design? Try to come up with some ideas please! Use the picture provided to assist your discussion

18 K-nearest neighbours algorithm 1. Determine K = number of nearest neighbours 2. Calculate the distance between test sample all the training samples Use euclidean distance measure:, 3. Sort the distances and determine nearst neigbours 4. Gather the categories of the nearest neighbors 5. Use the majority voting to predict the test sample class

19 Female samples: Red Missing gender information: * * * Male samples : Blue

20 Female samples: Red K = 3 Missing gender information: * * * Male samples : Blue

21 K-nearest neighbours algorithm 1. Determine K = number of nearest neighbours 2. Calculate the distance between test sample all the training samples Use euclidean distance measure:, 3. Sort the distances and determine nearst neigbours 4. Gather the categories of the nearest neighbors 5. Use the majority voting to predict the test sample class

22 Euclidean distance Euclidean distance, Training samples Test sample

23 Euclidean distance Training samples Eucidean distance, Training sample index Test sample

24 Euclidean distance Data dimension Training samples Eucidean distance, Training sample index Test sample Dimension index

25 Euclidean distance Data dimension M=2 Training samples Eucidean distance Training sample index, Test sample Dimension index

26 Female samples of training data Euclidean distance: d 1 Test sample * Male samples of training data

27 Female samples of training data d 2 Test sample * Male samples of training data

28 Female samples of training data d 3 Test sample * Male samples of training data

29 Female samples of training data d 4 Test sample * Male samples of training data

30 Female samples of training data Test sample * d 5 Male samples of training data

31 Female samples of training data Test sample * d 6 Male samples of training data

32 K-nearest neighbours algorithm 1. Determine K = number of nearest neighbours 2. Calculate the distance between test sample all the training samples Use euclidean distance measure:, 3. Sort the distances and determine nearest neigbours 4. Gather the categories of the nearest neighbors 5. Use the majority voting to predict the test sample class

33 Female samples of training data Test sample * 3 nearest neighbors Male samples of training data

34 K-nearest neighbours algorithm 1. Determine K = number of nearest neighbours 2. Calculate the distance between test sample all the training samples Use euclidean distance measure:, 3. Sort the distances and determine nearest neigbours 4. Gather the categories of the nearest neighbors 5. Use the majority voting to predict the test sample class

35 Female samples of training data Test sample * 3 nearest neighbors Male samples of training data All 3 neighbors were Male Class was male

36 Female samples of training data Test sample * 3 nearest neighbors Male samples of training data

37 Female samples of training data Test sample * 3 nearest neighbors 2 neighbors Female 1 neighbor Male More Females than Males Class is Female Male samples of training data

38 Classification problem Lisa has lost gender information of one of her customers, and does not know whether to make skirt or trousers. She is planning to throw a coin. Can you help her to make a better decision? The customer who is missing gender information: Gender , Waist 28, Hip 34, waist gender (cm) hip (cm) Male Male Female Female Molarius A, Seidell JC, Sans S, Tuomilehto J, Kuulasmaa K. (1999) "Waist and hip circumferences, and waist-hip ratio in 19 populations of the WHO MONICA Project", International Journal of Obesity and Related Metabolic Disorders :J. Internat. Association Study Obesity, 23:

39 Solution Gender waist (cm) hip (cm) distance Male (28-28) 2 +(34-32) 2 =4 Male (28-33) 2 +(34-35) 2 =26 Female (28-27) 2 +(34-33) 2 =2 Female (28-31) 2 +(34-36) 2 =13 Test sample 28, 34

40 Solution Gender waist (cm) hip (cm) distance Male (28-28) 2 +(34-32) 2 =4 Male (28-33) 2 +(34-35) 2 =26 Female (28-27) 2 +(34-33) 2 =2 Female (28-31) 2 +(34-36) 2 =13 Test sample 28, 34

41 Solution Gender waist (cm) hip (cm) Distance rank Male (28-28) 2 +(34-32) 2 =4 2 Male (28-33) 2 +(34-35) 2 =26 4 Female (28-27) 2 +(34-33) 2 =2 1 Female (28-31) 2 +(34-36) 2 =13 3 Test sample 28, 34

42 Solution Gender waist (cm) hip (cm) Distance rank belongs to the neighborhood (Yes or No) Male (28-28) 2 +(34-32) 2 =4 2 Yes Male (28-33) 2 +(34-35) 2 =26 4 No Female (28-27) 2 +(34-33) 2 =2 1 Yes Female (28-31) 2 +(34-36) 2 =13 3 Yes Test sample 28, 34

43 Solution Gender waist (cm) hip (cm) Distance rank belongs to the neighborhood (Yes or No) gender if in neigborhood Male (28-28) 2 +(34-32) 2 =4 2Yes Male Male (28-33) 2 +(34-35) 2 =26 4No Female (28-27) 2 +(34-33) 2 =2 1Yes Female Female (28-31) 2 +(34-36) 2 =13 3Yes Female Test sample 28, 34 Male 1 Female 2 Number of Female > Number of Male Class: Female

44

45 Summary We addressed briefly principles of machine learning 1. First we defined the machine learning 2. Classification as an important machine learning task 3. Solved a hands on problem of classification utilizing K- nearest neighbour algorithm Check out my website for These slides Exercise The code on the decision border calculations in previous slides

46 What next Supervised learning Classification Unsupervised learning Clustering Reinforcement learning

47 Face recognition

48 Speech recognition Spectrum over time for cat k a t

49 Searches