Machine Learning: Basic Principles

Machine Learning: Basic Principles Teaching demonstration Kalle Palomäki Department of Signal Processing and Acoustics Aalto University

Content 1. Goal 2. Machine learning: definition 3. Classification an important machine learning approach 4. A machine learning problem Hands on problem solving Demonstration 5. Summary

Goal Part of introductory sessions adjusted to 20 minutes 4 th year students with no background in machine learning Start building understanding of machine learning by Concrete examples Solving simple hands on problems

Machine learning - definition Wikipedia: Machine learning deals with the construction and study of systems that can learn from data, rather than follow only explicitly programmed instructions

Common sense definition: machines that learn a little like the brains http://www.paranormalpeopleonline.com/boskop-man-big-brains-and-increased-intelligence/ http://oldentech.files.wordpress.com/2010/07/1028528_29880053.jpg

Internet and machine learning - far beyond the single brains capacity http://www.slate.com/blogs/future_tense/2014/10/24/internet_sleep_new_research_from_usc_shows_internet_activity_changes_in.html

Machine learning categories Supervised learning Classification Unsupervised learning Clustering Reinforcement learning

Classifier

Problem Lisa is a tailor... http://ecx.images-amazon.com/images/i/51f9cnkx90l._sy300_.jpg http://upload.wikimedia.org/wikipedia/commons/3/39/leonardo_da_vinci_043-mod.jpg

Lisa makes uniforms Salvation army uniforms: men have trousers, women skirts http://www.bilerico.com/2009/03/army%20uniforms.jpg

Sometimes she makes mistakes These should be skirts.

Once she made a skirt for prince Charles! http://i.dailymail.co.uk/i/pix/2009/05/21/article-1186234-050b9cb2000005dc-834_224x423.jpg

Hip Waist Hip Waist

Here is Lisa s data waist (cm) hip (cm) gender 29.6 34.4 Female 28.9 34.4 Female 31.3 34.5??? 30.8 33.7 Male 29.8 34.5??? 32.5 33.6 Male 30.6 34.4???.........

Female samples: Red Missing gender information: * * * Male samples : Blue

Some help to Lisa? Discuss in pairs 2 min: How would you approach this problem? What kind of algorithm would you design? Try to come up with some ideas please! Use the picture provided to assist your discussion

K-nearest neighbours algorithm 1. Determine K = number of nearest neighbours 2. Calculate the distance between test sample all the training samples Use euclidean distance measure:, 3. Sort the distances and determine nearst neigbours 4. Gather the categories of the nearest neighbors 5. Use the majority voting to predict the test sample class http://people.revoledu.com/kardi/tutorial/knn/

Female samples: Red Missing gender information: * * * Male samples : Blue

Female samples: Red K = 3 Missing gender information: * * * Male samples : Blue

Euclidean distance Euclidean distance, Training samples Test sample http://people.revoledu.com/kardi/tutorial/knn/

Euclidean distance Training samples Eucidean distance, Training sample index Test sample http://people.revoledu.com/kardi/tutorial/knn/

Euclidean distance Data dimension Training samples Eucidean distance, Training sample index Test sample Dimension index http://people.revoledu.com/kardi/tutorial/knn/

Euclidean distance Data dimension M=2 Training samples Eucidean distance Training sample index, Test sample Dimension index http://people.revoledu.com/kardi/tutorial/knn/

Female samples of training data Euclidean distance: d 1 Test sample * Male samples of training data

Female samples of training data d 2 Test sample * Male samples of training data

Female samples of training data d 3 Test sample * Male samples of training data

Female samples of training data d 4 Test sample * Male samples of training data

Female samples of training data Test sample * d 5 Male samples of training data

Female samples of training data Test sample * d 6 Male samples of training data

K-nearest neighbours algorithm 1. Determine K = number of nearest neighbours 2. Calculate the distance between test sample all the training samples Use euclidean distance measure:, 3. Sort the distances and determine nearest neigbours 4. Gather the categories of the nearest neighbors 5. Use the majority voting to predict the test sample class http://people.revoledu.com/kardi/tutorial/knn/

Female samples of training data Test sample * 3 nearest neighbors Male samples of training data

K-nearest neighbours algorithm 1. Determine K = number of nearest neighbours 2. Calculate the distance between test sample all the training samples Use euclidean distance measure:, 3. Sort the distances and determine nearest neigbours 4. Gather the categories of the nearest neighbors 5. Use the majority voting to predict the test sample class http://people.revoledu.com/kardi/tutorial/knn/

Female samples of training data Test sample * 3 nearest neighbors Male samples of training data All 3 neighbors were Male Class was male

Female samples of training data Test sample * 3 nearest neighbors Male samples of training data

Female samples of training data Test sample * 3 nearest neighbors 2 neighbors Female 1 neighbor Male More Females than Males Class is Female Male samples of training data

Classification problem Lisa has lost gender information of one of her customers, and does not know whether to make skirt or trousers. She is planning to throw a coin. Can you help her to make a better decision? The customer who is missing gender information: Gender ------, Waist 28, Hip 34, waist gender (cm) hip (cm) Male 28 32 Male 33 35 Female 27 33 Female 31 36 http://www.dcs.gla.ac.uk/~srogers/firstcourseml/matlab/chapter5/knnexample.html#1 Molarius A, Seidell JC, Sans S, Tuomilehto J, Kuulasmaa K. (1999) "Waist and hip circumferences, and waist-hip ratio in 19 populations of the WHO MONICA Project", International Journal of Obesity and Related Metabolic Disorders :J. Internat. Association Study Obesity, 23:116-125.

Solution Gender waist (cm) hip (cm) distance Male 28 32 (28-28) 2 +(34-32) 2 =4 Male 33 35 (28-33) 2 +(34-35) 2 =26 Female 27 33 (28-27) 2 +(34-33) 2 =2 Female 31 36 (28-31) 2 +(34-36) 2 =13 Test sample 28, 34

Solution Gender waist (cm) hip (cm) Distance rank Male 28 32 (28-28) 2 +(34-32) 2 =4 2 Male 33 35 (28-33) 2 +(34-35) 2 =26 4 Female 27 33 (28-27) 2 +(34-33) 2 =2 1 Female 31 36 (28-31) 2 +(34-36) 2 =13 3 Test sample 28, 34

Solution Gender waist (cm) hip (cm) Distance rank belongs to the neighborhood (Yes or No) Male 28 32 (28-28) 2 +(34-32) 2 =4 2 Yes Male 33 35 (28-33) 2 +(34-35) 2 =26 4 No Female 27 33 (28-27) 2 +(34-33) 2 =2 1 Yes Female 31 36 (28-31) 2 +(34-36) 2 =13 3 Yes Test sample 28, 34

Solution Gender waist (cm) hip (cm) Distance rank belongs to the neighborhood (Yes or No) gender if in neigborhood Male 28 32 (28-28) 2 +(34-32) 2 =4 2Yes Male Male 33 35 (28-33) 2 +(34-35) 2 =26 4No Female 27 33 (28-27) 2 +(34-33) 2 =2 1Yes Female Female 31 36 (28-31) 2 +(34-36) 2 =13 3Yes Female Test sample 28, 34 Male 1 Female 2 Number of Female > Number of Male Class: Female

Summary We addressed briefly principles of machine learning 1. First we defined the machine learning 2. Classification as an important machine learning task 3. Solved a hands on problem of classification utilizing K- nearest neighbour algorithm Check out my website for These slides Exercise The code on the decision border calculations in previous slides http://users.spa.aalto.fi/kpalomak/demonstration_session

What next Supervised learning Classification Unsupervised learning Clustering Reinforcement learning

Face recognition http://cs.nyu.edu/~roweis/data.html

Speech recognition Spectrum over time for cat k a t

Searches