Machine Perception of Music & Audio. Topic 10: Classification

Transcription

1 Machine Perception of Music & Audio Topic 10: Classification 1

2 Classification Label objects as members of sets Things on the left Things on the right

3 There is a set of possible examples Each example is represented by an n-tuple of attribute values 1 1 There is a target function that maps X onto some finite set Y The DATA is a set of duples <example, target function values> D Find a hypothesis h such that... Learning a Classifier X = { x,... x } 1 n x =< a,..., ak f : X Y x, f ( x ) >,... < x, f ( x ) > } 1 m = { < 1 m x, h( x) f ( x) >

4 Learning a Classifier There is a set of possible examples: X = Dots in my powerpoint Each example is represented by an n-tuple of attribute values! x 1 = < x-position, y-position, size, color, etc > There is a target function that maps X onto some finite set of classes Y The DATA is a set of duples <example, target function values> Learn a hypothesis h such that... f : X! Y, Y={+, -} D = {<! x 1,+ >,... <! x m,! >} x, h( x) f ( x)

5 Different Classifiers Different classifications need different classifiers. Left Right Blue Red

6 The catch The things you can measure are typically NOT the labels you care about. You have to learn to classify things without being able to directly observe the class labels. 6

7 Feature Selection is KEY! How things cluster depend on what you are measuring. Are the features you choose correlated with the categories you care about? Is that correlation a coincidence? 7

8 Which of these go together? 8

9 Which of these go together? Beard Head Hair 9

10 Which of these go together? Movie Star Rock Star 10

11 Spurious correlation All the Hispanics in my training data are beardless. Beard beard = not Hispanic? NO!! Hispanic 11

12 Problem: Group into 2 categories

13 Problem: Group into 2 categories

14 Feature Extraction Choice of features is task dependent Bassoon vs. Tuba Cepstral coefficients? Spectral tilt? Pitch class Chroma? Pitch track? Harmonic Sounds vs. White Noise Autocorrelation?

15 Mel Frequency Cepstrograms bassoon tuba mel freq cepstral coef frame number frame number

16 Picking Good Features nd MFCC th MFCC st MFCC st MFCC Blue = Bassoon Red = Tuba

17 The best dimension? 150 BASSOON: 12th MFCC for 12bassoon th MFCC 100 COUNT TUBA: 12th MFCC 12 th for MFCC tuba COUNT

18 Linear Separators The idea: Draw a line that splits the data into the two groups we re interested in. For 1-d data, this line is a point For n-d data, this line is a hyperplane. How do we do this?

19 Finding a good linear separator 1. Collect some statistics on your data (which?) 2. Pick the best statistics (how?) 3. Define what you mean by classification error 4. Pick a hyperplane to split the points into 2 sets 5. Measure the classification error (how?) 6. Pick a different hyperplane (how?) 7. Repeat 5 through 6 until a condition is met. (what condition?)

20 Using a linear separator Take a new example See which side of the line it is on Give it the label of the stuff on that side. Done. Bryan Pardo, Machine Learning: EECS 349 Fall

21 Classifying with labeled clusters 1. Build the classifier Collect some statistics on your labeled data (which?) Pick the best statistics as your dimensions (how?) Find the center of gravity of all the points with a given label (Euclidean distance?) 2. Use the classifier to label an example Measure the distance between the example and each labeled cluster s center of gravity Give the example the label of the nearest cluster.

22 Nearest Neighbor Classifier Example of memory-based (a.k.a case-based) learning The basic idea: 1. Get some example set of cases with known outputs e.g diagnoses of infectious diseases by experts 2. When you see a new case, assign its output to be the same as the most similar known case. Your symptoms most resemble Mr X. Mr X had the flu. Ergo you have the flu. 22

23 Single nearest neighbor Given some set of training data D and query point = { < 1 m 1. Find the nearest member of the data set to the query x, f ( x ) >,... < x, f ( x ) > } 1 m x, predict f ( x ) q q x distance function = arg min( d( x, )) nn x q x D 2. Assign the nearest neighbor s output to the query h( xq ) = f ( xnn) Our hypothesis 23

24 A Univariate Example Find closest point. Give query its value x = arg min( d( x, )) nn x q x D f ( xq ) = f ( xnn) f (x) x 24

25 What makes a memory based learner? A distance measure Nearest neighbor: typically Euclidean Number of neighbors to consider Nearest neighbor: One A weighting function (optional) Nearest neighbor: unused (equal weights) How to fit with the neighbors Nearest neighbor: Same output as nearest neighbor 25

26 Weighting Dimensions Apparent clusters at one scaling of X are not so apparent at another scaling

27 Euclidean Distance What people intuitively think of as distance Dimension 2: y d 2 ( A, B) = ( ax bx ) + ( ay by ) 2 Dimension 1: x Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio

28 Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Generalized Euclidean Distance ) i(a,b b b b B a a a A b a B A d i i n n n i i i R = = = = and },...,, { },,...,, { where ), ( / 1 2

29 L p norms L p norms are all special cases of this function: d( x, y) n = i= 1 x i y i p 1/ p p changes the norm x 1 = L 1 norm = Manhattan Distance : p = 1 x 2 = L 2 norm = Euclidean Distance : p = 2 Hamming Distance : p = 1and x i,y i { 0,1} Bryan Pardo, EECS 352 Winter 2010

30 General Learning Task There is a set of possible examples Each example is an n-tuple of attribute values 1 1 There is a target function that maps X onto some finite set Y The DATA is a set of duples <example, target function values> D Find a hypothesis h such that... X = { x,... x } 1 n x =< a,..., ak f : X Y x, f ( x ) >,... < x, f ( x ) > } 1 m = { < 1 m x, h( x) f ( x) >

31 Weighting Dimensions Put point in the cluster with the closest center of gravity Which cluster should the red point go in? How do I measure distance in a way that gives the right answer for both situations? Bryan Pardo, EECS 352 Winter 2010

32 Weighted Norms You can compensate by weighting your dimensions. d n 1/ ( x, y) = w x y p i i i i= 1 p This lets you turn your circle of equal-distance into an elipse with axes parallel to the dimensions of the vectors. Bryan Pardo, EECS 352 Winter 2010