Quiz Section Week 8 May 17, Machine learning and Support Vector Machines

Transcription

1 Quiz Section Week 8 May 17, 2016 Machine learning and Support Vector Machines

2 Another definition of supervised machine learning Given N training examples (objects) {(x 1,y 1 ), (x 2,y 2 ),, (x N,y N )} Features and class labels Machine learning algorithm finds a function g: X Y Decision Tree HMM Parameters of g are trained from an objective function Decision tree: branch purity Maximum likelihood Mapping function g Decision Tree HMM features Binary variables (but could a lot of things) Nucleotides (but could be anything) Parameters θ How/when to branch emission and transition probabilities SVM Numbers hyperplane weights Optimization criterion Branch purity P(x θ) Maximummargin

3 A support vector machine classifier maps real-number features to classes Classify leukemia subtypes based on gene expression measured by microarray Gene expression in patient tumors Acute Myeloid leukemia Acute Lymphoblastic Leukemia

4 A support vector machine classifier maps real-number features to classes Classify leukemia subtypes based on gene expression measured by microarray Gene expression in patient tumors Acute Myeloid leukemia Acute Lymphoblastic Leukemia

5 Defines a line (like linear regression) that separates the two classes Classify leukemia subtypes based on gene expression measured by microarray Gene expression in patient tumors Separating line: y = mx + b Or, in another form ax + by + c = 0 Acute Myeloid leukemia Acute Lymphoblastic Leukemia If ax + by + c > 0, it is AML If ax + by + c < 0, it is ALL

6 Can be extended to higher dimensions by drawing a hyperplane Classify leukemia subtypes based on gene expression measured by microarray Gene expression in patient tumors Acute Myeloid leukemia Acute Lymphoblastic Leukemia Hyperplane is defined by weights w 1, w 2,, w N+1 paramaters for N features: w 1 Feature 1 + w 2 Feature w N Feature N + w N+1 = 0

7 There are many separating lines

8 SVM s, unlike other hyperplane methods, choose the hyperplane to maximize the margin margin Margin is the (perpendicular) distance between the separating line and the closest points Other methods maximize average distances, or some other function of the distances

9 If it isn t linearly separable, can we define more features until it is? One feature doesn t separate the points Add the feature e 2 that separates the points

10 Kernel SVMs draw curved lines by mapping points into more complex spaces or adding more features Possibly reasonable Possibly overfitting

11 SVM can combine variant effect predictions from multiple methods into a single score 1.Get deleteriousness scores from multiple methods 2. Find hyperplane that considers scores from all methods

12 Combination score (c-score) is more accurate than any individual score

13 HMMs can take into account numerical features or any feature that can be assigned a probability distribution So far our HMMs emitted discrete features with discrete probability tables For DNA sequence: A: 10%, T: 10%, G: 40%, C: 40% Chance of rain per week: Rain: 80%, no rain: 20%

14 How do we assign probabilities to real numbers?

15 Segway s hidden states use Gaussian distributions as the emission probabilities for observed read counts in each region Distribution of RNA-seq reads across introns and exons

16 Segway s learned Gaussian emission probabilities for different parts of the genome Learned Gaussian emission probabilities Observed data type Hidden states

17 HMMs can be made to be unsupervised like K-means clustering is K-means clustering algorithm 1. Randomly assign k centers 2. Assign points to closest center 3. Re-compute location of the k- centers based on assignment 4. Repeat 2 and 3 until convergence HMM learning algorithm 1. Randomly generate emission probabilities for k hidden states 2. Use Viterbi or F-B to assign sites to hidden states 3. Re-compute emission probabilities based on new assignments 4. Repeat 2 and 3 until convergence

18 Segway s learned Gaussian emission probabilities for different parts of the genome Learned Gaussian emission probabilities Observed data type Hidden states named by human scientist

19 random() returns a uniformly distributed random value from [0,1) Heads 0.5 Tails How can you convert this into a random coin flip with heads or tails? How can you convert this to a sample from a Gaussian distribution?

20

21 Rejection Sampling

22 Area under a curve sound familiar?