Decision Trees: Representa:on
|
|
|
- Charla Lewis
- 5 years ago
- Views:
Transcription
1 Decision Trees: Representa:on Machine Learning Fall 2017 Some slides from Tom Mitchell, Dan Roth and others 1
2 Key issues in machine learning Modeling How to formulate your problem as a machine learning problem? How to represent data? Which algorithms to use? What learning protocols? Representa:on Good hypothesis spaces and good features Algorithms What is a good learning algorithm? What is success? Generaliza:on vs overfimng The computa:onal ques:on: How long will learning take? 2
3 Coming up (the rest of the semester) Different hypothesis spaces and learning algorithms Decision trees and the ID3 algorithm Linear classifiers Perceptron Winnow SVM Logis:c regression Combining mul:ple classifiers Boos:ng, bagging Non- linear classifiers Nearest neighbors 3
4 Coming up (the rest of the semester) Different hypothesis spaces and learning algorithms Decision trees and the ID3 algorithm Linear classifiers Important issues to consider Perceptron Winnow SVM Logis:c regression 1. What do these hypotheses represent? 2. Implicit assump:ons and tradeoffs Combining mul:ple classifiers Boos:ng, 3. Generaliza:on? bagging Non- linear classifiers Nearest 4. neighbors How do we learn? 4
5 This lecture: Learning Decision Trees 1. Representa+on: What are decision trees? 2. Algorithm: Learning decision trees The ID3 algorithm: A greedy heuris:c 3. Some extensions 5
6 This lecture: Learning Decision Trees 1. Representa+on: What are decision trees? 2. Algorithm: Learning decision trees The ID3 algorithm: A greedy heuris:c 3. Some extensions 6
7 Represen:ng data Data can be represented as a big table, with columns deno:ng different a_ributes Name Label Claire Cardie - Peter Bartle_ + Eric Baum + Haym Hirsh + Shai Ben- David + Michael I. Jordan - 7
8 Represen:ng data Data can be represented as a big table, with columns deno:ng different a_ributes Name Special character in name? Second character of first name Length of first name>5? Gender Claire Cardie No l Yes Female - Peter Bartle_ No e No Male + Eric Baum No r No Male + Haym Hirsh No a No Male + Label Shai Ben- David Michael I. Jordan Yes h No Male + Yes i Yes Male - 8
9 Represen:ng data Data can be represented as a big table, with columns deno:ng different a_ributes Name Special character in name? Second character of first name Length of first name>5? Gender Claire With Cardie these four No a_ributes, how l many unique rows Yes are possible? Female = 2704 Peter Bartle_ No e No Male + Eric Baum If there are 100 No a_ributes, all binary, r how many No unique rows Male are possible? Haym Hirsh No a No Male + Label Shai Ben- David Michael I. Jordan Yes h No Male - Yes i Yes Male + 9
10 Represen:ng data Data can be represented as a big table, with columns deno:ng different a_ributes Name Special character in name? Second character of first name Length of first name>5? Gender Claire With Cardie these four No a_ributes, how l many unique rows Yes are possible? Female = 208 Peter Bartle_ No e No Male + Eric Baum If there are 100 No a_ributes, all binary, r how many No unique rows Male are possible? Haym Hirsh No a No Male + Label Shai Ben- David Michael I. Jordan Yes h No Male - Yes i Yes Male + 10
11 Represen:ng data Data can be represented as a big table, with columns deno:ng different a_ributes Name Special character in name? Second character of first name Length of first name>5? Gender Claire With Cardie these four No a_ributes, how l many unique rows Yes are possible? Female = 208 Peter Bartle_ No e No Male + Eric Baum If there are 100 No a_ributes, all binary, r how many No unique rows Male are possible? Haym Hirsh No a No Male + Label Shai Ben- David Michael I. Jordan Yes h No Male - Yes i Yes Male + 11
12 Represen:ng data Data can be represented as a big table, with columns deno:ng different a_ributes Name Special character in name? Second character of first name Length of first name>5? Gender Claire With Cardie these four No a_ributes, how l many unique rows Yes are possible? Female = 208 Peter Bartle_ No e No Male + Eric Baum If there are 100 No a_ributes, all binary, r how many No unique rows Male are possible? Haym Hirsh No a No Male + Label Shai Ben- David Michael I. Jordan Yes h No Male - Yes i Yes Male + 12
13 Represen:ng data Data can be represented as a big table, with columns deno:ng different a_ributes Name Special character in name? Second character of first name Length of first name>5? Gender Claire With Cardie these four No a_ributes, how l many unique rows Yes are possible? Female = 208 Peter Bartle_ No e No Male + Eric Baum If there are 100 No a_ributes, all binary, r how many No unique rows Male are possible? - Haym Hirsh No a No Male + We need to figure out how to represent in a be_er, more efficient way Shai Ben- Yes h No Male - David Michael I. Jordan Label Yes i Yes Male + 13
14 What are decision trees? A hierarchical data structure that represents data using a divide- and- conquer strategy Can be used as hypothesis class for non- parametric classifica:on or regression General idea: Given a collec:on of examples, learn a decision tree that represents it 14
15 What are decision trees? Decision trees are a family of classifiers for instances that are represented by collec:ons of a_ributes (i.e. features) Nodes are tests for feature values There is one branch for every value that the feature can take Leaves of the tree specify the class labels 15
16 Let s build a decision tree for classifying shapes Label=C Label=B Label=A 16
17 Let s build a decision tree for classifying shapes Label=C Label=B Label=A Before building a decision tree: What is the label for a red triangle? And why? 17
18 Let s build a decision tree for classifying shapes Label=C Label=B Label=A What are some a_ributes of the examples? 18
19 Let s build a decision tree for classifying shapes Label=C Label=B Label=A What are some a_ributes of the examples? Color, Shape 19
20 Let s build a decision tree for classifying shapes Label=C Label=B Label=A What are some a_ributes of the examples? Color, Shape Color? 20
21 Let s build a decision tree for classifying shapes Label=C Label=B What are some a_ributes of the examples? Color, Shape Color? Blue Red Label=A Green 21
22 Let s build a decision tree for classifying shapes Label=C Label=B What are some a_ributes of the examples? Color, Shape Color? Blue Red B Label=A Green 22
23 Let s build a decision tree for classifying shapes Label=C What are some a_ributes of the examples? Color, Shape Color? B triangle square Blue Shape? A Label=B circle C Red B Label=A Green 23
24 Let s build a decision tree for classifying shapes Label=C What are some a_ributes of the examples? Color, Shape Color? B triangle Blue Shape? square A Label=B circle C Red B square B Label=A Green Shape? circle A 24
25 Let s build a decision tree for classifying shapes 1. How to use a decision tree for predic7on? What is the label for a red triangle? Just follow a path from the root to a leaf What about a green triangle? What are some a_ributes of the examples? Color, Shape Label=C 2. How do we learn Label=B a decision tree? Coming up soon triangle B Label=A Color? Blue Red Green Shape? B Shape? circle square square A C B circle A 25
26 Expressivity of Decision trees What Boolean func:ons can decision trees represent? Any Boolean func:on (Color=blue AND Shape=triangle ) Label=B) AND (Color=blue AND Shape=square ) Label=A) AND (Color=blue AND Shape=circle ) Label=C) AND. Every path from the tree to a root is a rule The full tree is equivalent to the conjunc:on of all the rules Any Boolean func7on can be represented as a decision tree. 26
27 Expressivity of Decision trees What Boolean func:ons can decision trees represent? Any Boolean func:on (Color=blue AND Shape=triangle ) Label=B) AND (Color=blue AND Shape=square ) Label=A) AND (Color=blue AND Shape=circle ) Label=C) AND. Every path from the tree to a root is a rule The full tree is equivalent to the conjunc:on of all the rules Any Boolean func7on can be represented as a decision tree. 27
28 Expressivity of Decision trees What Boolean func:ons can decision trees represent? Any Boolean func:on (Color=blue AND Shape=triangle ) Label=B) AND (Color=blue AND Shape=square ) Label=A) AND (Color=blue AND Shape=circle ) Label=C) AND. Every path from the tree to a root is a rule The full tree is equivalent to the conjunc:on of all the rules Any Boolean func7on can be represented as a decision tree. 28
29 Decision Trees Outputs are discrete categories But real valued outputs are also possible (regression trees) Methods for handling noisy data (noise in the label or in the features) and for handling missing a_ributes Pruning trees helps with noise More on this later 29
30 Numeric a_ributes and decision boundaries We have seen instances represented as a_ribute- value pairs (color=blue, second le_er=e, etc.) Values have been categorical How do we deal with numeric feature values? (eg length =?) Discre:ze them or use thresholds on the numeric values This example divides the feature space into axis parallel rectangles 30
31 Numeric a_ributes and decision boundaries We have seen instances represented as a_ribute- value pairs (color=blue, second le_er=e, etc.) Values have been categorical How do we deal with numeric feature values? (eg length =?) Discre:ze them or use thresholds on the numeric values This example divides the feature space into axis parallel rectangles 31
32 Numeric a_ributes and decision boundaries We have seen instances represented as a_ribute- value pairs (color=blue, second le_er=e, etc.) Values have been categorical How do we deal with numeric feature values? (eg length =?) Discre:ze them or use thresholds on the numeric values This example divides the feature space into axis parallel rectangles 32
33 Numeric a_ributes and decision boundaries We have seen instances represented as a_ribute- value pairs (color=blue, second le_er=e, etc.) Values have been categorical How do we deal with numeric feature values? (eg length =?) Discre:ze them or use thresholds on the numeric values This example divides the feature space into axis parallel rectangles Y X 33
34 Numeric a_ributes and decision boundaries We have seen instances represented as a_ribute- value pairs (color=blue, second le_er=e, etc.) Values have been categorical How do we deal with numeric feature values? (eg length =?) Discre:ze them or use thresholds on the numeric values This example divides the feature space into axis parallel rectangles Y X no Y>7 no yes X<3 no yes Y<5 no yes X < 1 yes
35 Numeric a_ributes and decision boundaries We have seen instances represented as a_ribute- value pairs (color=blue, second le_er=e, etc.) Values have been categorical How do we deal with numeric feature values? (eg length =?) Discre:ze them or use thresholds on the numeric values This example divides the feature space into axis parallel rectangles Decision Y boundaries can be non- linear X no Y>7 no yes X<3 no yes Y<5 no yes X < 1 yes
36 Summary: Decision trees Decision trees can represent any Boolean func:on A way to represent lot of data A natural representa:on (think 20 ques:ons) Predic:ng with a decision tree is easy Clearly, given a dataset, there are many decision trees that can represent it. Why? Learning a good representa:on from data is the next ques:on 36
37 Summary: Decision trees Decision trees can represent any Boolean func:on A way to represent lot of data A natural representa:on (think 20 ques:ons) Predic:ng with a decision tree is easy Clearly, given a dataset, there are many decision trees that can represent it. Why? Learning a good representa:on from data is the next ques:on 37
38 Exercise Write down the decision tree for the shapes data if the root node was Shape instead of Color Will the two trees make the same predic:ons for unseen shapes/color combina:ons? Label=C Label=B Label=A 38
Decision Trees: Representation
Decision Trees: Representation Machine Learning Fall 2018 Some slides from Tom Mitchell, Dan Roth and others 1 Key issues in machine learning Modeling How to formulate your problem as a machine learning
Supervised Learning: The Setup. Spring 2018
Supervised Learning: The Setup Spring 2018 1 Homework 0 will be released today through Canvas Due: Jan. 19 (next Friday) midnight 2 Last lecture We saw What is learning? Learning as generalization The
Decision Trees: Discussion
Decision Trees: Discussion Machine Learning Spring 2018 The slides are mainly from Vivek Srikumar 1 This lecture: Learning Decision Trees 1. Representation: What are decision trees? 2. Algorithm: Learning
Machine Learning. Decision Trees. Manfred Huber
Machine Learning Decision Trees Manfred Huber 2015 1 Decision Trees Classifiers covered so far have been Non-parametric (KNN) Probabilistic with independence (Naïve Bayes) Linear in features (Logistic
Decision Trees Oct
Decision Trees Oct - 7-2009 Previously We learned two different classifiers Perceptron: LTU KNN: complex decision boundary If you are a novice in this field, given a classification application, are these
About the Course. Reading List. Assignments and Examina5on
Uppsala University Department of Linguis5cs and Philology About the Course Introduc5on to machine learning Focus on methods used in NLP Decision trees and nearest neighbor methods Linear models for classifica5on
Ensemble Methods, Decision Trees
CS 1675: Intro to Machine Learning Ensemble Methods, Decision Trees Prof. Adriana Kovashka University of Pittsburgh November 13, 2018 Plan for This Lecture Ensemble methods: introduction Boosting Algorithm
Classification Algorithms in Data Mining
August 9th, 2016 Suhas Mallesh Yash Thakkar Ashok Choudhary CIS660 Data Mining and Big Data Processing -Dr. Sunnie S. Chung Classification Algorithms in Data Mining Deciding on the classification algorithms
STA 4273H: Sta-s-cal Machine Learning
STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! h0p://www.cs.toronto.edu/~rsalakhu/ Lecture 3 Parametric Distribu>ons We want model the probability
Decision Trees, Random Forests and Random Ferns. Peter Kovesi
Decision Trees, Random Forests and Random Ferns Peter Kovesi What do I want to do? Take an image. Iden9fy the dis9nct regions of stuff in the image. Mark the boundaries of these regions. Recognize and
Nearest Neighbor Classification. Machine Learning Fall 2017
Nearest Neighbor Classification Machine Learning Fall 2017 1 This lecture K-nearest neighbor classification The basic algorithm Different distance measures Some practical aspects Voronoi Diagrams and Decision
All lecture slides will be available at CSC2515_Winter15.html
CSC2515 Fall 2015 Introduc3on to Machine Learning Lecture 9: Support Vector Machines All lecture slides will be available at http://www.cs.toronto.edu/~urtasun/courses/csc2515/ CSC2515_Winter15.html Many
Decision Tree CE-717 : Machine Learning Sharif University of Technology
Decision Tree CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Some slides have been adapted from: Prof. Tom Mitchell Decision tree Approximating functions of usually discrete
Logis&c Regression. Aar$ Singh & Barnabas Poczos. Machine Learning / Jan 28, 2014
Logis&c Regression Aar$ Singh & Barnabas Poczos Machine Learning 10-701/15-781 Jan 28, 2014 Linear Regression & Linear Classifica&on Weight Height Linear fit Linear decision boundary 2 Naïve Bayes Recap
PROBLEM 4
PROBLEM 2 PROBLEM 4 PROBLEM 5 PROBLEM 6 PROBLEM 7 PROBLEM 8 PROBLEM 9 PROBLEM 10 PROBLEM 11 PROBLEM 12 PROBLEM 13 PROBLEM 14 PROBLEM 16 PROBLEM 17 PROBLEM 22 PROBLEM 23 PROBLEM 24 PROBLEM 25
Symbolic AI. Andre Freitas. Photo by Vasilyev Alexandr
Symbolic AI Andre Freitas Photo by Vasilyev Alexandr Acknowledgements These slides were based on the slides of: Peter A. Flach, Rule induction tutorial, IDA Spring School 2001. Anoop & Hector, Inductive
What is Learning? CS 343: Artificial Intelligence Machine Learning. Raymond J. Mooney. Problem Solving / Planning / Control.
What is Learning? CS 343: Artificial Intelligence Machine Learning Herbert Simon: Learning is any process by which a system improves performance from experience. What is the task? Classification Problem
Text Categorization. Foundations of Statistic Natural Language Processing The MIT Press1999
Text Categorization Foundations of Statistic Natural Language Processing The MIT Press1999 Outline Introduction Decision Trees Maximum Entropy Modeling (optional) Perceptrons K Nearest Neighbor Classification
Learning Rules. CS 391L: Machine Learning: Rule Learning. Raymond J. Mooney. Rule Learning Approaches. Decision-Trees to Rules. Sequential Covering
Learning Rules CS 9L: Machine Learning: Rule Learning Raymond J. Mooney University of Texas at Austin If-then rules in logic are a standard representation of knowledge that have proven useful in expert-systems
Lecture 7: Decision Trees
Lecture 7: Decision Trees Instructor: Outline 1 Geometric Perspective of Classification 2 Decision Trees Geometric Perspective of Classification Perspective of Classification Algorithmic Geometric Probabilistic...
Classification: Decision Trees
Classification: Decision Trees IST557 Data Mining: Techniques and Applications Jessie Li, Penn State University 1 Decision Tree Example Will a pa)ent have high-risk based on the ini)al 24-hour observa)on?
Decision Tree Learning
Decision Tree Learning 1 Simple example of object classification Instances Size Color Shape C(x) x1 small red circle positive x2 large red circle positive x3 small red triangle negative x4 large blue circle
A Systematic Overview of Data Mining Algorithms
A Systematic Overview of Data Mining Algorithms 1 Data Mining Algorithm A well-defined procedure that takes data as input and produces output as models or patterns well-defined: precisely encoded as a
Machine Learning. Decision Trees. Le Song /15-781, Spring Lecture 6, September 6, 2012 Based on slides from Eric Xing, CMU
Machine Learning 10-701/15-781, Spring 2008 Decision Trees Le Song Lecture 6, September 6, 2012 Based on slides from Eric Xing, CMU Reading: Chap. 1.6, CB & Chap 3, TM Learning non-linear functions f:
Decision Trees. Petr Pošík. Czech Technical University in Prague Faculty of Electrical Engineering Dept. of Cybernetics
Decision Trees Petr Pošík Czech Technical University in Prague Faculty of Electrical Engineering Dept. of Cybernetics This lecture is largely based on the book Artificial Intelligence: A Modern Approach,
CS 229 Midterm Review
CS 229 Midterm Review Course Staff Fall 2018 11/2/2018 Outline Today: SVMs Kernels Tree Ensembles EM Algorithm / Mixture Models [ Focus on building intuition, less so on solving specific problems. Ask
Advanced branch predic.on algorithms. Ryan Gabrys Ilya Kolykhmatov
Advanced branch predic.on algorithms Ryan Gabrys Ilya Kolykhmatov Context Branches are frequent: 15-25 % A branch predictor allows the processor to specula.vely fetch and execute instruc.ons down the predicted
Data Preprocessing. Supervised Learning
Supervised Learning Regression Given the value of an input X, the output Y belongs to the set of real values R. The goal is to predict output accurately for a new input. The predictions or outputs y are
Machine Learning: Algorithms and Applications Mockup Examination
Machine Learning: Algorithms and Applications Mockup Examination 14 May 2012 FIRST NAME STUDENT NUMBER LAST NAME SIGNATURE Instructions for students Write First Name, Last Name, Student Number and Signature
Predic'ng ALS Progression with Bayesian Addi've Regression Trees
Predic'ng ALS Progression with Bayesian Addi've Regression Trees Lilly Fang and Lester Mackey November 13, 2012 RECOMB Conference on Regulatory and Systems Genomics The ALS Predic'on Prize Challenge: Predict
ML4Bio Lecture #1: Introduc3on. February 24 th, 2016 Quaid Morris
ML4Bio Lecture #1: Introduc3on February 24 th, 216 Quaid Morris Course goals Prac3cal introduc3on to ML Having a basic grounding in the terminology and important concepts in ML; to permit self- study,
Supervised Learning. Decision trees Artificial neural nets K-nearest neighbor Support vectors Linear regression Logistic regression...
Supervised Learning Decision trees Artificial neural nets K-nearest neighbor Support vectors Linear regression Logistic regression... Supervised Learning y=f(x): true function (usually not known) D: training
Fall 09, Homework 5
5-38 Fall 09, Homework 5 Due: Wednesday, November 8th, beginning of the class You can work in a group of up to two people. This group does not need to be the same group as for the other homeworks. You
UNIT II A. ENTITY RELATIONSHIP MODEL
UNIT II A. ENTITY RELATIONSHIP MODEL Agenda En0ty & En0ty Sets A6ributes Rela0onship & Rela0onship Sets Constraints Mapping Cardinali0es, Par0cipa0on Constraints, Keys E-R Diagrams & Design of Database
Nearest neighbor classification DSE 220
Nearest neighbor classification DSE 220 Decision Trees Target variable Label Dependent variable Output space Person ID Age Gender Income Balance Mortgag e payment 123213 32 F 25000 32000 Y 17824 49 M 12000-3000
Stages of (Batch) Machine Learning
Evalua&on Stages of (Batch) Machine Learning Given: labeled training data X, Y = {hx i,y i i} n i=1 Assumes each x i D(X ) with y i = f target (x i ) Train the model: model ß classifier.train(x, Y ) x
Supervised Learning (contd) Linear Separation. Mausam (based on slides by UW-AI faculty)
Supervised Learning (contd) Linear Separation Mausam (based on slides by UW-AI faculty) Images as Vectors Binary handwritten characters Treat an image as a highdimensional vector (e.g., by reading pixel
Task Description: Finding Similar Documents. Document Retrieval. Case Study 2: Document Retrieval
Case Study 2: Document Retrieval Task Description: Finding Similar Documents Machine Learning for Big Data CSE547/STAT548, University of Washington Sham Kakade April 11, 2017 Sham Kakade 2017 1 Document
Volume Visualiza0on. Today s Class. Grades & Homework feedback on Homework Submission Server
11/3/14 Volume Visualiza0on h3p://imgur.com/trjonqk h3p://i.imgur.com/zcjc9kp.jpg Today s Class Grades & Homework feedback on Homework Submission Server Everything except HW4 (didn t get to that yet) &
6.034 Quiz 2, Spring 2005
6.034 Quiz 2, Spring 2005 Open Book, Open Notes Name: Problem 1 (13 pts) 2 (8 pts) 3 (7 pts) 4 (9 pts) 5 (8 pts) 6 (16 pts) 7 (15 pts) 8 (12 pts) 9 (12 pts) Total (100 pts) Score 1 1 Decision Trees (13
Large Scale Data Analysis Using Deep Learning
Large Scale Data Analysis Using Deep Learning Machine Learning Basics - 1 U Kang Seoul National University U Kang 1 In This Lecture Overview of Machine Learning Capacity, overfitting, and underfitting
CS Machine Learning
CS 60050 Machine Learning Decision Tree Classifier Slides taken from course materials of Tan, Steinbach, Kumar 10 10 Illustrating Classification Task Tid Attrib1 Attrib2 Attrib3 Class 1 Yes Large 125K
Model Selection Introduction to Machine Learning. Matt Gormley Lecture 4 January 29, 2018
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Model Selection Matt Gormley Lecture 4 January 29, 2018 1 Q&A Q: How do we deal
Supervised vs unsupervised clustering
Classification Supervised vs unsupervised clustering Cluster analysis: Classes are not known a- priori. Classification: Classes are defined a-priori Sometimes called supervised clustering Extract useful
Tree-based methods for classification and regression
Tree-based methods for classification and regression Ryan Tibshirani Data Mining: 36-462/36-662 April 11 2013 Optional reading: ISL 8.1, ESL 9.2 1 Tree-based methods Tree-based based methods for predicting
Data Mining. Lecture 03: Nearest Neighbor Learning
Data Mining Lecture 03: Nearest Neighbor Learning Theses slides are based on the slides by Tan, Steinbach and Kumar (textbook authors) Prof. R. Mooney (UT Austin) Prof E. Keogh (UCR), Prof. F. Provost
Minimum Redundancy and Maximum Relevance Feature Selec4on. Hang Xiao
Minimum Redundancy and Maximum Relevance Feature Selec4on Hang Xiao Background Feature a feature is an individual measurable heuris4c property of a phenomenon being observed In character recogni4on: horizontal
CS6375: Machine Learning Gautam Kunapuli. Mid-Term Review
Gautam Kunapuli Machine Learning Data is identically and independently distributed Goal is to learn a function that maps to Data is generated using an unknown function Learn a hypothesis that minimizes
Bias-Variance Analysis of Ensemble Learning
Bias-Variance Analysis of Ensemble Learning Thomas G. Dietterich Department of Computer Science Oregon State University Corvallis, Oregon 97331 http://www.cs.orst.edu/~tgd Outline Bias-Variance Decomposition
Machine Learning - Lecture 2: Nearest-neighbour methods
Machine Learning - Lecture 2: Nearest-neighbour methods Chris Thornton January 8, 22 Brighton pier Switchback Data Data Vertical axis is age; horizontal axis is alchohol consumption per week. Data Vertical
Topics in Machine Learning
Topics in Machine Learning Gilad Lerman School of Mathematics University of Minnesota Text/slides stolen from G. James, D. Witten, T. Hastie, R. Tibshirani and A. Ng Machine Learning - Motivation Arthur
Random Forest A. Fornaser
Random Forest A. Fornaser alberto.fornaser@unitn.it Sources Lecture 15: decision trees, information theory and random forests, Dr. Richard E. Turner Trees and Random Forests, Adele Cutler, Utah State University
CSC411/2515 Tutorial: K-NN and Decision Tree
CSC411/2515 Tutorial: K-NN and Decision Tree Mengye Ren csc{411,2515}ta@cs.toronto.edu September 25, 2016 Cross-validation K-nearest-neighbours Decision Trees Review: Motivation for Validation Framework:
Mul$variate classifica$on. Astro 585 Spring 2013
Mul$variate classifica$on Astro 585 Spring 2013 Concepts of classifica$on Here we consider situa9ons where the mul9variate dataset under study represents a new test set that is a mixture of classes that
10/5/2017 MIST.6060 Business Intelligence and Data Mining 1. Nearest Neighbors. In a p-dimensional space, the Euclidean distance between two records,
10/5/2017 MIST.6060 Business Intelligence and Data Mining 1 Distance Measures Nearest Neighbors In a p-dimensional space, the Euclidean distance between two records, a = a, a,..., a ) and b = b, b,...,
Lecture 25: Review I
Lecture 25: Review I Reading: Up to chapter 5 in ISLR. STATS 202: Data mining and analysis Jonathan Taylor 1 / 18 Unsupervised learning In unsupervised learning, all the variables are on equal standing,
A Systematic Overview of Data Mining Algorithms. Sargur Srihari University at Buffalo The State University of New York
A Systematic Overview of Data Mining Algorithms Sargur Srihari University at Buffalo The State University of New York 1 Topics Data Mining Algorithm Definition Example of CART Classification Iris, Wine
CSE4334/5334 DATA MINING
CSE4334/5334 DATA MINING Lecture 4: Classification (1) CSE4334/5334 Data Mining, Fall 2014 Department of Computer Science and Engineering, University of Texas at Arlington Chengkai Li (Slides courtesy
9/6/14. Our first learning algorithm. Comp 135 Introduction to Machine Learning and Data Mining. knn Algorithm. knn Algorithm (simple form)
Comp 135 Introduction to Machine Learning and Data Mining Our first learning algorithm How would you classify the next example? Fall 2014 Professor: Roni Khardon Computer Science Tufts University o o o
Business Club. Decision Trees
Business Club Decision Trees Business Club Analytics Team December 2017 Index 1. Motivation- A Case Study 2. The Trees a. What is a decision tree b. Representation 3. Regression v/s Classification 4. Building
k-nn classification with R QMMA
k-nn classification with R QMMA Emanuele Taufer file:///c:/users/emanuele.taufer/google%20drive/2%20corsi/5%20qmma%20-%20mim/0%20labs/l1-knn-eng.html#(1) 1/16 HW (Height and weight) of adults Statistics
What is Search For? CS 188: Ar)ficial Intelligence. Constraint Sa)sfac)on Problems Sep 14, 2015
CS 188: Ar)ficial Intelligence Constraint Sa)sfac)on Problems Sep 14, 2015 What is Search For? Assump)ons about the world: a single agent, determinis)c ac)ons, fully observed state, discrete state space
Evaluating Classifiers
Evaluating Classifiers Reading for this topic: T. Fawcett, An introduction to ROC analysis, Sections 1-4, 7 (linked from class website) Evaluating Classifiers What we want: Classifier that best predicts
K Nearest Neighbor Wrap Up K- Means Clustering. Slides adapted from Prof. Carpuat
K Nearest Neighbor Wrap Up K- Means Clustering Slides adapted from Prof. Carpuat K Nearest Neighbor classification Classification is based on Test instance with Training Data K: number of neighbors that
7. Nearest neighbors. Learning objectives. Foundations of Machine Learning École Centrale Paris Fall 2015
Foundations of Machine Learning École Centrale Paris Fall 2015 7. Nearest neighbors Chloé-Agathe Azencott Centre for Computational Biology, Mines ParisTech chloe agathe.azencott@mines paristech.fr Learning
The Curse of Dimensionality
The Curse of Dimensionality ACAS 2002 p1/66 Curse of Dimensionality The basic idea of the curse of dimensionality is that high dimensional data is difficult to work with for several reasons: Adding more
Lecture 5: Decision Trees (Part II)
Lecture 5: Decision Trees (Part II) Dealing with noise in the data Overfitting Pruning Dealing with missing attribute values Dealing with attributes with multiple values Integrating costs into node choice
Classification: Feature Vectors
Classification: Feature Vectors Hello, Do you want free printr cartriges? Why pay more when you can get them ABSOLUTELY FREE! Just # free YOUR_NAME MISSPELLED FROM_FRIEND... : : : : 2 0 2 0 PIXEL 7,12
Geometric data structures:
Geometric data structures: Machine Learning for Big Data CSE547/STAT548, University of Washington Sham Kakade Sham Kakade 2017 1 Announcements: HW3 posted Today: Review: LSH for Euclidean distance Other
Machine Learning Techniques
Machine Learning Techniques ( 機器學習技法 ) Lecture 9: Decision Tree Hsuan-Tien Lin ( 林軒田 ) htlin@csie.ntu.edu.tw Department of Computer Science & Information Engineering National Taiwan University ( 國立台灣大學資訊工程系
CS 5614: (Big) Data Management Systems. B. Aditya Prakash Lecture #19: Machine Learning 1
CS 5614: (Big) Data Management Systems B. Aditya Prakash Lecture #19: Machine Learning 1 Supervised Learning Would like to do predicbon: esbmate a func3on f(x) so that y = f(x) Where y can be: Real number:
Homework 4: Clustering, Recommenders, Dim. Reduction, ML and Graph Mining (due November 19 th, 2014, 2:30pm, in class hard-copy please)
Virginia Tech. Computer Science CS 5614 (Big) Data Management Systems Fall 2014, Prakash Homework 4: Clustering, Recommenders, Dim. Reduction, ML and Graph Mining (due November 19 th, 2014, 2:30pm, in
SOCIAL MEDIA MINING. Data Mining Essentials
SOCIAL MEDIA MINING Data Mining Essentials Dear instructors/users of these slides: Please feel free to include these slides in your own material, or modify them as you see fit. If you decide to incorporate
Brief Guide on Using SPSS 10.0
Brief Guide on Using SPSS 10.0 (Use student data, 22 cases, studentp.dat in Dr. Chang s Data Directory Page) (Page address: http://www.cis.ysu.edu/~chang/stat/) I. Processing File and Data To open a new
Text classification II CE-324: Modern Information Retrieval Sharif University of Technology
Text classification II CE-324: Modern Information Retrieval Sharif University of Technology M. Soleymani Fall 2015 Some slides have been adapted from: Profs. Manning, Nayak & Raghavan (CS-276, Stanford)
CS273 Midterm Exam Introduction to Machine Learning: Winter 2015 Tuesday February 10th, 2014
CS273 Midterm Eam Introduction to Machine Learning: Winter 2015 Tuesday February 10th, 2014 Your name: Your UCINetID (e.g., myname@uci.edu): Your seat (row and number): Total time is 80 minutes. READ THE
k-nearest Neighbors + Model Selection
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University k-nearest Neighbors + Model Selection Matt Gormley Lecture 5 Jan. 30, 2019 1 Reminders
Nearest Neighbor Predictors
Nearest Neighbor Predictors September 2, 2018 Perhaps the simplest machine learning prediction method, from a conceptual point of view, and perhaps also the most unusual, is the nearest-neighbor method,
Announcements. CS 188: Artificial Intelligence Spring Classification: Feature Vectors. Classification: Weights. Learning: Binary Perceptron
CS 188: Artificial Intelligence Spring 2010 Lecture 24: Perceptrons and More! 4/20/2010 Announcements W7 due Thursday [that s your last written for the semester!] Project 5 out Thursday Contest running
Fall 2018 CSE 482 Big Data Analysis: Exam 1 Total: 36 (+3 bonus points)
Fall 2018 CSE 482 Big Data Analysis: Exam 1 Total: 36 (+3 bonus points) Name: This exam is open book and notes. You can use a calculator but no laptops, cell phones, nor other electronic devices are allowed.
CSE 473: Ar+ficial Intelligence
CSE 473: Ar+ficial Intelligence Search Instructor: Luke Ze=lemoyer University of Washington [These slides were adapted from Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials
Machine Learning. Topic 5: Linear Discriminants. Bryan Pardo, EECS 349 Machine Learning, 2013
Machine Learning Topic 5: Linear Discriminants Bryan Pardo, EECS 349 Machine Learning, 2013 Thanks to Mark Cartwright for his extensive contributions to these slides Thanks to Alpaydin, Bishop, and Duda/Hart/Stork
Network Traffic Measurements and Analysis
DEIB - Politecnico di Milano Fall, 2017 Sources Hastie, Tibshirani, Friedman: The Elements of Statistical Learning James, Witten, Hastie, Tibshirani: An Introduction to Statistical Learning Andrew Ng:
Practice EXAM: SPRING 2012 CS 6375 INSTRUCTOR: VIBHAV GOGATE
Practice EXAM: SPRING 0 CS 6375 INSTRUCTOR: VIBHAV GOGATE The exam is closed book. You are allowed four pages of double sided cheat sheets. Answer the questions in the spaces provided on the question sheets.
DS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University January 24 2019 Logistics HW 1 is due on Friday 01/25 Project proposal: due Feb 21 1 page description
COSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor
COSC160: Detection and Classification Jeremy Bolton, PhD Assistant Teaching Professor Outline I. Problem I. Strategies II. Features for training III. Using spatial information? IV. Reducing dimensionality
Decision trees. Decision trees are useful to a large degree because of their simplicity and interpretability
Decision trees A decision tree is a method for classification/regression that aims to ask a few relatively simple questions about an input and then predicts the associated output Decision trees are useful
Feature Extractors. CS 188: Artificial Intelligence Fall Nearest-Neighbor Classification. The Perceptron Update Rule.
CS 188: Artificial Intelligence Fall 2007 Lecture 26: Kernels 11/29/2007 Dan Klein UC Berkeley Feature Extractors A feature extractor maps inputs to feature vectors Dear Sir. First, I must solicit your
Uninformed Search Methods. Informed Search Methods. Midterm Exam 3/13/18. Thursday, March 15, 7:30 9:30 p.m. room 125 Ag Hall
Midterm Exam Thursday, March 15, 7:30 9:30 p.m. room 125 Ag Hall Covers topics through Decision Trees and Random Forests (does not include constraint satisfaction) Closed book 8.5 x 11 sheet with notes
Classification and Regression Trees
Classification and Regression Trees David S. Rosenberg New York University April 3, 2018 David S. Rosenberg (New York University) DS-GA 1003 / CSCI-GA 2567 April 3, 2018 1 / 51 Contents 1 Trees 2 Regression
Decision Trees Dr. G. Bharadwaja Kumar VIT Chennai
Decision Trees Decision Tree Decision Trees (DTs) are a nonparametric supervised learning method used for classification and regression. The goal is to create a model that predicts the value of a target
Machine Learning Techniques for Data Mining
Machine Learning Techniques for Data Mining Eibe Frank University of Waikato New Zealand 10/25/2000 1 PART VII Moving on: Engineering the input and output 10/25/2000 2 Applying a learner is not all Already
Feature Extractors. CS 188: Artificial Intelligence Fall Some (Vague) Biology. The Binary Perceptron. Binary Decision Rule.
CS 188: Artificial Intelligence Fall 2008 Lecture 24: Perceptrons II 11/24/2008 Dan Klein UC Berkeley Feature Extractors A feature extractor maps inputs to feature vectors Dear Sir. First, I must solicit
Part-based and local feature models for generic object recognition
Part-based and local feature models for generic object recognition May 28 th, 2015 Yong Jae Lee UC Davis Announcements PS2 grades up on SmartSite PS2 stats: Mean: 80.15 Standard Dev: 22.77 Vote on piazza
Multi-view Stereo. Ivo Boyadzhiev CS7670: September 13, 2011
Multi-view Stereo Ivo Boyadzhiev CS7670: September 13, 2011 What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape
Decision Tree (Continued) and K-Nearest Neighbour. Dr. Xiaowei Huang
Decision Tree (Continued) and K-Nearest Neighbour Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ Up to now, Recap basic knowledge Decision tree learning How to split Identify the best feature to
Chapter 4: Algorithms CS 795
Chapter 4: Algorithms CS 795 Inferring Rudimentary Rules 1R Single rule one level decision tree Pick each attribute and form a single level tree without overfitting and with minimal branches Pick that
Slides for Data Mining by I. H. Witten and E. Frank
Slides for Data Mining by I. H. Witten and E. Frank 7 Engineering the input and output Attribute selection Scheme-independent, scheme-specific Attribute discretization Unsupervised, supervised, error-
7. Nearest neighbors. Learning objectives. Centre for Computational Biology, Mines ParisTech
Foundations of Machine Learning CentraleSupélec Paris Fall 2016 7. Nearest neighbors Chloé-Agathe Azencot Centre for Computational Biology, Mines ParisTech chloe-agathe.azencott@mines-paristech.fr Learning
15-780: Graduate Artificial Intelligence. Decision trees
15-780: Graduate Artificial Intelligence Decision trees Graphical models So far we discussed models that capture joint probability distributions These have many uses, and can also be used to determine