Instance-Based Representations. k-nearest Neighbor. k-nearest Neighbor. k-nearest Neighbor. exemplars + distance measure. Challenges.
|
|
|
- Augustine Byrd
- 5 years ago
- Views:
Transcription
1 Instance-Based Representations exemplars + distance measure Challenges. algorithm: IB1 classify based on majority class of k nearest neighbors learned structure is not explicitly represented choosing k too low means that the result can be sensitive to noise too high means that the neighborhood may include too many items from other classes choice of distance measure should have the property that a smaller distance means a greater likelihood of belonging to the same class specific measure may depend on the domain Euclidean distance becomes less discriminating as the number of attributes increases may need to scale attribute values to avoid having some dominate CPSC 444: Artificial Intelligence Spring CPSC 444: Artificial Intelligence Spring Idea. find the k nearest neighbors to the item in the dataset choose the majority class of the neighbors as the class for the item Challenges. combining class labels majority vote can be problematic if the neighbors vary widely in distance as all are given the same weight weighted vote weights a neighbor's vote by its distance d commonly 1/d 2 classification of an item is relatively expensive must locate the k nearest neighbors there are improvements e.g. condensing (eliminating stored items), proximity graphs (to quickly find neighbors) CPSC 444: Artificial Intelligence Spring CPSC 444: Artificial Intelligence Spring
2 easy to understand and implement fast to build model can perform well in many situations in spite of its simplicity Kernel used: Linear Kernel: K(x,y) = <x,y> * outlook=sunny * outlook=overcast * outlook=rainy * temperature=hot * temperature=mild * temperature=cool * humidity=normal * windy=false if input instance matches the specified value, 0 if not result < 0 denotes one class, > 0 the other a b < classified as 7 2 a = yes 3 2 b = no Correctly Classified Instances % CPSC 444: Artificial Intelligence Spring CPSC 444: Artificial Intelligence Spring Idea binary classification (two classes). find the hyperplane that maximizes the margin between the two classes margin = shortest distance between closest item to the plane and the plane H 1 does not separate H 2 separates with a small margin H 3 separates with maximum margin outlook temperature humidity windy play rainy mild normal FALSE yes overcast hot normal FALSE yes rainy cool normal FALSE yes rainy mild high FALSE yes overcast hot high FALSE yes overcast cool normal TRUE yes sunny cool normal FALSE yes overcast mild high TRUE yes sunny mild normal TRUE yes rainy mild high TRUE no rainy cool normal TRUE no sunny mild high FALSE no sunny hot high TRUE no sunny hot high FALSE no CPSC 444: Artificial Intelligence Spring CPSC 444: Artificial Intelligence Spring
3 Extensions. use a soft margin to handle errors allow some items to be on the wrong side of the plane with different kernel functions, can be used when classes aren't linearly separable outlook temperature humidity windy play rainy mild normal FALSE yes overcast hot normal FALSE yes rainy cool normal FALSE yes rainy mild high FALSE yes overcast hot high FALSE yes overcast cool normal TRUE yes -1 sunny cool normal FALSE yes overcast mild high TRUE yes sunny mild normal TRUE yes rainy mild high TRUE no rainy cool normal TRUE no sunny mild high FALSE no sunny hot high TRUE no sunny hot high FALSE no CPSC 444: Artificial Intelligence Spring CPSC 444: Artificial Intelligence Spring Kernel used: Poly Kernel: K(x,y) = <x,y>^ * < > * X] * < > * X] * < > * X] * < > * X] * < > * X] * < > * X] * < > * X] * < > * X] * < > * X] * < > * X] <x,y> denotes the dot product of vectors x and y (dot product = sum of the pairwise product of the components) X is the input instance to be classified < > * X refers to K(< >,X) a b < classified as 6 3 a = yes 3 2 b = no Correctly Classified Instances % Extensions. for multiple classes, use pairwise classification (1-vs-1) or one-against-all method pairwise train separate classifiers for each pairing of classes pick the majority classification one-against-all train separate classifiers for each class to distinguish that class from everything else pick the highest-confidence classification CPSC 444: Artificial Intelligence Spring CPSC 444: Artificial Intelligence Spring
4 SVM Multiple Classes Kernel used: Linear Kernel: K(x,y) = <x,y> Classifier for classes: Iris setosa, Iris versicolor * sepallength * sepalwidth * petallength * petalwidth Classifier for classes: Iris setosa, Iris virginica * sepallength * sepalwidth * petallength * petalwidth Classifier for classes: Iris versicolor, Iris virginica * sepallength * sepalwidth * petallength * petalwidth a b c Correctly Classified Instances % < classified as a = Iris setosa b = Iris versicolor c = Iris virginica CPSC 444: Artificial Intelligence Spring Idea binary classification (two classes). based on the posterior probability = probability of an occurrence given evidence assume attributes are independent idea example for yes outcomes, consider separately the probability of a rainy outlook, a mild temperature, a normal humidity, and not windy for independent attributes, the probability of all of these things happening at once is the product of the individual probabilities also factor in the likelihood of a yes outcome compare to no outcomes outlook temperature humidity windy rainy mild normal FALSE CPSC 444: Artificial Intelligence Spring has proven to be robust and accurate in many cases does not require large training sets not sensitive to the number of dimensions efficient training methods solid theoretical foundation Compute ln ( P(1 x) P(x 1) P(1) )=ln( P (0 x) P(x 0) P(0) ) P(i x) = probability of x belonging to class i P(i) = probability of an object belonging to class i the sign of the log indicates whether the probability of x belonging to class 1 is larger or smaller than the probability of x belonging to class 0 P(x i) = probability of x within class i if the components of x are independent, can estimate as the product of P(x j i) for each component x j of x sign of the result indicates the class challenge: if probabilities are estimated from the training set, it could be the case that P(x i i) = 0 solution: use Laplace smoothing use count+1 and total+number of possible values instead CPSC 444: Artificial Intelligence Spring CPSC 444: Artificial Intelligence Spring
5 Example Class Attribute yes no (0.63) (0.38) ============================= outlook sunny overcast rainy [total] temperature hot mild cool [total] humidity high normal [total] windy TRUE FALSE [total] uses Laplace smoothing, so counts are increased by 1 and totals are increased by the number of possible values for the attribute (avoids 0s if there are no training instances with a given value) a b < classified as 7 2 a = yes 4 1 b = no Correctly Classified Instances % CPSC 444: Artificial Intelligence Spring for more than two classes compute P(x i) P(i) for each class i choose the class i that maximizes P(x i) P(i) CPSC 444: Artificial Intelligence Spring Example Example outlook temperature humidity windy play sunny hot high TRUE no sunny hot high FALSE no rainy mild high TRUE no sunny mild high FALSE no rainy mild high FALSE yes sunny mild normal TRUE yes overcast mild high TRUE yes overcast hot high FALSE yes rainy cool normal TRUE no sunny cool normal FALSE yes rainy mild normal FALSE yes rainy cool normal FALSE yes overcast cool normal TRUE yes overcast hot normal FALSE yes CPSC 444: Artificial Intelligence Spring Class Attribute soft hard none (0.22) (0.19) (0.59) ========================================== age young pre presbyopic presbyopic [total] spectacle prescrip myope hypermetrope [total] astigmatism no yes [total] tear prod rate reduced normal [total] a b c < classified as a = soft b = hard c = none Correctly Classified Instances % CPSC 444: Artificial Intelligence Spring
6 age spectacle-prescrip astigmatism Example tearprod-rate contact -lenses soft hard none young hypermetrope yes normal hard pre-presbyopic myope yes normal hard presbyopic myope yes normal hard young myope yes normal hard young myope no reduced none young myope yes reduced none young hypermetrope no reduced none pre-presbyopic myope no reduced none young hypermetrope yes reduced none pre-presbyopic myope yes reduced none pre-presbyopic hypermetrope no reduced none presbyopic myope no reduced none pre-presbyopic hypermetrope yes normal none presbyopic hypermetrope yes normal none pre-presbyopic hypermetrope yes reduced none presbyopic myope yes reduced none presbyopic hypermetrope no reduced none presbyopic hypermetrope yes reduced none presbyopic myope no normal none presbyopic hypermetrope no normal soft young myope no normal soft pre-presbyopic myope no normal soft young hypermetrope no normal soft pre-presbyopic CPSC 444: Artificial Intelligence hypermetrope Spring 2019 no normal soft easy to implement easy to interpret / understand the resulting classification can be applied to large datasets tends to perform well frequently used in text classification and spam filtering many extensions / modifications Observations. assumption of independence of attributes is not necessarily a problem can start with attribute selection to eliminate highly correlated attributes even with correlated attributes, results based on the independence assumption aren't necessarily wrong CPSC 444: Artificial Intelligence Spring for numeric values discretize can assume a normal distribution and compute probabilities based on that Ensemble Learning Idea. use multiple classifiers to improve on the performance of any one Class Attribute Iris setosa Iris versicolor Iris virginica (0.33) (0.33) (0.33) =============================================================== sepallength mean std. dev weight sum precision sepalwidth mean std. dev weight sum precision petallength mean std. dev weight sum precision CPSC 444: Artificial Intelligence Spring CPSC 444: Artificial Intelligence Spring
7 AdaBoost AdaBoost works with weak classifiers (accuracy just above random chance) often decision stumps (single level decision trees) simple algorithm accurate often does not overfit (but it can) solid theoretical foundation CPSC 444: Artificial Intelligence Spring CPSC 444: Artificial Intelligence Spring AdaBoost Algorithm. assign equal weights (1/n) to each training instance n = size of the training set repeat for T rounds or until no further improvement train a classifier using the current training set weights if the classifier algorithm can't deal with weights directly, choose training elements in accordance with their weights test the classifier on the training examples and determine the error adjust the weights based on the error increase the weight of incorrectly classified examples use weighted majority voting amongst the classifiers from each round to determine the class weight is based on the error more accurate models are given higher weights CPSC 444: Artificial Intelligence Spring
Input: Concepts, Instances, Attributes
Input: Concepts, Instances, Attributes 1 Terminology Components of the input: Concepts: kinds of things that can be learned aim: intelligible and operational concept description Instances: the individual,
Homework 1 Sample Solution
Homework 1 Sample Solution 1. Iris: All attributes of iris are numeric, therefore ID3 of weka cannt be applied to this data set. Contact-lenses: tear-prod-rate = reduced: none tear-prod-rate = normal astigmatism
Data Mining Algorithms: Basic Methods
Algorithms: The basic methods Inferring rudimentary rules Data Mining Algorithms: Basic Methods Chapter 4 of Data Mining Statistical modeling Constructing decision trees Constructing rules Association
Representing structural patterns: Reading Material: Chapter 3 of the textbook by Witten
Representing structural patterns: Plain Classification rules Decision Tree Rules with exceptions Relational solution Tree for Numerical Prediction Instance-based presentation Reading Material: Chapter
Summary. Machine Learning: Introduction. Marcin Sydow
Outline of this Lecture Data Motivation for Data Mining and Learning Idea of Learning Decision Table: Cases and Attributes Supervised and Unsupervised Learning Classication and Regression Examples Data:
Machine Learning Chapter 2. Input
Machine Learning Chapter 2. Input 2 Input: Concepts, instances, attributes Terminology What s a concept? Classification, association, clustering, numeric prediction What s in an example? Relations, flat
Data Mining. Practical Machine Learning Tools and Techniques. Slides for Chapter 3 of Data Mining by I. H. Witten, E. Frank and M. A.
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 3 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Input: Concepts, instances, attributes Terminology What s a concept?
Data Mining Practical Machine Learning Tools and Techniques
Input: Concepts, instances, attributes Data ining Practical achine Learning Tools and Techniques Slides for Chapter 2 of Data ining by I. H. Witten and E. rank Terminology What s a concept z Classification,
Data Mining Part 4. Tony C Smith WEKA Machine Learning Group Department of Computer Science University of Waikato
Data Mining Part 4 Tony C Smith WEKA Machine Learning Group Department of Computer Science University of Waikato Algorithms: The basic methods Inferring rudimentary rules Statistical modeling Constructing
Data Mining. Covering algorithms. Covering approach At each stage you identify a rule that covers some of instances. Fig. 4.
Data Mining Chapter 4. Algorithms: The Basic Methods (Covering algorithm, Association rule, Linear models, Instance-based learning, Clustering) 1 Covering approach At each stage you identify a rule that
Machine Learning: Algorithms and Applications Mockup Examination
Machine Learning: Algorithms and Applications Mockup Examination 14 May 2012 FIRST NAME STUDENT NUMBER LAST NAME SIGNATURE Instructions for students Write First Name, Last Name, Student Number and Signature
Basic Concepts Weka Workbench and its terminology
Changelog: 14 Oct, 30 Oct Basic Concepts Weka Workbench and its terminology Lecture Part Outline Concepts, instances, attributes How to prepare the input: ARFF, attributes, missing values, getting to know
Chapter 4: Algorithms CS 795
Chapter 4: Algorithms CS 795 Inferring Rudimentary Rules 1R Single rule one level decision tree Pick each attribute and form a single level tree without overfitting and with minimal branches Pick that
Practical Data Mining COMP-321B. Tutorial 1: Introduction to the WEKA Explorer
Practical Data Mining COMP-321B Tutorial 1: Introduction to the WEKA Explorer Gabi Schmidberger Mark Hall Richard Kirkby July 12, 2006 c 2006 University of Waikato 1 Setting up your Environment Before
Classification with Decision Tree Induction
Classification with Decision Tree Induction This algorithm makes Classification Decision for a test sample with the help of tree like structure (Similar to Binary Tree OR k-ary tree) Nodes in the tree
Tour-Based Mode Choice Modeling: Using An Ensemble of (Un-) Conditional Data-Mining Classifiers
Tour-Based Mode Choice Modeling: Using An Ensemble of (Un-) Conditional Data-Mining Classifiers James P. Biagioni Piotr M. Szczurek Peter C. Nelson, Ph.D. Abolfazl Mohammadian, Ph.D. Agenda Background
Data Mining Tools. Jean-Gabriel Ganascia LIP6 University Pierre et Marie Curie 4, place Jussieu, Paris, Cedex 05
Data Mining Tools Jean-Gabriel Ganascia LIP6 University Pierre et Marie Curie 4, place Jussieu, 75252 Paris, Cedex 05 Jean-Gabriel.Ganascia@lip6.fr DATA BASES Data mining Extraction Data mining Interpretation/
Chapter 4: Algorithms CS 795
Chapter 4: Algorithms CS 795 Inferring Rudimentary Rules 1R Single rule one level decision tree Pick each attribute and form a single level tree without overfitting and with minimal branches Pick that
Naïve Bayes Classification. Material borrowed from Jonathan Huang and I. H. Witten s and E. Frank s Data Mining and Jeremy Wyatt and others
Naïve Bayes Classification Material borrowed from Jonathan Huang and I. H. Witten s and E. Frank s Data Mining and Jeremy Wyatt and others Things We d Like to Do Spam Classification Given an email, predict
Data Mining and Analytics
Data Mining and Analytics Aik Choon Tan, Ph.D. Associate Professor of Bioinformatics Division of Medical Oncology Department of Medicine aikchoon.tan@ucdenver.edu 9/22/2017 http://tanlab.ucdenver.edu/labhomepage/teaching/bsbt6111/
ECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
Data Mining and Knowledge Discovery Practice notes: Numeric Prediction, Association Rules
Keywords Data Mining and Knowledge Discovery: Practice Notes Petra Kralj Novak Petra.Kralj.Novak@ijs.si Data Attribute, example, attribute-value data, target variable, class, discretization Algorithms
Data Mining. Practical Machine Learning Tools and Techniques. Slides for Chapter 3 of Data Mining by I. H. Witten, E. Frank and M. A.
Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 3 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Output: Knowledge representation Tables Linear models Trees Rules
Decision Trees In Weka,Data Formats
CS 4510/9010 Applied Machine Learning 1 Decision Trees In Weka,Data Formats Paula Matuszek Fall, 2016 J48: Decision Tree in Weka 2 NAME: weka.classifiers.trees.j48 SYNOPSIS Class for generating a pruned
k Nearest Neighbors Super simple idea! Instance-based learning as opposed to model-based (no pre-processing)
k Nearest Neighbors k Nearest Neighbors To classify an observation: Look at the labels of some number, say k, of neighboring observations. The observation is then classified based on its nearest neighbors
Naïve Bayes for text classification
Road Map Basic concepts Decision tree induction Evaluation of classifiers Rule induction Classification using association rules Naïve Bayesian classification Naïve Bayes for text classification Support
Performance Analysis of Data Mining Classification Techniques
Performance Analysis of Data Mining Classification Techniques Tejas Mehta 1, Dr. Dhaval Kathiriya 2 Ph.D. Student, School of Computer Science, Dr. Babasaheb Ambedkar Open University, Gujarat, India 1 Principal
DATA MINING INTRODUCTION TO CLASSIFICATION USING LINEAR CLASSIFIERS
DATA MINING INTRODUCTION TO CLASSIFICATION USING LINEAR CLASSIFIERS 1 Classification: Definition Given a collection of records (training set ) Each record contains a set of attributes and a class attribute
9/6/14. Our first learning algorithm. Comp 135 Introduction to Machine Learning and Data Mining. knn Algorithm. knn Algorithm (simple form)
Comp 135 Introduction to Machine Learning and Data Mining Our first learning algorithm How would you classify the next example? Fall 2014 Professor: Roni Khardon Computer Science Tufts University o o o
DATA MINING LECTURE 10B. Classification k-nearest neighbor classifier Naïve Bayes Logistic Regression Support Vector Machines
DATA MINING LECTURE 10B Classification k-nearest neighbor classifier Naïve Bayes Logistic Regression Support Vector Machines NEAREST NEIGHBOR CLASSIFICATION 10 10 Illustrating Classification Task Tid Attrib1
Data Representation Information Retrieval and Data Mining. Prof. Matteo Matteucci
Data Representation Information Retrieval and Data Mining Prof. Matteo Matteucci Instances, Attributes, Concepts 2 Instances The atomic elements of information from a dataset Also known as records, prototypes,
Support Vector Machines
Support Vector Machines About the Name... A Support Vector A training sample used to define classification boundaries in SVMs located near class boundaries Support Vector Machines Binary classifiers whose
Applying Supervised Learning
Applying Supervised Learning When to Consider Supervised Learning A supervised learning algorithm takes a known set of input data (the training set) and known responses to the data (output), and trains
Feature Extractors. CS 188: Artificial Intelligence Fall Some (Vague) Biology. The Binary Perceptron. Binary Decision Rule.
CS 188: Artificial Intelligence Fall 2008 Lecture 24: Perceptrons II 11/24/2008 Dan Klein UC Berkeley Feature Extractors A feature extractor maps inputs to feature vectors Dear Sir. First, I must solicit
Data Mining and Machine Learning: Techniques and Algorithms
Instance based classification Data Mining and Machine Learning: Techniques and Algorithms Eneldo Loza Mencía eneldo@ke.tu-darmstadt.de Knowledge Engineering Group, TU Darmstadt International Week 2019,
We extend SVM s in order to support multi-class classification problems. Consider the training dataset
p. / One-versus-the-Rest We extend SVM s in order to support multi-class classification problems. Consider the training dataset D = {(x, y ),(x, y ),..., (x l, y l )} R n {,..., M}, where the label y i
Announcements. CS 188: Artificial Intelligence Spring Generative vs. Discriminative. Classification: Feature Vectors. Project 4: due Friday.
CS 188: Artificial Intelligence Spring 2011 Lecture 21: Perceptrons 4/13/2010 Announcements Project 4: due Friday. Final Contest: up and running! Project 5 out! Pieter Abbeel UC Berkeley Many slides adapted
Data Mining Practical Machine Learning Tools and Techniques
Output: Knowledge representation Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter of Data Mining by I. H. Witten and E. Frank Decision tables Decision trees Decision rules
Data Mining and Machine Learning. Instance-Based Learning. Rote Learning k Nearest-Neighbor Classification. IBL and Rule Learning
Data Mining and Machine Learning Instance-Based Learning Rote Learning k Nearest-Neighbor Classification Prediction, Weighted Prediction choosing k feature weighting (RELIEF) instance weighting (PEBLS)
Introduction to Artificial Intelligence
Introduction to Artificial Intelligence COMP307 Machine Learning 2: 3-K Techniques Yi Mei yi.mei@ecs.vuw.ac.nz 1 Outline K-Nearest Neighbour method Classification (Supervised learning) Basic NN (1-NN)
Decision Tree Learning
Decision Tree Learning 1 Simple example of object classification Instances Size Color Shape C(x) x1 small red circle positive x2 large red circle positive x3 small red triangle negative x4 large blue circle
The Curse of Dimensionality
The Curse of Dimensionality ACAS 2002 p1/66 Curse of Dimensionality The basic idea of the curse of dimensionality is that high dimensional data is difficult to work with for several reasons: Adding more
Data Mining and Knowledge Discovery Practice notes 2
Keywords Data Mining and Knowledge Discovery: Practice Notes Petra Kralj Novak Petra.Kralj.Novak@ijs.si Data Attribute, example, attribute-value data, target variable, class, discretization Algorithms
Supervised vs unsupervised clustering
Classification Supervised vs unsupervised clustering Cluster analysis: Classes are not known a- priori. Classification: Classes are defined a-priori Sometimes called supervised clustering Extract useful
Performance Evaluation of Various Classification Algorithms
Performance Evaluation of Various Classification Algorithms Shafali Deora Amritsar College of Engineering & Technology, Punjab Technical University -----------------------------------------------------------***----------------------------------------------------------
KTH ROYAL INSTITUTE OF TECHNOLOGY. Lecture 14 Machine Learning. K-means, knn
KTH ROYAL INSTITUTE OF TECHNOLOGY Lecture 14 Machine Learning. K-means, knn Contents K-means clustering K-Nearest Neighbour Power Systems Analysis An automated learning approach Understanding states in
Naïve Bayes Classification. Material borrowed from Jonathan Huang and I. H. Witten s and E. Frank s Data Mining and Jeremy Wyatt and others
Naïve Bayes Classification Material borrowed from Jonathan Huang and I. H. Witten s and E. Frank s Data Mining and Jeremy Wyatt and others Things We d Like to Do Spam Classification Given an email, predict
Prediction. What is Prediction. Simple methods for Prediction. Classification by decision tree induction. Classification and regression evaluation
Prediction Prediction What is Prediction Simple methods for Prediction Classification by decision tree induction Classification and regression evaluation 2 Prediction Goal: to predict the value of a given
Ensemble Learning. Another approach is to leverage the algorithms we have via ensemble methods
Ensemble Learning Ensemble Learning So far we have seen learning algorithms that take a training set and output a classifier What if we want more accuracy than current algorithms afford? Develop new learning
Data Mining and Knowledge Discovery: Practice Notes
Data Mining and Knowledge Discovery: Practice Notes Petra Kralj Novak Petra.Kralj.Novak@ijs.si 8.11.2017 1 Keywords Data Attribute, example, attribute-value data, target variable, class, discretization
Artificial Intelligence. Programming Styles
Artificial Intelligence Intro to Machine Learning Programming Styles Standard CS: Explicitly program computer to do something Early AI: Derive a problem description (state) and use general algorithms to
CS 584 Data Mining. Classification 1
CS 584 Data Mining Classification 1 Classification: Definition Given a collection of records (training set ) Each record contains a set of attributes, one of the attributes is the class. Find a model for
Topics in Machine Learning
Topics in Machine Learning Gilad Lerman School of Mathematics University of Minnesota Text/slides stolen from G. James, D. Witten, T. Hastie, R. Tibshirani and A. Ng Machine Learning - Motivation Arthur
Lecture 6: Unsupervised Machine Learning Dagmar Gromann International Center For Computational Logic
SEMANTIC COMPUTING Lecture 6: Unsupervised Machine Learning Dagmar Gromann International Center For Computational Logic TU Dresden, 23 November 2018 Overview Unsupervised Machine Learning overview Association
Text Categorization. Foundations of Statistic Natural Language Processing The MIT Press1999
Text Categorization Foundations of Statistic Natural Language Processing The MIT Press1999 Outline Introduction Decision Trees Maximum Entropy Modeling (optional) Perceptrons K Nearest Neighbor Classification
Nominal Data. May not have a numerical representation Distance measures might not make sense. PR and ANN
NonMetric Data Nominal Data So far we consider patterns to be represented by feature vectors of real or integer values Easy to come up with a distance (similarity) measure by using a variety of mathematical
Supervised Learning: K-Nearest Neighbors and Decision Trees
Supervised Learning: K-Nearest Neighbors and Decision Trees Piyush Rai CS5350/6350: Machine Learning August 25, 2011 (CS5350/6350) K-NN and DT August 25, 2011 1 / 20 Supervised Learning Given training
Data Mining Input: Concepts, Instances, and Attributes
Data Mining Input: Concepts, Instances, and Attributes Chapter 2 of Data Mining Terminology Components of the input: Concepts: kinds of things that can be learned Goal: intelligible and operational concept
CSCI567 Machine Learning (Fall 2014)
CSCI567 Machine Learning (Fall 2014) Drs. Sha & Liu {feisha,yanliu.cs}@usc.edu September 9, 2014 Drs. Sha & Liu ({feisha,yanliu.cs}@usc.edu) CSCI567 Machine Learning (Fall 2014) September 9, 2014 1 / 47
Computational Statistics The basics of maximum likelihood estimation, Bayesian estimation, object recognitions
Computational Statistics The basics of maximum likelihood estimation, Bayesian estimation, object recognitions Thomas Giraud Simon Chabot October 12, 2013 Contents 1 Discriminant analysis 3 1.1 Main idea................................
Introduction to Machine Learning CANB 7640
Introduction to Machine Learning CANB 7640 Aik Choon Tan, Ph.D. Associate Professor of Bioinformatics Division of Medical Oncology Department of Medicine aikchoon.tan@ucdenver.edu 9/5/2017 http://tanlab.ucdenver.edu/labhomepage/teaching/canb7640/
Final Exam. Introduction to Artificial Intelligence. CS 188 Spring 2010 INSTRUCTIONS. You have 3 hours.
CS 188 Spring 2010 Introduction to Artificial Intelligence Final Exam INSTRUCTIONS You have 3 hours. The exam is closed book, closed notes except a two-page crib sheet. Please use non-programmable calculators
CS6375: Machine Learning Gautam Kunapuli. Mid-Term Review
Gautam Kunapuli Machine Learning Data is identically and independently distributed Goal is to learn a function that maps to Data is generated using an unknown function Learn a hypothesis that minimizes
Decision Trees: Discussion
Decision Trees: Discussion Machine Learning Spring 2018 The slides are mainly from Vivek Srikumar 1 This lecture: Learning Decision Trees 1. Representation: What are decision trees? 2. Algorithm: Learning
Data Mining Practical Machine Learning Tools and Techniques
Algorithms: The basic methods Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter of Data Mining by I. H. Witten and. Frank Inferring rudimentary rules Statistical modeling Constructing
Data analysis case study using R for readily available data set using any one machine learning Algorithm
Assignment-4 Data analysis case study using R for readily available data set using any one machine learning Algorithm Broadly, there are 3 types of Machine Learning Algorithms.. 1. Supervised Learning
Data Mining - Data. Dr. Jean-Michel RICHER Dr. Jean-Michel RICHER Data Mining - Data 1 / 47
Data Mining - Data Dr. Jean-Michel RICHER 2018 jean-michel.richer@univ-angers.fr Dr. Jean-Michel RICHER Data Mining - Data 1 / 47 Outline 1. Introduction 2. Data preprocessing 3. CPA with R 4. Exercise
Application of Support Vector Machine Algorithm in Spam Filtering
Application of Support Vector Machine Algorithm in E-Mail Spam Filtering Julia Bluszcz, Daria Fitisova, Alexander Hamann, Alexey Trifonov, Advisor: Patrick Jähnichen Abstract The problem of spam classification
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 12 Combining
Association Rules. Charles Sutton Data Mining and Exploration Spring Based on slides by Chris Williams and Amos Storkey. Thursday, 8 March 12
Association Rules Charles Sutton Data Mining and Exploration Spring 2012 Based on slides by Chris Williams and Amos Storkey The Goal Find patterns : local regularities that occur more often than you would
Unsupervised: no target value to predict
Clustering Unsupervised: no target value to predict Differences between models/algorithms: Exclusive vs. overlapping Deterministic vs. probabilistic Hierarchical vs. flat Incremental vs. batch learning
Normalization and denormalization Missing values Outliers detection and removing Noisy Data Variants of Attributes Meta Data Data Transformation
Preprocessing Data Normalization and denormalization Missing values Outliers detection and removing Noisy Data Variants of Attributes Meta Data Data Transformation Reading material: Chapters 2 and 3 of
What is Learning? CS 343: Artificial Intelligence Machine Learning. Raymond J. Mooney. Problem Solving / Planning / Control.
What is Learning? CS 343: Artificial Intelligence Machine Learning Herbert Simon: Learning is any process by which a system improves performance from experience. What is the task? Classification Problem
Ensemble Learning: An Introduction. Adapted from Slides by Tan, Steinbach, Kumar
Ensemble Learning: An Introduction Adapted from Slides by Tan, Steinbach, Kumar 1 General Idea D Original Training data Step 1: Create Multiple Data Sets... D 1 D 2 D t-1 D t Step 2: Build Multiple Classifiers
To earn the extra credit, one of the following has to hold true. Please circle and sign.
CS 188 Spring 2011 Introduction to Artificial Intelligence Practice Final Exam To earn the extra credit, one of the following has to hold true. Please circle and sign. A I spent 3 or more hours on the
arulescba: Classification for Factor and Transactional Data Sets Using Association Rules
arulescba: Classification for Factor and Transactional Data Sets Using Association Rules Ian Johnson Southern Methodist University Abstract This paper presents an R package, arulescba, which uses association
CSEP 573: Artificial Intelligence
CSEP 573: Artificial Intelligence Machine Learning: Perceptron Ali Farhadi Many slides over the course adapted from Luke Zettlemoyer and Dan Klein. 1 Generative vs. Discriminative Generative classifiers:
Multi-label classification using rule-based classifier systems
Multi-label classification using rule-based classifier systems Shabnam Nazmi (PhD candidate) Department of electrical and computer engineering North Carolina A&T state university Advisor: Dr. A. Homaifar
Nominal Data. May not have a numerical representation Distance measures might not make sense PR, ANN, & ML
Decision Trees Nominal Data So far we consider patterns to be represented by feature vectors of real or integer values Easy to come up with a distance (similarity) measure by using a variety of mathematical
Slides for Data Mining by I. H. Witten and E. Frank
Slides for Data Mining by I. H. Witten and E. Frank 7 Engineering the input and output Attribute selection Scheme-independent, scheme-specific Attribute discretization Unsupervised, supervised, error-
Data Mining and Knowledge Discovery: Practice Notes
Data Mining and Knowledge Discovery: Practice Notes Petra Kralj Novak Petra.Kralj.Novak@ijs.si 2016/01/12 1 Keywords Data Attribute, example, attribute-value data, target variable, class, discretization
Slides for Data Mining by I. H. Witten and E. Frank
Slides for Data Mining by I. H. Witten and E. Frank 6 Implementation: Real machine learning schemes Decision trees: from ID3 to C4.5 Pruning, missing values, numeric attributes, efficiency Decision rules:
BITS F464: MACHINE LEARNING
BITS F464: MACHINE LEARNING Lecture-16: Decision Tree (contd.) + Random Forest Dr. Kamlesh Tiwari Assistant Professor Department of Computer Science and Information Systems Engineering, BITS Pilani, Rajasthan-333031
CS513-Data Mining. Lecture 2: Understanding the Data. Waheed Noor
CS513-Data Mining Lecture 2: Understanding the Data Waheed Noor Computer Science and Information Technology, University of Balochistan, Quetta, Pakistan Waheed Noor (CS&IT, UoB, Quetta) CS513-Data Mining
Stat 602X Exam 2 Spring 2011
Stat 60X Exam Spring 0 I have neither given nor received unauthorized assistance on this exam. Name Signed Date Name Printed . Below is a small p classification training set (for classes) displayed in
Introduction to R and Statistical Data Analysis
Microarray Center Introduction to R and Statistical Data Analysis PART II Petr Nazarov petr.nazarov@crp-sante.lu 22-11-2010 OUTLINE PART II Descriptive statistics in R (8) sum, mean, median, sd, var, cor,
Supervised Learning (contd) Linear Separation. Mausam (based on slides by UW-AI faculty)
Supervised Learning (contd) Linear Separation Mausam (based on slides by UW-AI faculty) Images as Vectors Binary handwritten characters Treat an image as a highdimensional vector (e.g., by reading pixel
Machine Learning. Classification
10-701 Machine Learning Classification Inputs Inputs Inputs Where we are Density Estimator Probability Classifier Predict category Today Regressor Predict real no. Later Classification Assume we want to
Robot Learning. There are generally three types of robot learning: Learning from data. Learning by demonstration. Reinforcement learning
Robot Learning 1 General Pipeline 1. Data acquisition (e.g., from 3D sensors) 2. Feature extraction and representation construction 3. Robot learning: e.g., classification (recognition) or clustering (knowledge
Based on Raymond J. Mooney s slides
Instance Based Learning Based on Raymond J. Mooney s slides University of Texas at Austin 1 Example 2 Instance-Based Learning Unlike other learning algorithms, does not involve construction of an explicit
Ensemble Methods, Decision Trees
CS 1675: Intro to Machine Learning Ensemble Methods, Decision Trees Prof. Adriana Kovashka University of Pittsburgh November 13, 2018 Plan for This Lecture Ensemble methods: introduction Boosting Algorithm
COMP33111: Tutorial and lab exercise 7
COMP33111: Tutorial and lab exercise 7 Guide answers for Part 1: Understanding clustering 1. Explain the main differences between classification and clustering. main differences should include being unsupervised
Data Mining Classification: Alternative Techniques. Lecture Notes for Chapter 4. Instance-Based Learning. Introduction to Data Mining, 2 nd Edition
Data Mining Classification: Alternative Techniques Lecture Notes for Chapter 4 Instance-Based Learning Introduction to Data Mining, 2 nd Edition by Tan, Steinbach, Karpatne, Kumar Instance Based Classifiers
Support Vector Machines
Support Vector Machines . Importance of SVM SVM is a discriminative method that brings together:. computational learning theory. previously known methods in linear discriminant functions 3. optimization
Search Engines. Information Retrieval in Practice
Search Engines Information Retrieval in Practice All slides Addison Wesley, 2008 Classification and Clustering Classification and clustering are classical pattern recognition / machine learning problems
Lecture 9: Support Vector Machines
Lecture 9: Support Vector Machines William Webber (william@williamwebber.com) COMP90042, 2014, Semester 1, Lecture 8 What we ll learn in this lecture Support Vector Machines (SVMs) a highly robust and
Modern Medical Image Analysis 8DC00 Exam
Parts of answers are inside square brackets [... ]. These parts are optional. Answers can be written in Dutch or in English, as you prefer. You can use drawings and diagrams to support your textual answers.
Hsiaochun Hsu Date: 12/12/15. Support Vector Machine With Data Reduction
Support Vector Machine With Data Reduction 1 Table of Contents Summary... 3 1. Introduction of Support Vector Machines... 3 1.1 Brief Introduction of Support Vector Machines... 3 1.2 SVM Simple Experiment...
Recognition Part I: Machine Learning. CSE 455 Linda Shapiro
Recognition Part I: Machine Learning CSE 455 Linda Shapiro Visual Recognition What does it mean to see? What is where, Marr 1982 Get computers to see Visual Recognition Verification Is this a car? Visual
Contents. Preface to the Second Edition
Preface to the Second Edition v 1 Introduction 1 1.1 What Is Data Mining?....................... 4 1.2 Motivating Challenges....................... 5 1.3 The Origins of Data Mining....................