Pattern Recognition Winter Semester 2009/2010. Exemplary Solution
|
|
- MargaretMargaret Walton
- 5 years ago
- Views:
Transcription
1 Pattern Recognition Winter Semester 2009/2010 Exemplary Solution 4 Exercise The focus of this exercise will be neural networks. You will train and apply a multi-layer perceptron (MLP) as well as a radial basis function network (RBF) on the Imaging Mass Spectrometry Dataset (again we will only use the first two mass channels to simplify visualization). We will use the netlab toolbox ( this way you do not have to implement the neural networks yourself, but can rely on existing matlab code and focus on the interpretation. 4.1 Netlab Toolbox Example To familiarize yourself with the netlab toolbox, follow through the example below: % Generate the matrix of inputs x and targets t. x = [0:1/19:1]'; ndata = size(x,1); t = sin(2*pi*x) + 0.2*randn(ndata, 1); %Set up network parameters. nin = 1; % Number of inputs. nhidden = 3; % Number of hidden units. nout = 1; % Number of outputs. outfunction= linear ; % outfunction (linear, logistic ) %alpha = 0.01; % Coefficient of weight-decay prior. net = mlp(nin, nhidden, nout, outfunction); options(1) = 1; % This provides display of error values. options(9) = 1; % Check the gradient calculations. options(14) = 1000; % Number of training cycles. % Train using scaled conjugate gradients. [net, options] = netopt(net, options, x, t, 'scg'); % Plot the trained network predictions. plotvals = [0:0.01:1]'; y = mlpfwd(net, plotvals); plot(plotvals, y, 'ob') Interpret the example (1 point) Print the resulting plot and comment on what the program does and also what the quality of the result looks like.
2 The example code produces datapoints from a sine curve with additional noise and the mlp with 3 hidden units is trained on 1000 iterations on the data. The plot shows the successful recovery of the sine structure from only 20 data points Influence of noise on MLP (2 points) The example in 4.1 uses a fixed noise factor of 0.2 investigate the performance of the MLP when you change the noise level in the interval [0,2] create plots of the results and comment. The code below was used to repeat the example program with various noise levels: for noise=0:0.5:2 % Generate the matrix of inputs x and targets t. x = [0:1/19:1]'; ndata = size(x,1); t = sin(2*pi*x) + noise*randn(ndata, 1); %Set up network parameters. nin = 1; % Number of inputs. nhidden = 3; % Number of hidden units. nout = 1; % Number of outputs. outfunction='linear'; % outfunction (linear, logistic ) %alpha = 0.01; % Coefficient of weight-decay prior. net = mlp(nin, nhidden, nout, outfunction); %options(1) = 1; % This provides display of error values. %options(9) = 1; % Check the gradient calculations. options(14) = 1000; % Number of training cycles. % Train using scaled conjugate gradients. [net, options] = netopt(net, options, x, t, 'scg'); % Plot the trained network predictions. plotvals = [0:0.01:1]'; y = mlpfwd(net, plotvals);
3 figure; plot(plotvals, y, 'ob'); hold; plot(x,t,'or'); title(noise); xlabel('x'); ylabel('f(x)');
4 It is evident that with the increasing noise levels, the quality of the prediction decreases rapidly. While at a noise level of 0.5, the function shows at least the same general overall tr, that is no longer true for higher values. Still, it should be noted that for high noise levels, this task would also be unfeasible for a human. 4.2 Multi Layer Perceptron Application to breast cancer data set (5 points) Set up a Multi-Layer Perceptron on the training data (again reduced to the first 2 mass channels) of the imaging mass spectrometry data set with the same parameters as in the example above and compute the correct classification rate on the training as well as the test data. Repeat this step several times and evaluate the results (round the prediction results to 0/1!) - do you think the settings are useful? Comment on the choice of each parameter and make a plot of the classifications and approximate the decision boundary. training2c=trainingx(1:2,:); test2c=testx(1:2,:); net = mlp(2, 3, 1, 'linear'); options(1) = 1; % This provides display of error values. options(9) = 1; % Check the gradient calculations. options(14) = 100; % Number of training cycles.
5 % Train using scaled conjugate gradients. [net, options] = netopt(net, options, training2c', trainingy', 'scg'); predicty = mlpfwd(net, training2c'); sum(abs(round(predicty)'-trainingy)) predicty = mlpfwd(net, test2c'); sum(abs(round(predicty)'-testy)) Repeated 10 times, we see strong variation in the number of incorrect predictions on the test data set (33,34,33,34,105,28,33,104,32,33). This indicates that the number of iterations are not sufficient to reliably train the model. Also, the linear outfunction is not a wise choice since we are dealing with 1/0 labeled data, thus a logistic outfunction would be a more natural fit. The low number of misclassification (at least with regard to the median of the various runs) on the test data suggests that 3 hidden units are already sufficient to separate the data. This is also supported by the fact that we see almost 100% correct classifications on the training data (again as a median, even their we see strongly divergent results). This is also connected to the good separation achieved by the two mass channels into the two label classes. For more complex data, more hidden units might be needed. % and now plot training data figure; plot(training2c(1,round(predicty)==1),training2c(2,round(predicty )==1),'oc') title('mlp results on training data'); xlabel('mass channel 1'); ylabel('mass channel 2'); hold plot(training2c(1,round(predicty)==0),training2c(2,round(predicty )==0),'og') plot(training2c(1,trainingy==1),training2c(2,trainingy==1),'xr') plot(training2c(1,trainingy==0),training2c(2,trainingy==0),'xy') % and test data
6 figure; plot(test2c(1,round(predicty)==1),test2c(2,round(predicty)==1),'o c') title('mlp results on test data'); xlabel('mass channel 1'); ylabel('mass channel 2'); hold plot(test2c(1,round(predicty)==0),test2c(2,round(predicty)==0),'o g') plot(test2c(1,testy==1),test2c(2,testy==1),'xr') plot(test2c(1,testy==0),test2c(2,testy==0),'xy') %plot decision boundary %produce grid of input values: decisiongrid=zeros(100,100); for i=1:100; for j =1:100; %perform qda decisiongrid(i,j)= round(mlpfwd(net, [i, j])); %make a plot imagesc(decisiongrid') title('decision boundary for MLP'); xlabel('mass channel 1'); ylabel('mass channel 2');
7 4.2.2 Influence of Parameters on MLP performance (3 points) Compute a MLP with 50 hidden units and a logistic outfunction and use 5000 iterations to fit it. What is the error on the training data? What is the error on the test data? What do you think is the reason for a difference? Comment. Using these settings, we obtain far more stable results over various runs, 5000 iterations and 50 hidden units appear (more than) suitable for fitting a stable network. But we see reasons to expect overfitting since at least in some of the iterations, the training error went down to zero and the decision boundary closely follows the training data. We do not see a decrease in the number of misclassifications on the test data, they are constant around 34, but we do not have any large outliers any longer. net = mlp(2, 50, 1, 'logistic'); options(14) = 5000; % Number of training cycles. % Train using scaled conjugate gradients. [net, options] = netopt(net, options, training2c', trainingy', 'scg'); predicty = mlpfwd(net, training2c'); sum(abs(round(predicty)'-trainingy)) predicty = mlpfwd(net, test2c'); sum(abs(round(predicty)'-testy)) Parameter tuning (3 points) What do you think is a sensible number of hidden units and iterations? What is an appropriate outfunction? Comment and then compute the results using the setting you proposed. If the results are not significantly better than the results in and what do you think is the reason?
8 It becomes evident from and that using sufficient iterations and hidden units is necessary to have stable results, but too many might lead to overfitting. A logistic outfunction is strongly suggested by the data. Therefore, we try the following model: net = mlp(2, 15, 1, 'logistic'); options(14) = 1500; % Number of training cycles. % Train using scaled conjugate gradients. [net, options] = netopt(net, options, training2c', trainingy', 'scg'); predicty = mlpfwd(net, training2c'); sum(abs(round(predicty)'-trainingy)) predicty = mlpfwd(net, test2c'); sum(abs(round(predicty)'-testy)) Overall, we only see very moderate changes between the parametrizations. With the above setting, we obtain approximately 30 misclassifications on the test data set. We do not observe large outliers and have lower training costs and fewer signs of overfitting than with the more complex model of Radial Basis Function Network (6 points) Now use a RBF network (read demrbf1.m to familiarize yourself with the most important commands) and train a network with 2 hidden units and a Gaussian radial basis functions. What does the correct classification rate on the training and test data look like? Explain what the model you applied corresponds to and how you could interpret it. % Create and initialize network weight and parameter vectors. net = rbf(2, 2, 1, 'gaussian'); options = foptions; options(1) = 1; % Display EM training options(14) = 20; % number of iterations of EM net = rbftrain(net, options, training2c', trainingy'); predicty = rbffwd(net, training2c'); sum(abs(round(predicty)'-trainingy)) predicty = rbffwd(net, test2c'); sum(abs(round(predicty)'-testy)) The RBF network with two Gaussian basis functions is very closely related to QDA. The
9 difference is that while in QDA the covariance matrix was directly estimated from the data using a maximum likelihood approach, it is learned iteratively in the RBF network. We see more misclassifications (7) on the training data set than with the MLP, but we see fewer misclassification (24) on the test data set. This indicates that the RBF network with so little complexity (only 2 Gaussians) generalizes very well and captures the essence of the data without the danger of overfitting. 4.4 Bonus: PCA computation using Neural Networks (6 points) Now, use Oja s Rule to implement a neural network which computes the first principal component. Write a function w=function(trainingx, rate, delta, maxiter) which takes the p x n dimensional training data with n observations as an input as well as the learning rate rate, the convergence criterion delta, which the norm of the residuals is compared to in each step, and the maximum number of iterations maxiter as parameters and outputs the first principal component w. Oja s Rule states that for each observation i; the current result is computed by y = w'*x(:,i); and w is updated by w = w + rate*(y*trainingx(:,i) - y^2*w); Normalizing w simplifies the convergence. Comment on the performance of the approach. function w = oja(x, rate,delta,maxiteration) [m n] = size(x); % X is the set of input vectors. Each column of X is one sample. % w is the first principal component % % random initial weights w=rand(m,1); % normalize weights w=w/norm(w); % run through all input samples for iter = 1:maxIteration wold = w; %keep old results for residuals computation for ii = 1:n y = w'*x(:,ii); w = w + rate*(y*x(:,ii) - y^2*w); %normalize again w=w/norm(w); if (norm(w - wold) < delta) %compare residuals to delta break; The algorithm is strongly depent on the input values and might not converge too easily in many settings thus many iterations are required.
Exercise: Training Simple MLP by Backpropagation. Using Netlab.
Exercise: Training Simple MLP by Backpropagation. Using Netlab. Petr Pošík December, 27 File list This document is an explanation text to the following script: demomlpklin.m script implementing the beckpropagation
More informationModel Answers to The Next Pixel Prediction Task
Model Answers to The Next Pixel Prediction Task December 2, 25. (Data preprocessing and visualization, 8 marks) (a) Solution. In Algorithm we are told that the data was discretized to 64 grey scale values,...,
More informationCPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:
More informationChap.12 Kernel methods [Book, Chap.7]
Chap.12 Kernel methods [Book, Chap.7] Neural network methods became popular in the mid to late 1980s, but by the mid to late 1990s, kernel methods have also become popular in machine learning. The first
More informationClassification and Regression using Linear Networks, Multilayer Perceptrons and Radial Basis Functions
ENEE 739Q SPRING 2002 COURSE ASSIGNMENT 2 REPORT 1 Classification and Regression using Linear Networks, Multilayer Perceptrons and Radial Basis Functions Vikas Chandrakant Raykar Abstract The aim of the
More informationProgramming Exercise 4: Neural Networks Learning
Programming Exercise 4: Neural Networks Learning Machine Learning Introduction In this exercise, you will implement the backpropagation algorithm for neural networks and apply it to the task of hand-written
More informationRecitation Supplement: Creating a Neural Network for Classification SAS EM December 2, 2002
Recitation Supplement: Creating a Neural Network for Classification SAS EM December 2, 2002 Introduction Neural networks are flexible nonlinear models that can be used for regression and classification
More informationWhat is machine learning?
Machine learning, pattern recognition and statistical data modelling Lecture 12. The last lecture Coryn Bailer-Jones 1 What is machine learning? Data description and interpretation finding simpler relationship
More informationLearning from Data: Adaptive Basis Functions
Learning from Data: Adaptive Basis Functions November 21, 2005 http://www.anc.ed.ac.uk/ amos/lfd/ Neural Networks Hidden to output layer - a linear parameter model But adapt the features of the model.
More informationGaussian Process Latent Variable Models for Visualisation of High Dimensional Data
Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data Neil D. Lawrence Department of Computer Science University of Sheffield Regent Court, 211 Portobello Street, Sheffield,
More informationContents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation
Contents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation Learning 4 Supervised Learning 4 Unsupervised Learning 4
More informationAssignment 2. Classification and Regression using Linear Networks, Multilayer Perceptron Networks, and Radial Basis Functions
ENEE 739Q: STATISTICAL AND NEURAL PATTERN RECOGNITION Spring 2002 Assignment 2 Classification and Regression using Linear Networks, Multilayer Perceptron Networks, and Radial Basis Functions Aravind Sundaresan
More informationThe exam is closed book, closed notes except your one-page (two-sided) cheat sheet.
CS 189 Spring 2015 Introduction to Machine Learning Final You have 2 hours 50 minutes for the exam. The exam is closed book, closed notes except your one-page (two-sided) cheat sheet. No calculators or
More informationRobust Regression. Robust Data Mining Techniques By Boonyakorn Jantaranuson
Robust Regression Robust Data Mining Techniques By Boonyakorn Jantaranuson Outline Introduction OLS and important terminology Least Median of Squares (LMedS) M-estimator Penalized least squares What is
More informationECE 285 Class Project Report
ECE 285 Class Project Report Based on Source localization in an ocean waveguide using supervised machine learning Yiwen Gong ( yig122@eng.ucsd.edu), Yu Chai( yuc385@eng.ucsd.edu ), Yifeng Bu( ybu@eng.ucsd.edu
More informationLouis Fourrier Fabien Gaie Thomas Rolf
CS 229 Stay Alert! The Ford Challenge Louis Fourrier Fabien Gaie Thomas Rolf Louis Fourrier Fabien Gaie Thomas Rolf 1. Problem description a. Goal Our final project is a recent Kaggle competition submitted
More informationKnowledge Discovery and Data Mining. Neural Nets. A simple NN as a Mathematical Formula. Notes. Lecture 13 - Neural Nets. Tom Kelsey.
Knowledge Discovery and Data Mining Lecture 13 - Neural Nets Tom Kelsey School of Computer Science University of St Andrews http://tom.home.cs.st-andrews.ac.uk twk@st-andrews.ac.uk Tom Kelsey ID5059-13-NN
More informationWeek 3: Perceptron and Multi-layer Perceptron
Week 3: Perceptron and Multi-layer Perceptron Phong Le, Willem Zuidema November 12, 2013 Last week we studied two famous biological neuron models, Fitzhugh-Nagumo model and Izhikevich model. This week,
More informationKnowledge Discovery and Data Mining
Knowledge Discovery and Data Mining Lecture 13 - Neural Nets Tom Kelsey School of Computer Science University of St Andrews http://tom.home.cs.st-andrews.ac.uk twk@st-andrews.ac.uk Tom Kelsey ID5059-13-NN
More informationIntroduction to Matlab
Introduction to Matlab Christopher K. I. Williams Division of Informatics, University of Edinburgh October 1999 Background This document has the objective of introducing you to some of the facilities available
More informationMore on Neural Networks. Read Chapter 5 in the text by Bishop, except omit Sections 5.3.3, 5.3.4, 5.4, 5.5.4, 5.5.5, 5.5.6, 5.5.7, and 5.
More on Neural Networks Read Chapter 5 in the text by Bishop, except omit Sections 5.3.3, 5.3.4, 5.4, 5.5.4, 5.5.5, 5.5.6, 5.5.7, and 5.6 Recall the MLP Training Example From Last Lecture log likelihood
More information>> x2=-5.01:0.25:5.01; >> [x, y]=meshgrid(x1,x2); >> f=mexhat(x,y); >> mesh(x1,x2,f); The training data and test data sets with 100 instances each are
2D1431 Machine Learning Lab 3: Instance Based Learning & Neural Networks Frank Hoffmann e-mail: hoffmann@nada.kth.se November 27, 2002 1 Introduction In this lab you will learn about instance based learning
More informationOnline Algorithm Comparison points
CS446: Machine Learning Spring 2017 Problem Set 3 Handed Out: February 15 th, 2017 Due: February 27 th, 2017 Feel free to talk to other members of the class in doing the homework. I am more concerned that
More informationNeural Networks (pp )
Notation: Means pencil-and-paper QUIZ Means coding QUIZ Neural Networks (pp. 106-121) The first artificial neural network (ANN) was the (single-layer) perceptron, a simplified model of a biological neuron.
More information2. Linear Regression and Gradient Descent
Pattern Recognition And Machine Learning - EPFL - Fall 2015 Emtiyaz Khan, Timur Bagautdinov, Carlos Becker, Ilija Bogunovic & Ksenia Konyushkova 2. Linear Regression and Gradient Descent 2.1 Goals The
More informationNeural Networks. CE-725: Statistical Pattern Recognition Sharif University of Technology Spring Soleymani
Neural Networks CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Biological and artificial neural networks Feed-forward neural networks Single layer
More informationPerceptron as a graph
Neural Networks Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University October 10 th, 2007 2005-2007 Carlos Guestrin 1 Perceptron as a graph 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0-6 -4-2
More informationLearning from Data Linear Parameter Models
Learning from Data Linear Parameter Models Copyright David Barber 200-2004. Course lecturer: Amos Storkey a.storkey@ed.ac.uk Course page : http://www.anc.ed.ac.uk/ amos/lfd/ 2 chirps per sec 26 24 22 20
More informationBagging for One-Class Learning
Bagging for One-Class Learning David Kamm December 13, 2008 1 Introduction Consider the following outlier detection problem: suppose you are given an unlabeled data set and make the assumptions that one
More informationAnalysis of Functional MRI Timeseries Data Using Signal Processing Techniques
Analysis of Functional MRI Timeseries Data Using Signal Processing Techniques Sea Chen Department of Biomedical Engineering Advisors: Dr. Charles A. Bouman and Dr. Mark J. Lowe S. Chen Final Exam October
More informationApplying Supervised Learning
Applying Supervised Learning When to Consider Supervised Learning A supervised learning algorithm takes a known set of input data (the training set) and known responses to the data (output), and trains
More informationMachine Learning. Topic 5: Linear Discriminants. Bryan Pardo, EECS 349 Machine Learning, 2013
Machine Learning Topic 5: Linear Discriminants Bryan Pardo, EECS 349 Machine Learning, 2013 Thanks to Mark Cartwright for his extensive contributions to these slides Thanks to Alpaydin, Bishop, and Duda/Hart/Stork
More informationPredictive Analytics: Demystifying Current and Emerging Methodologies. Tom Kolde, FCAS, MAAA Linda Brobeck, FCAS, MAAA
Predictive Analytics: Demystifying Current and Emerging Methodologies Tom Kolde, FCAS, MAAA Linda Brobeck, FCAS, MAAA May 18, 2017 About the Presenters Tom Kolde, FCAS, MAAA Consulting Actuary Chicago,
More informationCS 229 Midterm Review
CS 229 Midterm Review Course Staff Fall 2018 11/2/2018 Outline Today: SVMs Kernels Tree Ensembles EM Algorithm / Mixture Models [ Focus on building intuition, less so on solving specific problems. Ask
More informationLarge-Scale Lasso and Elastic-Net Regularized Generalized Linear Models
Large-Scale Lasso and Elastic-Net Regularized Generalized Linear Models DB Tsai Steven Hillion Outline Introduction Linear / Nonlinear Classification Feature Engineering - Polynomial Expansion Big-data
More informationSELECTION OF A MULTIVARIATE CALIBRATION METHOD
SELECTION OF A MULTIVARIATE CALIBRATION METHOD 0. Aim of this document Different types of multivariate calibration methods are available. The aim of this document is to help the user select the proper
More informationDeep Learning With Noise
Deep Learning With Noise Yixin Luo Computer Science Department Carnegie Mellon University yixinluo@cs.cmu.edu Fan Yang Department of Mathematical Sciences Carnegie Mellon University fanyang1@andrew.cmu.edu
More informationCSC 2515 Introduction to Machine Learning Assignment 2
CSC 2515 Introduction to Machine Learning Assignment 2 Zhongtian Qiu(1002274530) Problem 1 See attached scan files for question 1. 2. Neural Network 2.1 Examine the statistics and plots of training error
More informationNeural Networks and Deep Learning
Neural Networks and Deep Learning Example Learning Problem Example Learning Problem Celebrity Faces in the Wild Machine Learning Pipeline Raw data Feature extract. Feature computation Inference: prediction,
More informationAn MM Algorithm for Multicategory Vertex Discriminant Analysis
An MM Algorithm for Multicategory Vertex Discriminant Analysis Tong Tong Wu Department of Epidemiology and Biostatistics University of Maryland, College Park May 22, 2008 Joint work with Professor Kenneth
More information10/14/2017. Dejan Sarka. Anomaly Detection. Sponsors
Dejan Sarka Anomaly Detection Sponsors About me SQL Server MVP (17 years) and MCT (20 years) 25 years working with SQL Server Authoring 16 th book Authoring many courses, articles Agenda Introduction Simple
More information4. Feedforward neural networks. 4.1 Feedforward neural network structure
4. Feedforward neural networks 4.1 Feedforward neural network structure Feedforward neural network is one of the most common network architectures. Its structure and some basic preprocessing issues required
More informationClassification: Linear Discriminant Functions
Classification: Linear Discriminant Functions CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Discriminant functions Linear Discriminant functions
More informationCIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]
CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, 2015. 11:59pm, PDF to Canvas [100 points] Instructions. Please write up your responses to the following problems clearly and concisely.
More informationCSE 5526: Introduction to Neural Networks Radial Basis Function (RBF) Networks
CSE 5526: Introduction to Neural Networks Radial Basis Function (RBF) Networks Part IV 1 Function approximation MLP is both a pattern classifier and a function approximator As a function approximator,
More informationCS229 Final Project: Predicting Expected Response Times
CS229 Final Project: Predicting Expected Email Response Times Laura Cruz-Albrecht (lcruzalb), Kevin Khieu (kkhieu) December 15, 2017 1 Introduction Each day, countless emails are sent out, yet the time
More informationClustering. Mihaela van der Schaar. January 27, Department of Engineering Science University of Oxford
Department of Engineering Science University of Oxford January 27, 2017 Many datasets consist of multiple heterogeneous subsets. Cluster analysis: Given an unlabelled data, want algorithms that automatically
More informationNeural Network Learning. Today s Lecture. Continuation of Neural Networks. Artificial Neural Networks. Lecture 24: Learning 3. Victor R.
Lecture 24: Learning 3 Victor R. Lesser CMPSCI 683 Fall 2010 Today s Lecture Continuation of Neural Networks Artificial Neural Networks Compose of nodes/units connected by links Each link has a numeric
More informationECE 662 Hw2 4/1/2008
ECE 662 Hw2 4/1/28 1. Fisher s Linear Discriminant Analysis n 1 We would like to find a projection from R to R, such that we can maximize the separation between 2 classes. We define the cost function,
More informationPattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
More informationDynamic Analysis of Structures Using Neural Networks
Dynamic Analysis of Structures Using Neural Networks Alireza Lavaei Academic member, Islamic Azad University, Boroujerd Branch, Iran Alireza Lohrasbi Academic member, Islamic Azad University, Boroujerd
More informationLearning from Data Mixture Models
Learning from Data Mixture Models Copyright David Barber 2001-2004. Course lecturer: Amos Storkey a.storkey@ed.ac.uk Course page : http://www.anc.ed.ac.uk/ amos/lfd/ 1 It is not uncommon for data to come
More informationCANCER PREDICTION USING PATTERN CLASSIFICATION OF MICROARRAY DATA. By: Sudhir Madhav Rao &Vinod Jayakumar Instructor: Dr.
CANCER PREDICTION USING PATTERN CLASSIFICATION OF MICROARRAY DATA By: Sudhir Madhav Rao &Vinod Jayakumar Instructor: Dr. Michael Nechyba 1. Abstract The objective of this project is to apply well known
More informationICA as a preprocessing technique for classification
ICA as a preprocessing technique for classification V.Sanchez-Poblador 1, E. Monte-Moreno 1, J. Solé-Casals 2 1 TALP Research Center Universitat Politècnica de Catalunya (Catalonia, Spain) enric@gps.tsc.upc.es
More information3 Nonlinear Regression
CSC 4 / CSC D / CSC C 3 Sometimes linear models are not sufficient to capture the real-world phenomena, and thus nonlinear models are necessary. In regression, all such models will have the same basic
More informationSUPERVISED LEARNING METHODS. Stanley Liang, PhD Candidate, Lassonde School of Engineering, York University Helix Science Engagement Programs 2018
SUPERVISED LEARNING METHODS Stanley Liang, PhD Candidate, Lassonde School of Engineering, York University Helix Science Engagement Programs 2018 2 CHOICE OF ML You cannot know which algorithm will work
More informationThe exam is closed book, closed notes except your one-page cheat sheet.
CS 189 Fall 2015 Introduction to Machine Learning Final Please do not turn over the page before you are instructed to do so. You have 2 hours and 50 minutes. Please write your initials on the top-right
More informationDS Machine Learning and Data Mining I. Alina Oprea Associate Professor, CCIS Northeastern University
DS 4400 Machine Learning and Data Mining I Alina Oprea Associate Professor, CCIS Northeastern University January 24 2019 Logistics HW 1 is due on Friday 01/25 Project proposal: due Feb 21 1 page description
More informationThe Automation of the Feature Selection Process. Ronen Meiri & Jacob Zahavi
The Automation of the Feature Selection Process Ronen Meiri & Jacob Zahavi Automated Data Science http://www.kdnuggets.com/2016/03/automated-data-science.html Outline The feature selection problem Objective
More informationMachine Learning. Chao Lan
Machine Learning Chao Lan Machine Learning Prediction Models Regression Model - linear regression (least square, ridge regression, Lasso) Classification Model - naive Bayes, logistic regression, Gaussian
More informationStatistical Analysis of Metabolomics Data. Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte
Statistical Analysis of Metabolomics Data Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte Outline Introduction Data pre-treatment 1. Normalization 2. Centering,
More informationProbabilistic Modelling and Reasoning Assignment 2
Probabilistic Modelling and Reasoning Assignment 2 Instructor: Prof. Chris Williams Published: Fri November 4 2011, Revised Nov 18 Due: Fri December 2, 2011 by 4pm Remember that plagiarism is a university
More information3 Nonlinear Regression
3 Linear models are often insufficient to capture the real-world phenomena. That is, the relation between the inputs and the outputs we want to be able to predict are not linear. As a consequence, nonlinear
More informationCPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2017
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2017 Assignment 3: 2 late days to hand in tonight. Admin Assignment 4: Due Friday of next week. Last Time: MAP Estimation MAP
More informationAdvanced Machine Learning Practical 1: Manifold Learning (PCA and Kernel PCA)
Advanced Machine Learning Practical : Manifold Learning (PCA and Kernel PCA) Professor: Aude Billard Assistants: Nadia Figueroa, Ilaria Lauzana and Brice Platerrier E-mails: aude.billard@epfl.ch, nadia.figueroafernandez@epfl.ch
More informationChapter 7 UNSUPERVISED LEARNING TECHNIQUES FOR MAMMOGRAM CLASSIFICATION
UNSUPERVISED LEARNING TECHNIQUES FOR MAMMOGRAM CLASSIFICATION Supervised and unsupervised learning are the two prominent machine learning algorithms used in pattern recognition and classification. In this
More informationMLPQNA-LEMON Multi Layer Perceptron neural network trained by Quasi Newton or Levenberg-Marquardt optimization algorithms
MLPQNA-LEMON Multi Layer Perceptron neural network trained by Quasi Newton or Levenberg-Marquardt optimization algorithms 1 Introduction In supervised Machine Learning (ML) we have a set of data points
More informationNotes on Multilayer, Feedforward Neural Networks
Notes on Multilayer, Feedforward Neural Networks CS425/528: Machine Learning Fall 2012 Prepared by: Lynne E. Parker [Material in these notes was gleaned from various sources, including E. Alpaydin s book
More informationAkarsh Pokkunuru EECS Department Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
Akarsh Pokkunuru EECS Department 03-16-2017 Contractive Auto-Encoders: Explicit Invariance During Feature Extraction 1 AGENDA Introduction to Auto-encoders Types of Auto-encoders Analysis of different
More informationCLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS
CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS CHAPTER 4 CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS 4.1 Introduction Optical character recognition is one of
More informationscikit-learn (Machine Learning in Python)
scikit-learn (Machine Learning in Python) (PB13007115) 2016-07-12 (PB13007115) scikit-learn (Machine Learning in Python) 2016-07-12 1 / 29 Outline 1 Introduction 2 scikit-learn examples 3 Captcha recognize
More informationPreface to the Second Edition. Preface to the First Edition. 1 Introduction 1
Preface to the Second Edition Preface to the First Edition vii xi 1 Introduction 1 2 Overview of Supervised Learning 9 2.1 Introduction... 9 2.2 Variable Types and Terminology... 9 2.3 Two Simple Approaches
More informationDeep Learning. Volker Tresp Summer 2014
Deep Learning Volker Tresp Summer 2014 1 Neural Network Winter and Revival While Machine Learning was flourishing, there was a Neural Network winter (late 1990 s until late 2000 s) Around 2010 there
More informationAn Algorithm For Training Multilayer Perceptron (MLP) For Image Reconstruction Using Neural Network Without Overfitting.
An Algorithm For Training Multilayer Perceptron (MLP) For Image Reconstruction Using Neural Network Without Overfitting. Mohammad Mahmudul Alam Mia, Shovasis Kumar Biswas, Monalisa Chowdhury Urmi, Abubakar
More information1. Approximation and Prediction Problems
Neural and Evolutionary Computing Lab 2: Neural Networks for Approximation and Prediction 1. Approximation and Prediction Problems Aim: extract from data a model which describes either the depence between
More informationIndex. Umberto Michelucci 2018 U. Michelucci, Applied Deep Learning,
A Acquisition function, 298, 301 Adam optimizer, 175 178 Anaconda navigator conda command, 3 Create button, 5 download and install, 1 installing packages, 8 Jupyter Notebook, 11 13 left navigation pane,
More informationClustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
More informationMachine Learning: Think Big and Parallel
Day 1 Inderjit S. Dhillon Dept of Computer Science UT Austin CS395T: Topics in Multicore Programming Oct 1, 2013 Outline Scikit-learn: Machine Learning in Python Supervised Learning day1 Regression: Least
More informationNeural Networks (Overview) Prof. Richard Zanibbi
Neural Networks (Overview) Prof. Richard Zanibbi Inspired by Biology Introduction But as used in pattern recognition research, have little relation with real neural systems (studied in neurology and neuroscience)
More informationLecture #11: The Perceptron
Lecture #11: The Perceptron Mat Kallada STAT2450 - Introduction to Data Mining Outline for Today Welcome back! Assignment 3 The Perceptron Learning Method Perceptron Learning Rule Assignment 3 Will be
More informationNeural Network Optimization and Tuning / Spring 2018 / Recitation 3
Neural Network Optimization and Tuning 11-785 / Spring 2018 / Recitation 3 1 Logistics You will work through a Jupyter notebook that contains sample and starter code with explanations and comments throughout.
More informationLearning from Data Introduction to Matlab
Learning from Data Introduction to Matlab Amos Storkey, David Barber and Chris Williams a.storkey@ed.ac.uk Course page : http://www.anc.ed.ac.uk/ amos/lfd/ This is a modified version of a text written
More informationClassifying Depositional Environments in Satellite Images
Classifying Depositional Environments in Satellite Images Alex Miltenberger and Rayan Kanfar Department of Geophysics School of Earth, Energy, and Environmental Sciences Stanford University 1 Introduction
More informationMachine Learning Lecture 3
Machine Learning Lecture 3 Probability Density Estimation II 19.10.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Announcements Exam dates We re in the process
More informationIntroduction to SNNS
Introduction to SNNS Caren Marzban http://www.nhn.ou.edu/ marzban Introduction In this lecture we will learn about a Neural Net (NN) program that I know a little about - the Stuttgart Neural Network Simulator
More informationData Mining. Neural Networks
Data Mining Neural Networks Goals for this Unit Basic understanding of Neural Networks and how they work Ability to use Neural Networks to solve real problems Understand when neural networks may be most
More informationComparison of Optimization Methods for L1-regularized Logistic Regression
Comparison of Optimization Methods for L1-regularized Logistic Regression Aleksandar Jovanovich Department of Computer Science and Information Systems Youngstown State University Youngstown, OH 44555 aleksjovanovich@gmail.com
More informationMachine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves
Machine Learning A 708.064 11W 1sst KU Exercises Problems marked with * are optional. 1 Conditional Independence I [2 P] a) [1 P] Give an example for a probability distribution P (A, B, C) that disproves
More informationLocally Weighted Learning for Control. Alexander Skoglund Machine Learning Course AASS, June 2005
Locally Weighted Learning for Control Alexander Skoglund Machine Learning Course AASS, June 2005 Outline Locally Weighted Learning, Christopher G. Atkeson et. al. in Artificial Intelligence Review, 11:11-73,1997
More informationGeneral Instructions. Questions
CS246: Mining Massive Data Sets Winter 2018 Problem Set 2 Due 11:59pm February 8, 2018 Only one late period is allowed for this homework (11:59pm 2/13). General Instructions Submission instructions: These
More informationLatent Variable Models and Expectation Maximization
Latent Variable Models and Expectation Maximization Oliver Schulte - CMPT 726 Bishop PRML Ch. 9 2 4 6 8 1 12 14 16 18 2 4 6 8 1 12 14 16 18 5 1 15 2 25 5 1 15 2 25 2 4 6 8 1 12 14 2 4 6 8 1 12 14 5 1 15
More informationArtificial Neural Networks (Feedforward Nets)
Artificial Neural Networks (Feedforward Nets) y w 03-1 w 13 y 1 w 23 y 2 w 01 w 21 w 22 w 02-1 w 11 w 12-1 x 1 x 2 6.034 - Spring 1 Single Perceptron Unit y w 0 w 1 w n w 2 w 3 x 0 =1 x 1 x 2 x 3... x
More informationEfficient Tuning of SVM Hyperparameters Using Radius/Margin Bound and Iterative Algorithms
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 13, NO. 5, SEPTEMBER 2002 1225 Efficient Tuning of SVM Hyperparameters Using Radius/Margin Bound and Iterative Algorithms S. Sathiya Keerthi Abstract This paper
More informationRandom Forest A. Fornaser
Random Forest A. Fornaser alberto.fornaser@unitn.it Sources Lecture 15: decision trees, information theory and random forests, Dr. Richard E. Turner Trees and Random Forests, Adele Cutler, Utah State University
More informationCOMPUTATIONAL INTELLIGENCE
COMPUTATIONAL INTELLIGENCE Radial Basis Function Networks Adrian Horzyk Preface Radial Basis Function Networks (RBFN) are a kind of artificial neural networks that use radial basis functions (RBF) as activation
More informationUnsupervised learning in Vision
Chapter 7 Unsupervised learning in Vision The fields of Computer Vision and Machine Learning complement each other in a very natural way: the aim of the former is to extract useful information from visual
More informationGrundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Unsupervised learning Daniel Hennes 29.01.2018 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Supervised learning Regression (linear
More informationFurther Maths Notes. Common Mistakes. Read the bold words in the exam! Always check data entry. Write equations in terms of variables
Further Maths Notes Common Mistakes Read the bold words in the exam! Always check data entry Remember to interpret data with the multipliers specified (e.g. in thousands) Write equations in terms of variables
More informationAnnotated multitree output
Annotated multitree output A simplified version of the two high-threshold (2HT) model, applied to two experimental conditions, is used as an example to illustrate the output provided by multitree (version
More informationMachine Learning : Clustering, Self-Organizing Maps
Machine Learning Clustering, Self-Organizing Maps 12/12/2013 Machine Learning : Clustering, Self-Organizing Maps Clustering The task: partition a set of objects into meaningful subsets (clusters). The
More information