Using Procrustes Analysis and Principal Component Analysis to Detect Schizophrenic Brains. Anthony Dotterer

Size: px
Start display at page:

Download "Using Procrustes Analysis and Principal Component Analysis to Detect Schizophrenic Brains. Anthony Dotterer"

Transcription

1 Using Procrustes Analysis and Principal Component Analysis to Detect Schizophrenic Brains Anthony Dotterer

2 Introduction Bookstein has studied the difference between normal and schizophrenic brains by doing shape analysis on landmark points described in [1]. This paper attempts to use Procrustes Analysis on his dataset of 14 normal and 14 schizophrenic brains to create their mean shapes. With these means, Principle Component Analysis is applied to find the distribution of possible shapes for normal brains and schizophrenic brains. Setup As described in the introduction this paper uses 14 normal and 14 schizophrenic brain shapes from Bookstein's experiments. Each of the shapes contain 13 different points that correspond between each other shape. The following figures show the 14 normal brain shapes and then the 14 schizophrenic brain shapes. Illustration 1: Normal Brain Shapes

3 Illustration 2: Schizophrenic Brain Shapes General Procrustes Analysis The first step to applying Principle Component Analysis (PCA) to a step of shapes is finding the shapes' mean shape. The job of find a mean shape form a set of shapes is best done with General Procrustes Analysis. Lets define a set of N shapes, S, such that each shape contains a set of n points. In our case N will equal 14 and n will equal 13. S {S 1, S 2,... S N } S i {{x 1, y 1 },{x 2, y 2 },...{ x n, y n }} S i, j {x j, y j } Now we normalize all of the shapes such that their center of mass is at the origin and their square root of the sum of squares is equal to 1.

4 S i ' =S i j S i, j n S i = j S i ' x 2 2 i, j y i, j Next we assume the first shape is the mean shape and we begin to iterate until the previous mean shape is within some sum of square distances of the newest mean shape. As we iterate, each shape is rotationally normalized to the mean shape. V DV T =SVD S it S S it S T W DW T =SVD S it S T S it S R=V W T S i =S i R In the next step of the iteration, the new mean shape is calculated by normalizing the scale and translation of the shape created by taking the mean of each point location from the rotation normalized shapes. S new = i N S i, j S new =norm S new Once the proper convergence has been met, the loop will stop iterating. The following figures are example of the mean shapes of the normal brain and the schizophrenic brain with the convergence of 0.1.

5 Illustration 3: Mean Normal Brain Shape Illustration 4: Mean Schizophrenic Brain Shape

6 The described General Procrustes Analysis is implemented in Matlab code using the normalizing function in Appendix A and Procrustes function in Appendix B. Principle Component Analysis PCA will create two sets, Φ and b. Φ will contain the largest eigenvectors of the covariance matrix of the set of shapes compared with their mean shape. b will contain the parameters that when used with Φ and the mean shape will recreated a particular shape. In order to begin PCA, all of the shapes in a set must be normalized to the mean shape. The following figures shows the normalized shapes. Notice the strange alignment in the schizophrenic figures. Illustration 5: Normalized Normal Brain Shapes

7 Illustration 6: Normalized Schizophrenic Brain Shapes The first step is create to the covariance matrix of the set of shapes versus their mean shape. S i S S i S T i = N Next the largest eigenvectors are extracted from the covariance matrix into Φ. With Φ and the mean shape, the parameter set can be created. b i = T S i S The following figure shows the set b in two dimensions where green represents normal brains and blue represents schizophrenic brains and their means denoted by the red. Notice how the strangle alignment in the schizophrenic brain shapes cause a small group of outliers in its parameter set.

8 Illustration 7: Parameter Sets of Normal (green) and Schizophrenic (blue) Brain Shapes and their means (red) This PCA method was implemented in Matlab code as a function is Appendix C. Matching Brain Shapes Once Φ and b of the sets of brains have been calculated, each shape can try to be matched to either the normal or schizophrenic brain sets. The first step in matching a brain shape is finding that shape's normalized shape with respect each set's mean shape. With the new normalized shapes, their parameter sets, b N and b S, can be calculated using each shape set's Φ. Finally the shape's parameter set who's Euclidean distance to each shape sets' mean parameter set is closest becomes matched to that brain type. This method requires Φ to be calculated for only two eigenvectors. The final matching results are not at all satisfactory. The following table describes how each shape and the type it matched. Normal Shapes Normal Schizophrenic Schizophrenic Shapes Normal Schizophrenic 1 TRUE FALSE 1 TRUE FALSE 2 TRUE FALSE 2 FALSE TRUE 3 TRUE FALSE 3 TRUE FALSE 4 TRUE FALSE 4 TRUE FALSE 5 TRUE FALSE 5 TRUE FALSE

9 Normal Shapes Normal Schizophrenic Schizophrenic Shapes Normal Schizophrenic 6 TRUE FALSE 6 TRUE FALSE 7 FALSE TRUE 7 TRUE FALSE 8 TRUE FALSE 8 TRUE FALSE 9 FALSE TRUE 9 TRUE FALSE 10 FALSE TRUE 10 TRUE FALSE 11 TRUE FALSE 11 TRUE FALSE 12 TRUE FALSE 12 TRUE FALSE 13 FALSE TRUE 13 FALSE TRUE 14 TRUE FALSE 14 TRUE FALSE The matching algorithm was implemented in Matlab code using a driver in Appendix E and a rotational normalizing function in Appendix D. Discussion This paper uses General Procrustes Analysis and Principle Component Analysis to find a parameter set that is used to classify new shapes. Unfortunately, this approach employed only two of the largest eigenvectors in Φ and the Euclidean distance to label shapes. A better approach could possibly use more eigenvectors in Φ with a multi-dimensional Gaussian probability distribution using the mean of the parameter set and the covariance matrix of all the parameters. In fact, my original approach did use such a method. Unfortunately, my implementation would complain about eigenvectors that were too small and the probability distribution's output would not be between 0 and 1 but in the order of in size. Therefore, my approached was simplified to the described method. References [1] The Shape of Madness. A mathematician figures out how to predict who will become schizophrenic. D MACKENZIE - DISCOVER-NEW YORK-, DISCOVER-PALM COAST DATA Appendix A: normalize.m Normalize the passed shape in the following was ways: Translation normalization: center of mass at origin Scale normalization: divide by square root of sum of squares of center of mass input:

10 - shapes the shapes before normalization output: - normshapes the translation and scale normalized shapes authored by: Anthony Dotterer function normshapes = normalize(shapes) get the dimensions of the shapes [ncoords, npoints, nshapes] = size(shapes); normalize the translation means = mean(shapes, 2); for i = 1:nshapes normshapes(1, :, i) = shapes(1, :, i) - means(1, i); normshapes(2, :, i) = shapes(2, :, i) - means(2, i); normalize the scale SqrtSSQ = sqrt(sum(normshapes(1, :, :).^2 + normshapes(2, :, :).^2, 2)); for i = 1:nshapes normshapes(:, :, i) = normshapes(:, :, i) / SqrtSSQ(1, 1, i); return; Appendix B: procrustes.m Preforms procrustes analyzes on the passed shapes input - shapes Shapes to align in a mean shape - convergence The difference between iterations to be done output - meanshape The mean shape for the passed shapes - normalshapes The shape set normalized to the mean shape authored by Anthony Dotterer function [meanshape, normalshapes] = procrustes(shapes, convergence) normalize the shapes by tranlation and scale normalshapes = normalize(shapes); iterate through shapes to find mean shape i = 0; meanshapes(:, :, 1) = normalshapes(:, :, 1); SSD = [ convergence+1 ]; while (i == 0 SSD(i) > convergence) i = i + 1; rotation normalize shapes to mean shape

11 for j = 1:size(normalShapes, 3) calculate the rotation tmp = normalshapes(:, :, j)' * meanshapes(:, :, i); [V, D, Vt] = svd(tmp * tmp'); [W, d, Wt] = svd(tmp' * tmp); rotation = V * W'; apply the rotation to the shape normalshapes(:, :, j) = normalshapes(:, :, j) * rotation; find the new mean shape tmpmeanshape = mean(normalshapes, 3); normalize scale and translation meanshapes(:, :, i+1) = normalize(tmpmeanshape); get this iterations sum of if (i ~= 1) SSD(i) = sum(sum((meanshapes(:, :, i-1) - meanshapes(:, :, i)).^2)); meanshape = meanshapes(:, :, i+1); return; Appendix C: PCA.m Finds the PCA distribution matrix and eigenvectors vector input - shapes The shapes to do PCA - meanshapes The mean shape of the passed shapes - eigens The number of eigenvectors to add to phi output - b The PCA distribution matrix - phi The eigenvectors vector of the covariance shape matrix authored by: Anthony Dotterer function [b, phi] = PCA(shapes, meanshape, eigens) get the covariance matrix tmp = zeros(size(shapes, 2)*2); for i = 1:size(shapes, 3) tmp2 = [shapes(1, :, i), shapes(2, :, i)]' - [meanshape(1, :), meanshape(2, :)]'; tmp(:, :) = tmp(:, :) + (tmp2 * tmp2'); covar = tmp / size(shapes, 3);

12 get the eigen vectors and values of the covariance matrix [phi, eigvalues] = eigs(covar, eye(size(covar)), eigens); calculate the PCA distrubitions for each shape b = []; for i = 1:size(shapes, 3) b(:, i) = phi' * ([shapes(1, :, i), shapes(2, :, i)]' - [meanshape(1, :), meanshape(1, :)]'); return; Appendix D: normalizewithmean.m Normalize the passed shape in the following was ways: Translation normalization: center of mass at origin Scale normalization: divide by square root of sum of squares of center of mass Rotation normalization: rotates with respect to the passed mean shape input: - shapes the shapes before normalization - meanshape the normalized mean shape output: - normshapes the translation, scale, and rotation normalized shapes authored by: Anthony Dotterer function normshapes = normalizewithmean(shapes, meanshape) get the dimensions of the shapes [ncoords, npoints, nshapes] = size(shapes); normalize the translation means = mean(shapes, 2); for i = 1:nshapes normshapes(1, :, i) = shapes(1, :, i) - means(1, i); normshapes(2, :, i) = shapes(2, :, i) - means(2, i); normalize the scale SqrtSSQ = sqrt(sum(normshapes(1, :, :).^2 + normshapes(2, :, :).^2, 2)); for i = 1:nshapes normshapes(:, :, i) = normshapes(:, :, i) / SqrtSSQ(1, 1, i); normalize rotation for i = 1:size(shapes, 3) calculate the rotation tmp = shapes(:, :, i)' * meanshape; [V, D, Vt] = svd(tmp * tmp'); [W, d, Wt] = svd(tmp' * tmp);

13 rotation = V * W'; apply the rotation to the shape shapes(:, :, i) = shapes(:, :, i) * rotation; return; Appendix E: driver.m test shell file for reading Bookstein small schizoprenia dataset for testing algorithms on Procrustes Analysis Bob Collins, Penn State, CSE586 Midterm, Feb 2006 load 'schizo.txt' note, landmark point data will be in a 3D matrix first index: 2 coordinates, x and y second index: 13 points per person third index: 28 people. first 14 are "normal", second 14 are schizophrenic see comments in "schizo.txt" for more info on the landmarks schizo = reshape(schizo',[ ]); plot the normal shapes and the schizo shapes in preshape showshapes(schizo(:, :, 1:14), 1, [-1, 1, -1, 1]); showshapes(schizo(:, :, 15:28), 2, [-1, 1, -1, 1]); separate the normal from the schizo normalshapes = schizo(:, :, 1:14); schizoshapes = schizo(:, :, 15:28); do procrustes on normal and schizo brains [normalmean, normalizenshapes] = procrustes(normalshapes, 0.1); [schizomean, normalizesshapes] = procrustes(schizoshapes, 0.1); show the normalized shapes showshapes(normalizenshapes, 3, [-1, 1, -1, 1]); showshapes(normalizesshapes, 4, [-1, 1, -1, 1]); show the means showshapes(normalmean, 5, [-1, 1, -1, 1]); showshapes(schizomean, 6, [-1, 1, -1, 1]); calculate the PCA of the shapes [bnormal, phinormal] = PCA(normalizeNShapes, normalmean, 2); [bschizo, phischizo] = PCA(normalizeSShapes, schizomean, 2); find the mean of the normal and the schizo bnormalmean = mean(bnormal, 2); bschizomean = mean(bschizo, 2); show the parameter sets figure(7); hold on; axis([ 0, 3, -.5, 1.5 ]); plot(bnormal(1, :), bnormal(2,:), 'g*'); plot(bschizo(1, :), bschizo(2, :), 'b+');

14 plot(bnormalmean(1), bnormalmean(2), 'r*'); plot(bschizomean(1), bschizomean(2), 'r+'); hold off; try to score each of the normal brains versus the schizo brains normalmatches = []; schizomatches = []; for i = 1:size(schizo, 3) normalize this shape normnshape = normalizewithmean(schizo(:, :, i), normalmean); normsshape = normalizewithmean(schizo(:, :, i), schizomean); calculate the normal and schizo b distrubitions normalbparams = phinormal' * ( [normnshape(1, :), normnshape(2, :)]' - [normalmean(1, :), normalmean(2, :)]' ); schizobparams = phischizo' * ( [normsshape(1, :), normsshape(2, :)]' - [schizomean(1, :), schizomean(2, :)]' ); find the distances of the parameter sets for each normal and schizo shape sets normaldistance = sqrt(sum((normalbparams - bnormalmean).^2)); schizodistance = sqrt(sum((schizobparams - bschizomean).^2)); fill the match array with this shape's matching parameters if (normaldistance < schizodistance) normalmatches(size(normalmatches, 2)+1) = i; else schizomatches(size(schizomatches, 2)+1) = i; normalmatches schizomatches

Clustering and Visualisation of Data

Clustering and Visualisation of Data Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some

More information

Robust Kernel Methods in Clustering and Dimensionality Reduction Problems

Robust Kernel Methods in Clustering and Dimensionality Reduction Problems Robust Kernel Methods in Clustering and Dimensionality Reduction Problems Jian Guo, Debadyuti Roy, Jing Wang University of Michigan, Department of Statistics Introduction In this report we propose robust

More information

CSE 547: Machine Learning for Big Data Spring Problem Set 2. Please read the homework submission policies.

CSE 547: Machine Learning for Big Data Spring Problem Set 2. Please read the homework submission policies. CSE 547: Machine Learning for Big Data Spring 2019 Problem Set 2 Please read the homework submission policies. 1 Principal Component Analysis and Reconstruction (25 points) Let s do PCA and reconstruct

More information

General Instructions. Questions

General Instructions. Questions CS246: Mining Massive Data Sets Winter 2018 Problem Set 2 Due 11:59pm February 8, 2018 Only one late period is allowed for this homework (11:59pm 2/13). General Instructions Submission instructions: These

More information

If you are not familiar with IMP you should download and read WhatIsImp and CoordGenManual before proceeding.

If you are not familiar with IMP you should download and read WhatIsImp and CoordGenManual before proceeding. IMP: PCAGen6- Principal Components Generation Utility PCAGen6 This software package is a Principal Components Analysis utility intended for use with landmark based morphometric data. The program loads

More information

Inf2B assignment 2. Natural images classification. Hiroshi Shimodaira and Pol Moreno. Submission due: 4pm, Wednesday 30 March 2016.

Inf2B assignment 2. Natural images classification. Hiroshi Shimodaira and Pol Moreno. Submission due: 4pm, Wednesday 30 March 2016. Inf2B assignment 2 (Ver. 1.2) Natural images classification Submission due: 4pm, Wednesday 30 March 2016 Hiroshi Shimodaira and Pol Moreno This assignment is out of 100 marks and forms 12.5% of your final

More information

Programming Exercise 7: K-means Clustering and Principal Component Analysis

Programming Exercise 7: K-means Clustering and Principal Component Analysis Programming Exercise 7: K-means Clustering and Principal Component Analysis Machine Learning May 13, 2012 Introduction In this exercise, you will implement the K-means clustering algorithm and apply it

More information

CSE 40171: Artificial Intelligence. Learning from Data: Unsupervised Learning

CSE 40171: Artificial Intelligence. Learning from Data: Unsupervised Learning CSE 40171: Artificial Intelligence Learning from Data: Unsupervised Learning 32 Homework #6 has been released. It is due at 11:59PM on 11/7. 33 CSE Seminar: 11/1 Amy Reibman Purdue University 3:30pm DBART

More information

Supervised vs. Unsupervised Learning

Supervised vs. Unsupervised Learning Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now

More information

Statistical Shape Analysis

Statistical Shape Analysis Statistical Shape Analysis I. L. Dryden and K. V. Mardia University ofleeds, UK JOHN WILEY& SONS Chichester New York Weinheim Brisbane Singapore Toronto Contents Preface Acknowledgements xv xix 1 Introduction

More information

Version 2.4 of Idiogrid

Version 2.4 of Idiogrid Version 2.4 of Idiogrid Structural and Visual Modifications 1. Tab delimited grids in Grid Data window. The most immediately obvious change to this newest version of Idiogrid will be the tab sheets that

More information

Network Traffic Measurements and Analysis

Network Traffic Measurements and Analysis DEIB - Politecnico di Milano Fall, 2017 Introduction Often, we have only a set of features x = x 1, x 2,, x n, but no associated response y. Therefore we are not interested in prediction nor classification,

More information

Clustering & Dimensionality Reduction. 273A Intro Machine Learning

Clustering & Dimensionality Reduction. 273A Intro Machine Learning Clustering & Dimensionality Reduction 273A Intro Machine Learning What is Unsupervised Learning? In supervised learning we were given attributes & targets (e.g. class labels). In unsupervised learning

More information

Dimension Reduction CS534

Dimension Reduction CS534 Dimension Reduction CS534 Why dimension reduction? High dimensionality large number of features E.g., documents represented by thousands of words, millions of bigrams Images represented by thousands of

More information

Using Augmented Measurements to Improve the Convergence of ICP. Jacopo Serafin and Giorgio Grisetti

Using Augmented Measurements to Improve the Convergence of ICP. Jacopo Serafin and Giorgio Grisetti Jacopo Serafin and Giorgio Grisetti Point Cloud Registration We want to find the rotation and the translation that maximize the overlap between two point clouds Page 2 Point Cloud Registration We want

More information

Recognizing Handwritten Digits Using the LLE Algorithm with Back Propagation

Recognizing Handwritten Digits Using the LLE Algorithm with Back Propagation Recognizing Handwritten Digits Using the LLE Algorithm with Back Propagation Lori Cillo, Attebury Honors Program Dr. Rajan Alex, Mentor West Texas A&M University Canyon, Texas 1 ABSTRACT. This work is

More information

Intensity Augmented ICP for Registration of Laser Scanner Point Clouds

Intensity Augmented ICP for Registration of Laser Scanner Point Clouds Intensity Augmented ICP for Registration of Laser Scanner Point Clouds Bharat Lohani* and Sandeep Sashidharan *Department of Civil Engineering, IIT Kanpur Email: blohani@iitk.ac.in. Abstract While using

More information

Unsupervised Learning

Unsupervised Learning Unsupervised Learning Learning without Class Labels (or correct outputs) Density Estimation Learn P(X) given training data for X Clustering Partition data into clusters Dimensionality Reduction Discover

More information

Clustering. Supervised vs. Unsupervised Learning

Clustering. Supervised vs. Unsupervised Learning Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now

More information

Image Coding with Active Appearance Models

Image Coding with Active Appearance Models Image Coding with Active Appearance Models Simon Baker, Iain Matthews, and Jeff Schneider CMU-RI-TR-03-13 The Robotics Institute Carnegie Mellon University Abstract Image coding is the task of representing

More information

Discriminate Analysis

Discriminate Analysis Discriminate Analysis Outline Introduction Linear Discriminant Analysis Examples 1 Introduction What is Discriminant Analysis? Statistical technique to classify objects into mutually exclusive and exhaustive

More information

Small Libraries of Protein Fragments through Clustering

Small Libraries of Protein Fragments through Clustering Small Libraries of Protein Fragments through Clustering Varun Ganapathi Department of Computer Science Stanford University June 8, 2005 Abstract When trying to extract information from the information

More information

Face Detection and Alignment. Prof. Xin Yang HUST

Face Detection and Alignment. Prof. Xin Yang HUST Face Detection and Alignment Prof. Xin Yang HUST Applications Virtual Makeup Applications Virtual Makeup Applications Video Editing Active Shape Models Suppose we have a statistical shape model Trained

More information

Clustering: Overview and K-means algorithm

Clustering: Overview and K-means algorithm Clustering: Overview and K-means algorithm Informal goal Given set of objects and measure of similarity between them, group similar objects together K-Means illustrations thanks to 2006 student Martin

More information

K-means Clustering & PCA

K-means Clustering & PCA K-means Clustering & PCA Andreas C. Kapourani (Credit: Hiroshi Shimodaira) 02 February 2018 1 Introduction In this lab session we will focus on K-means clustering and Principal Component Analysis (PCA).

More information

Statistical Analysis of Metabolomics Data. Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte

Statistical Analysis of Metabolomics Data. Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte Statistical Analysis of Metabolomics Data Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte Outline Introduction Data pre-treatment 1. Normalization 2. Centering,

More information

AN IMPROVED HYBRIDIZED K- MEANS CLUSTERING ALGORITHM (IHKMCA) FOR HIGHDIMENSIONAL DATASET & IT S PERFORMANCE ANALYSIS

AN IMPROVED HYBRIDIZED K- MEANS CLUSTERING ALGORITHM (IHKMCA) FOR HIGHDIMENSIONAL DATASET & IT S PERFORMANCE ANALYSIS AN IMPROVED HYBRIDIZED K- MEANS CLUSTERING ALGORITHM (IHKMCA) FOR HIGHDIMENSIONAL DATASET & IT S PERFORMANCE ANALYSIS H.S Behera Department of Computer Science and Engineering, Veer Surendra Sai University

More information

Recognition, SVD, and PCA

Recognition, SVD, and PCA Recognition, SVD, and PCA Recognition Suppose you want to find a face in an image One possibility: look for something that looks sort of like a face (oval, dark band near top, dark band near bottom) Another

More information

CSE 255 Lecture 5. Data Mining and Predictive Analytics. Dimensionality Reduction

CSE 255 Lecture 5. Data Mining and Predictive Analytics. Dimensionality Reduction CSE 255 Lecture 5 Data Mining and Predictive Analytics Dimensionality Reduction Course outline Week 4: I ll cover homework 1, and get started on Recommender Systems Week 5: I ll cover homework 2 (at the

More information

The Novel Approach for 3D Face Recognition Using Simple Preprocessing Method

The Novel Approach for 3D Face Recognition Using Simple Preprocessing Method The Novel Approach for 3D Face Recognition Using Simple Preprocessing Method Parvin Aminnejad 1, Ahmad Ayatollahi 2, Siamak Aminnejad 3, Reihaneh Asghari Abstract In this work, we presented a novel approach

More information

Work 2. Case-based reasoning exercise

Work 2. Case-based reasoning exercise Work 2. Case-based reasoning exercise Marc Albert Garcia Gonzalo, Miquel Perelló Nieto November 19, 2012 1 Introduction In this exercise we have implemented a case-based reasoning system, specifically

More information

Support Vector Machines

Support Vector Machines Support Vector Machines Michael Tagare De Guzman May 19, 2012 Support Vector Machines Linear Learning Machines and The Maximal Margin Classifier In Supervised Learning, a learning machine is given a training

More information

Algorithm research of 3D point cloud registration based on iterative closest point 1

Algorithm research of 3D point cloud registration based on iterative closest point 1 Acta Technica 62, No. 3B/2017, 189 196 c 2017 Institute of Thermomechanics CAS, v.v.i. Algorithm research of 3D point cloud registration based on iterative closest point 1 Qian Gao 2, Yujian Wang 2,3,

More information

10-701/15-781, Fall 2006, Final

10-701/15-781, Fall 2006, Final -7/-78, Fall 6, Final Dec, :pm-8:pm There are 9 questions in this exam ( pages including this cover sheet). If you need more room to work out your answer to a question, use the back of the page and clearly

More information

Unsupervised Learning : Clustering

Unsupervised Learning : Clustering Unsupervised Learning : Clustering Things to be Addressed Traditional Learning Models. Cluster Analysis K-means Clustering Algorithm Drawbacks of traditional clustering algorithms. Clustering as a complex

More information

Tensor Based Approaches for LVA Field Inference

Tensor Based Approaches for LVA Field Inference Tensor Based Approaches for LVA Field Inference Maksuda Lillah and Jeff Boisvert The importance of locally varying anisotropy (LVA) in model construction can be significant; however, it is often ignored

More information

CS251 Spring 2014 Lecture 7

CS251 Spring 2014 Lecture 7 CS251 Spring 2014 Lecture 7 Stephanie R Taylor Feb 19, 2014 1 Moving on to 3D Today, we move on to 3D coordinates. But first, let s recap of what we did in 2D: 1. We represented a data point in 2D data

More information

CSE 481C Imitation Learning in Humanoid Robots Motion capture, inverse kinematics, and dimensionality reduction

CSE 481C Imitation Learning in Humanoid Robots Motion capture, inverse kinematics, and dimensionality reduction 1 CSE 481C Imitation Learning in Humanoid Robots Motion capture, inverse kinematics, and dimensionality reduction Robotic Imitation of Human Actions 2 The inverse kinematics problem Joint angles Human-robot

More information

Does BRT return a predicted group membership or value for continuous variable? Yes, in the $fit object In this dataset, R 2 =0.346.

Does BRT return a predicted group membership or value for continuous variable? Yes, in the $fit object In this dataset, R 2 =0.346. Does BRT return a predicted group membership or value for continuous variable? Yes, in the $fit object In this dataset, R 2 =0.346 Is BRT the same as random forest? Very similar, randomforest package in

More information

Classifying Images with Visual/Textual Cues. By Steven Kappes and Yan Cao

Classifying Images with Visual/Textual Cues. By Steven Kappes and Yan Cao Classifying Images with Visual/Textual Cues By Steven Kappes and Yan Cao Motivation Image search Building large sets of classified images Robotics Background Object recognition is unsolved Deformable shaped

More information

Clustering: Overview and K-means algorithm

Clustering: Overview and K-means algorithm Clustering: Overview and K-means algorithm Informal goal Given set of objects and measure of similarity between them, group similar objects together K-Means illustrations thanks to 2006 student Martin

More information

A New Improved Hybridized K-MEANS Clustering Algorithm with Improved PCA Optimized with PSO for High Dimensional Data Set

A New Improved Hybridized K-MEANS Clustering Algorithm with Improved PCA Optimized with PSO for High Dimensional Data Set International Journal of Soft Computing and Engineering (IJSCE) ISSN: 2231-2307, Volume-2, Issue-2, May 2012 A New Improved Hybridized K-MEANS Clustering Algorithm with Improved PCA Optimized with PSO

More information

Shape spaces. Shape usually defined explicitly to be residual: independent of size.

Shape spaces. Shape usually defined explicitly to be residual: independent of size. Shape spaces Before define a shape space, must define shape. Numerous definitions of shape in relation to size. Many definitions of size (which we will explicitly define later). Shape usually defined explicitly

More information

CSE 158. Web Mining and Recommender Systems. Midterm recap

CSE 158. Web Mining and Recommender Systems. Midterm recap CSE 158 Web Mining and Recommender Systems Midterm recap Midterm on Wednesday! 5:10 pm 6:10 pm Closed book but I ll provide a similar level of basic info as in the last page of previous midterms CSE 158

More information

Data Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University

Data Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Data Mining Chapter 3: Visualizing and Exploring Data Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Exploratory data analysis tasks Examine the data, in search of structures

More information

( ) =cov X Y = W PRINCIPAL COMPONENT ANALYSIS. Eigenvectors of the covariance matrix are the principal components

( ) =cov X Y = W PRINCIPAL COMPONENT ANALYSIS. Eigenvectors of the covariance matrix are the principal components Review Lecture 14 ! PRINCIPAL COMPONENT ANALYSIS Eigenvectors of the covariance matrix are the principal components 1. =cov X Top K principal components are the eigenvectors with K largest eigenvalues

More information

CSE 258 Lecture 5. Web Mining and Recommender Systems. Dimensionality Reduction

CSE 258 Lecture 5. Web Mining and Recommender Systems. Dimensionality Reduction CSE 258 Lecture 5 Web Mining and Recommender Systems Dimensionality Reduction This week How can we build low dimensional representations of high dimensional data? e.g. how might we (compactly!) represent

More information

Machine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves

Machine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves Machine Learning A 708.064 11W 1sst KU Exercises Problems marked with * are optional. 1 Conditional Independence I [2 P] a) [1 P] Give an example for a probability distribution P (A, B, C) that disproves

More information

Image Processing. Image Features

Image Processing. Image Features Image Processing Image Features Preliminaries 2 What are Image Features? Anything. What they are used for? Some statements about image fragments (patches) recognition Search for similar patches matching

More information

Local Features: Detection, Description & Matching

Local Features: Detection, Description & Matching Local Features: Detection, Description & Matching Lecture 08 Computer Vision Material Citations Dr George Stockman Professor Emeritus, Michigan State University Dr David Lowe Professor, University of British

More information

Edge and corner detection

Edge and corner detection Edge and corner detection Prof. Stricker Doz. G. Bleser Computer Vision: Object and People Tracking Goals Where is the information in an image? How is an object characterized? How can I find measurements

More information

Biometrics Technology: Image Processing & Pattern Recognition (by Dr. Dickson Tong)

Biometrics Technology: Image Processing & Pattern Recognition (by Dr. Dickson Tong) Biometrics Technology: Image Processing & Pattern Recognition (by Dr. Dickson Tong) References: [1] http://homepages.inf.ed.ac.uk/rbf/hipr2/index.htm [2] http://www.cs.wisc.edu/~dyer/cs540/notes/vision.html

More information

SYDE Winter 2011 Introduction to Pattern Recognition. Clustering

SYDE Winter 2011 Introduction to Pattern Recognition. Clustering SYDE 372 - Winter 2011 Introduction to Pattern Recognition Clustering Alexander Wong Department of Systems Design Engineering University of Waterloo Outline 1 2 3 4 5 All the approaches we have learned

More information

University of Florida CISE department Gator Engineering. Clustering Part 2

University of Florida CISE department Gator Engineering. Clustering Part 2 Clustering Part 2 Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville Partitional Clustering Original Points A Partitional Clustering Hierarchical

More information

Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin

Clustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014

More information

Sets the default folder to be the folder where this notebook is saved. That is a fancy way of saying

Sets the default folder to be the folder where this notebook is saved. That is a fancy way of saying 1 1 1 1 1 1 1 1 1 1 1 1 In this module, you will write algorithms that bring two meshes into alignment using translations, rotations, and local Laplacian-based deformations. Alignment is useful for establishing

More information

Face detection and recognition. Many slides adapted from K. Grauman and D. Lowe

Face detection and recognition. Many slides adapted from K. Grauman and D. Lowe Face detection and recognition Many slides adapted from K. Grauman and D. Lowe Face detection and recognition Detection Recognition Sally History Early face recognition systems: based on features and distances

More information

Face Recognition At-a-Distance Based on Sparse-Stereo Reconstruction

Face Recognition At-a-Distance Based on Sparse-Stereo Reconstruction Face Recognition At-a-Distance Based on Sparse-Stereo Reconstruction Ham Rara, Shireen Elhabian, Asem Ali University of Louisville Louisville, KY {hmrara01,syelha01,amali003}@louisville.edu Mike Miller,

More information

Deep Learning for Computer Vision

Deep Learning for Computer Vision Deep Learning for Computer Vision Spring 2018 http://vllab.ee.ntu.edu.tw/dlcv.html (primary) https://ceiba.ntu.edu.tw/1062dlcv (grade, etc.) FB: DLCV Spring 2018 Yu Chiang Frank Wang 王鈺強, Associate Professor

More information

Lecture 8 Object Descriptors

Lecture 8 Object Descriptors Lecture 8 Object Descriptors Azadeh Fakhrzadeh Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University 2 Reading instructions Chapter 11.1 11.4 in G-W Azadeh Fakhrzadeh

More information

Clustering. Robert M. Haralick. Computer Science, Graduate Center City University of New York

Clustering. Robert M. Haralick. Computer Science, Graduate Center City University of New York Clustering Robert M. Haralick Computer Science, Graduate Center City University of New York Outline K-means 1 K-means 2 3 4 5 Clustering K-means The purpose of clustering is to determine the similarity

More information

Low Dimensional Surface Parameterisation with Applications in Biometrics

Low Dimensional Surface Parameterisation with Applications in Biometrics Low Dimensional Surface Parameterisation with Applications in Biometrics Wei Quan, Bogdan J. Matuszewski, Lik-Kwan Shark, Djamel Ait-Boudaoud Applied Digital Signal and Image Processing Centre University

More information

CPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016

CPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016 CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:

More information

UNIVERSITY OF CALIFORNIA COLLEGE OF ENGINEERING

UNIVERSITY OF CALIFORNIA COLLEGE OF ENGINEERING Page 1 of 7 E77 Midterm 1/Student ID: UNIVERSITY OF CALIFORNIA COLLEGE OF ENGINEERING E77: INTRODUCTION TO COMPUTER PROGRAMMING FOR SCIENTISTS AND ENGINEERS Spring 2006 First Midterm Exam February 22,

More information

Advanced Applied Multivariate Analysis

Advanced Applied Multivariate Analysis Advanced Applied Multivariate Analysis STAT, Fall 3 Sungkyu Jung Department of Statistics University of Pittsburgh E-mail: sungkyu@pitt.edu http://www.stat.pitt.edu/sungkyu/ / 3 General Information Course

More information

Unsupervised Learning

Unsupervised Learning Networks for Pattern Recognition, 2014 Networks for Single Linkage K-Means Soft DBSCAN PCA Networks for Kohonen Maps Linear Vector Quantization Networks for Problems/Approaches in Machine Learning Supervised

More information

Cluster Analysis. Ying Shen, SSE, Tongji University

Cluster Analysis. Ying Shen, SSE, Tongji University Cluster Analysis Ying Shen, SSE, Tongji University Cluster analysis Cluster analysis groups data objects based only on the attributes in the data. The main objective is that The objects within a group

More information

Unsupervised learning in Vision

Unsupervised learning in Vision Chapter 7 Unsupervised learning in Vision The fields of Computer Vision and Machine Learning complement each other in a very natural way: the aim of the former is to extract useful information from visual

More information

Matching and Recognition in 3D. Based on slides by Tom Funkhouser and Misha Kazhdan

Matching and Recognition in 3D. Based on slides by Tom Funkhouser and Misha Kazhdan Matching and Recognition in 3D Based on slides by Tom Funkhouser and Misha Kazhdan From 2D to 3D: Some Things Easier No occlusion (but sometimes missing data instead) Segmenting objects often simpler From

More information

Sage: Symmetry and Asymmetry in Geometric Data Version 1.21 (compiled 03/1114)

Sage: Symmetry and Asymmetry in Geometric Data Version 1.21 (compiled 03/1114) Sage: Symmetry and Asymmetry in Geometric Data Version 1.21 (compiled 03/1114) Eladio Marquez 2012-2014 Mammals Division University of Michigan Museum of Zoology http://www-personal.umich.edu/~emarquez/morph/

More information

Clustering. Informal goal. General types of clustering. Applications: Clustering in information search and analysis. Example applications in search

Clustering. Informal goal. General types of clustering. Applications: Clustering in information search and analysis. Example applications in search Informal goal Clustering Given set of objects and measure of similarity between them, group similar objects together What mean by similar? What is good grouping? Computation time / quality tradeoff 1 2

More information

COSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor

COSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor COSC160: Detection and Classification Jeremy Bolton, PhD Assistant Teaching Professor Outline I. Problem I. Strategies II. Features for training III. Using spatial information? IV. Reducing dimensionality

More information

Direct Matrix Factorization and Alignment Refinement: Application to Defect Detection

Direct Matrix Factorization and Alignment Refinement: Application to Defect Detection Direct Matrix Factorization and Alignment Refinement: Application to Defect Detection Zhen Qin (University of California, Riverside) Peter van Beek & Xu Chen (SHARP Labs of America, Camas, WA) 2015/8/30

More information

Solution Sketches Midterm Exam COSC 6342 Machine Learning March 20, 2013

Solution Sketches Midterm Exam COSC 6342 Machine Learning March 20, 2013 Your Name: Your student id: Solution Sketches Midterm Exam COSC 6342 Machine Learning March 20, 2013 Problem 1 [5+?]: Hypothesis Classes Problem 2 [8]: Losses and Risks Problem 3 [11]: Model Generation

More information

Geometric Registration for Deformable Shapes 3.3 Advanced Global Matching

Geometric Registration for Deformable Shapes 3.3 Advanced Global Matching Geometric Registration for Deformable Shapes 3.3 Advanced Global Matching Correlated Correspondences [ASP*04] A Complete Registration System [HAW*08] In this session Advanced Global Matching Some practical

More information

Image Warping. Srikumar Ramalingam School of Computing University of Utah. [Slides borrowed from Ross Whitaker] 1

Image Warping. Srikumar Ramalingam School of Computing University of Utah. [Slides borrowed from Ross Whitaker] 1 Image Warping Srikumar Ramalingam School of Computing University of Utah [Slides borrowed from Ross Whitaker] 1 Geom Trans: Distortion From Optics Barrel Distortion Pincushion Distortion Straight lines

More information

CLUSTERING. CSE 634 Data Mining Prof. Anita Wasilewska TEAM 16

CLUSTERING. CSE 634 Data Mining Prof. Anita Wasilewska TEAM 16 CLUSTERING CSE 634 Data Mining Prof. Anita Wasilewska TEAM 16 1. K-medoids: REFERENCES https://www.coursera.org/learn/cluster-analysis/lecture/nj0sb/3-4-the-k-medoids-clustering-method https://anuradhasrinivas.files.wordpress.com/2013/04/lesson8-clustering.pdf

More information

Analysis of Functional MRI Timeseries Data Using Signal Processing Techniques

Analysis of Functional MRI Timeseries Data Using Signal Processing Techniques Analysis of Functional MRI Timeseries Data Using Signal Processing Techniques Sea Chen Department of Biomedical Engineering Advisors: Dr. Charles A. Bouman and Dr. Mark J. Lowe S. Chen Final Exam October

More information

Unsupervised Learning

Unsupervised Learning Unsupervised Learning Fabio G. Cozman - fgcozman@usp.br November 16, 2018 What can we do? We just have a dataset with features (no labels, no response). We want to understand the data... no easy to define

More information

Face Recognition. Programming Project. Haofu Liao, BSEE. Department of Electrical and Computer Engineering. Northeastern University.

Face Recognition. Programming Project. Haofu Liao, BSEE. Department of Electrical and Computer Engineering. Northeastern University. Face Recognition Programming Project Haofu Liao, BSEE June 23, 2013 Department of Electrical and Computer Engineering Northeastern University 1. How to build the PCA Mex Funtion 1.1 Basic Information The

More information

Note Set 4: Finite Mixture Models and the EM Algorithm

Note Set 4: Finite Mixture Models and the EM Algorithm Note Set 4: Finite Mixture Models and the EM Algorithm Padhraic Smyth, Department of Computer Science University of California, Irvine Finite Mixture Models A finite mixture model with K components, for

More information

Data Mining - Data. Dr. Jean-Michel RICHER Dr. Jean-Michel RICHER Data Mining - Data 1 / 47

Data Mining - Data. Dr. Jean-Michel RICHER Dr. Jean-Michel RICHER Data Mining - Data 1 / 47 Data Mining - Data Dr. Jean-Michel RICHER 2018 jean-michel.richer@univ-angers.fr Dr. Jean-Michel RICHER Data Mining - Data 1 / 47 Outline 1. Introduction 2. Data preprocessing 3. CPA with R 4. Exercise

More information

K-means clustering Based in part on slides from textbook, slides of Susan Holmes. December 2, Statistics 202: Data Mining.

K-means clustering Based in part on slides from textbook, slides of Susan Holmes. December 2, Statistics 202: Data Mining. K-means clustering Based in part on slides from textbook, slides of Susan Holmes December 2, 2012 1 / 1 K-means Outline K-means, K-medoids Choosing the number of clusters: Gap test, silhouette plot. Mixture

More information

Introduction to MATLAB Practical 1

Introduction to MATLAB Practical 1 Introduction to MATLAB Practical 1 Daniel Carrera November 2016 1 Introduction I believe that the best way to learn Matlab is hands on, and I tried to design this practical that way. I assume no prior

More information

Lecture 4 Face Detection and Classification. Lin ZHANG, PhD School of Software Engineering Tongji University Spring 2018

Lecture 4 Face Detection and Classification. Lin ZHANG, PhD School of Software Engineering Tongji University Spring 2018 Lecture 4 Face Detection and Classification Lin ZHANG, PhD School of Software Engineering Tongji University Spring 2018 Any faces contained in the image? Who are they? Outline Overview Face detection Introduction

More information

9.1. K-means Clustering

9.1. K-means Clustering 424 9. MIXTURE MODELS AND EM Section 9.2 Section 9.3 Section 9.4 view of mixture distributions in which the discrete latent variables can be interpreted as defining assignments of data points to specific

More information

Image Compression with Singular Value Decomposition & Correlation: a Graphical Analysis

Image Compression with Singular Value Decomposition & Correlation: a Graphical Analysis ISSN -7X Volume, Issue June 7 Image Compression with Singular Value Decomposition & Correlation: a Graphical Analysis Tamojay Deb, Anjan K Ghosh, Anjan Mukherjee Tripura University (A Central University),

More information

CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]

CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points] CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, 2015. 11:59pm, PDF to Canvas [100 points] Instructions. Please write up your responses to the following problems clearly and concisely.

More information

INFO0948 Fitting and Shape Matching

INFO0948 Fitting and Shape Matching INFO0948 Fitting and Shape Matching Renaud Detry University of Liège, Belgium Updated March 31, 2015 1 / 33 These slides are based on the following book: D. Forsyth and J. Ponce. Computer vision: a modern

More information

Feature Extractors. CS 188: Artificial Intelligence Fall Nearest-Neighbor Classification. The Perceptron Update Rule.

Feature Extractors. CS 188: Artificial Intelligence Fall Nearest-Neighbor Classification. The Perceptron Update Rule. CS 188: Artificial Intelligence Fall 2007 Lecture 26: Kernels 11/29/2007 Dan Klein UC Berkeley Feature Extractors A feature extractor maps inputs to feature vectors Dear Sir. First, I must solicit your

More information

Machine Learning for Signal Processing Clustering. Bhiksha Raj Class Oct 2016

Machine Learning for Signal Processing Clustering. Bhiksha Raj Class Oct 2016 Machine Learning for Signal Processing Clustering Bhiksha Raj Class 11. 13 Oct 2016 1 Statistical Modelling and Latent Structure Much of statistical modelling attempts to identify latent structure in the

More information

Clustering human body shapes using k-means algorithm

Clustering human body shapes using k-means algorithm Clustering human body shapes using k-means algorithm Guillermo Vinué Visús PhD Student Faculty of Mathematics University of Valencia, Spain Jointly with Guillermo Ayala Gallego, Juan Domingo Esteve, Esther

More information

Exercise 2. AMTH/CPSC 445a/545a - Fall Semester September 21, 2017

Exercise 2. AMTH/CPSC 445a/545a - Fall Semester September 21, 2017 Exercise 2 AMTH/CPSC 445a/545a - Fall Semester 2016 September 21, 2017 Problem 1 Compress your solutions into a single zip file titled assignment2.zip, e.g. for a student named

More information

Last time... Coryn Bailer-Jones. check and if appropriate remove outliers, errors etc. linear regression

Last time... Coryn Bailer-Jones. check and if appropriate remove outliers, errors etc. linear regression Machine learning, pattern recognition and statistical data modelling Lecture 3. Linear Methods (part 1) Coryn Bailer-Jones Last time... curse of dimensionality local methods quickly become nonlocal as

More information

Regression on SAT Scores of 374 High Schools and K-means on Clustering Schools

Regression on SAT Scores of 374 High Schools and K-means on Clustering Schools Regression on SAT Scores of 374 High Schools and K-means on Clustering Schools Abstract In this project, we study 374 public high schools in New York City. The project seeks to use regression techniques

More information

Least-Squares Fitting Algorithms of the NIST Algorithm Testing System

Least-Squares Fitting Algorithms of the NIST Algorithm Testing System [J. Res. Natl. Inst. Stand. Technol. 103, 633 (1998)] Least-Squares Fitting Algorithms of the NIST Algorithm Testing System Volume 103 Number 6 November December 1998 Craig M. Shakarji National Institute

More information

CSE 6242 A / CS 4803 DVA. Feb 12, Dimension Reduction. Guest Lecturer: Jaegul Choo

CSE 6242 A / CS 4803 DVA. Feb 12, Dimension Reduction. Guest Lecturer: Jaegul Choo CSE 6242 A / CS 4803 DVA Feb 12, 2013 Dimension Reduction Guest Lecturer: Jaegul Choo CSE 6242 A / CS 4803 DVA Feb 12, 2013 Dimension Reduction Guest Lecturer: Jaegul Choo Data is Too Big To Do Something..

More information

Learning from Data Linear Parameter Models

Learning from Data Linear Parameter Models Learning from Data Linear Parameter Models Copyright David Barber 200-2004. Course lecturer: Amos Storkey a.storkey@ed.ac.uk Course page : http://www.anc.ed.ac.uk/ amos/lfd/ 2 chirps per sec 26 24 22 20

More information

Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes

Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes TDT 03 - Advanced Topics in Computer Graphics Presentation by Ruben H. Fagerli Paper to be summarized Using Spin Images for Efficient

More information

Linear Discriminant Analysis for 3D Face Recognition System

Linear Discriminant Analysis for 3D Face Recognition System Linear Discriminant Analysis for 3D Face Recognition System 3.1 Introduction Face recognition and verification have been at the top of the research agenda of the computer vision community in recent times.

More information