Data Foundations: Data Types and Data Preprocessing. Introduction. Data, tasks and simple visualizations. Data sets. Some key data factors?

Size: px

Start display at page:

Download "Data Foundations: Data Types and Data Preprocessing. Introduction. Data, tasks and simple visualizations. Data sets. Some key data factors?"

Eric Waters
6 years ago
Views:

Insttute for Vsualzaton and Percepton Research Insttute for Vsualzaton and Percepton Research Data Foundatons: Data Types and Data Preprocessng Lecture 2 - Data Foundatons September 8, 2010 Georges

1 Insttute for Vsualzaton and Percepton Research Insttute for Vsualzaton and Percepton Research Data Foundatons: Data Types and Data Preprocessng Lecture 2 - Data Foundatons September 8, 2010 Georges Grnsten, Unversty of Massachusetts Lowell nformaton vsualzed knowledge dscovered decsons made nformaton vsualzed knowledge dscovered decsons made Introducton Data has many sources t can be gathered from sensors or surveys, or t can be generated by smulatons and computatons Data can be raw (untreated) or t can be derved from raw data va some process smoothng, nose-removal, scalng, or nterpolaton Data, tasks and smple vsualzatons Data 1D, 2D, 3D,, nd Structured and unstructured Tasks Present, Confrm, Explore Query Summarze Analyze Smple Vsualzatons Ponts Lne and curves Charts and graphs Some key data factors? Very large number of parameters more than 10 5 to 10 6 Very large data sets more than 10 6 not uncommon, some petabyte now (10 15 ) Multple data types easy: dscrete and contnuous; tougher: categorcal, nomnal Nosy data often not unform Mssng values could be mportant Lots of dfferent tasks Many vsualzatons Data sets Lst of records Each record conssts of one or more observatons Each observaton or varable may be A sngle number or symbol A more complex structure Each varable may be ndependent of dependent Data may be generated by a process or functon The ndependent varables defne the functon s doman The dependent varables defne the functon s range 1

2 Types or Categores of Data There are many ways to classfy data Quanttatve or qualtatve Precse or approxmate The followng four levels were proposed by Stanley Smth Stevens n hs 1946 artcle On the theory of scales of measurement Nomnal, ordnal, nterval, or rato Nomnal Data (symbolc, categorcal) Varables are placed nto mutually exclusve categores n whch members of one category are qualtatvely dfferent from the members of any other category The order of the categores s arbtrary Numerals may be assgned as labels or names (e.g., yes = 0, no = 1) Nomnal Data (symbolc, categorcal) Examples har color: brown, black, blond, red gender: male, female genomc base pars: A, C, T, G martal status: yes, no or sngle, marred, dvorced, wdowed Mappng of numbers to labels possble many ways male = 0, female = 1 or male = 1, female = 2 One value s not necessarly greater than another Statstcal computatons typcally have no meanng (although some, such as mode, can be defned) Nomnal (symbolc, categorcal) Measurement Only one way to measure a nomnal varable (relgon, gender, ) Operatons Mappng (many possble mappngs) Equalty/Inequalty (same/dfferent) Ordnal Data Rank-orderng The order of the categores s relevant and numerals or labels havng nterpretaton are assgned to the labels Categorzaton of data wth orderng order nformaton avalable but there s no nformaton about the magntude of the dstance between adjacent categores Some statstcal computatons may not have any meanng Perceptual dffculty scale very dffcult = 10, moderately dffcult = 8, average dffculty = 5,, easy = 0 Weapon used by severty Machne gun = 1, rfle = 2, gun = 3, knfe = 4, paper clp = 5, Lkert scale of agreement 5 = strongly agree to 1 = strongly dsagree Ordnal Data Operatons equalty/nequalty less than/more than (order) Example: students 1 st, 2 nd, 3 rd Lmtaton 1 st better than 2 nd - by how much? cannot compare dfferences between categores 2

3 Numerc (dscrete, contnuous) Dscrete numerc: Integer Numercal dstance between adjacent unts s equal Contnuous numerc: Real Any value wth arbtrary precson s possble / no gaps n scale May lack an absolute zero (a value whch represents the complete absence of the characterstc beng measured) the zero value s an arbtrary startng pont that could be replaced by any other value Numerc (dscrete, contnuous) Operatons a) equalty/nequalty b) less than/more than (order) c) addton/subtracton d) dstance metrcs Interval Data Contnuous data where the data falls n a range of numbers and where data dfferences are meanngful Ratos may have no meanng snce ranges can be lnearly transformed to other scales changng the nterpretaton of zero Dstance dfferences have meanng ( and are smlar) Ratos of dfferences can have meanng and the mean and medan have meanng Temperature n Celsus or Fahrenhet Twce the temperature depends on whch scale s used IQ measure Rato Data Contnuous data where both dfferences and ratos are meanngful and where zero has meanng Data whch can be classfed as Interval data can often as well be classfed as rato data Geometrc mean can only be appled to rato data and arthmetc mean s extremely meanngful Temperature, mass, energy,... Age, weght Number of students at colloqua Examples of Ordnal, Interval and Rato Weght n pounds Heght n meters Age n years Gene expresson Salary Dstance n feet from pump Temperature n degrees Relatonshp among categores Snce each category provdes more computatonal possbltes one can say Rato s more meanngful than nterval Interval s more meanngful than ordnal Ordnal s more meanngful than nomnal 3

network) data Complex data models nformaton vsualzed knowledge dscovered decsons made Basc defntons Let DB = { x,.

4 Levels of Measurement Typcal Data Cars make model year mles per gallon cost number of cylnders weght... Insttute for Vsualzaton and Percepton Research Typcal Data Classes Typcal Data Classes 1D scalar 2D scalar 3D scalar Multvarate (multdmensonal) data Vector data Lnked (herarchcal, network) data Complex data models nformaton vsualzed knowledge dscovered decsons made Basc defntons Let DB = { x,..., x n } be a database wth n 1 elements { } Dom = char, nt, real, strng s called the doman of the elements of database DB x " D1!...! Dd, D1,..., Dd! Dom 1D scalar data 1D scalar x! DB s an ordered d-tuple of 1 d the form x = x,..., x wth j x! ( ) (1) x " D!...! D, 1 d (2) D wth () j () d = 1 d-tuple = 1-tuple D!{ nt,real} 1 D j! Dom for all j! 1 4

5 2D Scalar Sequence of ordered pars v = (x, y) wth x and y n some scalar set Where the ndces are for example ε {1, 2, 3,..., n} ε {a, b, c,..., z} ε a subset of R Examples a tme seres a set of ponts n the (x, y) plane 3D Scalar Sequence of ordered trplets v = (x, y, z) wth x, y and z n some scalar set Where the ndces are for example ε {1, 2, 3,..., n} ε {a, b, c,..., z} ε a subset of R Examples a tme seres of 2D ponts a set of ponts n (x, y, z) space Formal Defntons: 3D scalar data 3D scalar x! DB s ordered d-tuple of 1 d the form x = x,..., x wth j x! ( ) (1) x " D!...! D, 1 d (2) D wth () () j D k!{ nt,real}, D j! Dom forall k!{ 1,2,3} j! k D scalar data d = 3 d-tuple = 3-tuple Vector Data A generalzaton of the above n-dmensonal vectors v k = (x 1, x 2, x 3,..., x n ) where x s n some scalar set And where the ndces are for example k ε {1, 2, 3,..., n} k ε {a, b, c,..., z} k ε a subset of R Examples a tme seres of n - 1 dmensonal ponts a set of ponts n n dmensonal space Vector data n-dmensonal vector DB s an ordered 1 d d-tuple of the form x = x,..., x wth (1) x " D!...! D, j x 1 d x! ( ) (2) s a vector forsome j!{ 1,...,n} Vector data d = n d-tuple = n-tuple Tme Seres Data A generalzaton of the above n-dmensonal vectors v k = (x 1, x 2, x 3,..., x n ) where x s n some scalar set And the ndex set {k} s based on tme 0 t 1 < t 2 < t 3 <... < t n The ndex set s often ncluded as a parameter n the n-dmensonal vector but t s brought out here as a specal case because of ts mportance Ths dentfes tme as the specal varable 5

extends the concept of vector n whch each coordnate has the same data type to one n whch the data types are dfference Examples patent records census data Insttute for Vsualzaton and Percepton

6 Multdmensonal Data A generalzaton of the above n-dmensonal vector v k = (x 1, x 2, x 3,..., x n ) where each x s of some possbly varyng data type, not necessarly all the same Each record thus conssts of a number of varables each havng ts own data type The ndex set {k} s as before Ths extends the concept of vector n whch each coordnate has the same data type to one n whch the data types are dfference Examples patent records census data Insttute for Vsualzaton and Percepton Research Trees, Graphs and Networks Lots of exampl es nformaton vsualzed knowledge dscovered decsons made Herarchcal data Herarchcal data herarchcal data can be represented through Graphs Fle systems are typcal herarchcal data Example of a Web Ste Herarchy Herarchcal data Relatonal database model Webste ID Parent ID Chld ID Name of the Webste 0 NULL 1 Index Index Index Index Index Index Index Index Index About Me 2 0 NULL Resume 3 0 NULL GuestBook 10 1 NULL Boot 11 5 NULL NULL Dune 13 5 NULL Multplcty Star Wars Star Wars NULL Books NULL Lucas Herarchcal data In most cases the data are not gven n herarchcal form but are stored n multdmensonal varables Goal: Transform the data nto herarchcal form Algorthm: repeat (1) Select a dmenson - the sequence of selectng the dmensons s mportant, select the most mportant dmenson (could do t dfferently) (2) Segment the attrbutes nto some classes provded the chosen attrbutes are not categorcal untl (maxmum herarchy level s reached) 6

7 Complex structured data (graph) A graph G = (V, E) conssts of a set V = { v k }, the vertces e! E a set E = { e m } the edges Each edge e ε E s assgned n a unque way to an ordered or unordered par of (not necessarly dfferent) vertces: e = (v k, v m ) These edges are sad to connect the vertces Complex structured data (graph) If every e ε E s assgned to an ordered par of nodes e = (v, w), then the graph s called a drected graph If every e ε E s assgned to an unordered par of nodes e = {v, w}, then the graph s called an undrected graph Edges may have addtonal meanngs (weghts) n whch case the graph s often called a network We then can defne cyclc, acyclc, DAG, tree, and varous metrcs such as n-number and out-number Drected Graph: Paper Flow Undrected graph: Socal Network of 9-11 Terrorsts Insttute for Vsualzaton and Percepton Research Data Preprocessng Movng raw data nto usable data nformaton vsualzed knowledge dscovered decsons made Data preprocessng Measurement and error Metadata and statstcs Mssng values Data Cleansng Normalzaton Segmentaton Samplng and subsettng Dmensonal reducton Aggregaton and summarzaton Smoothng and flterng 7

Measurement and Error Random Error: Nose Any observaton s composed of the true value

data set For example, the count of the tems n (1, 3, 6, 4) s 4 sum the tems n a lst For

For example, the avg of the tems n (1, 3, 6, 4) s 3.

(name of dataset, ) Data Qualty (completeness, attrbute accuracy, ) Dstrbuton (formats,

8 Measurement and Error Random Error: Nose Any observaton s composed of the true value plus some error value Systematc Error: Bas Aggregaton/Summarzaton count the tems n a data set For example, the count of the tems n (1, 3, 6, 4) s 4 sum the tems n a lst For example, the sum of the lst (1, 3, 6, 4) s 14 average (avg) of all tems n a data set For example, the avg of the tems n (1, 3, 6, 4) s 3.5 Metadata and Statstcs Data about Data Metadata descrbes the content, qualty, condton, and other characterstcs of data e.g. mn, max, avg, Metadata are not the actual data tself Metadata may nclude Identfcaton (name of dataset, ) Data Qualty (completeness, attrbute accuracy, ) Dstrbuton (formats, meda, who holds the data, ) Important for a correct/useful vsualzaton Insttute for Vsualzaton and Percepton Research Mssng Values nformaton vsualzed knowledge dscovered decsons made 8

9 Mssng Values and Data Cleanng Mssng and empty values Problem defnton Approxmaton vs nterpolaton Lnear regresson Pecewse polynomal (splne) nterpolaton Mssng and Empty Values mssng value of a varable s one that has not been entered nto the data set but there exsts an actual value n the real world n whch the measurement was made empty value n a varable s one for whch no real world value exsts Mssng and Empty Values: Example: marketng research company Fve testers were hred to test fve dfferent products for ease of use and effectveness Smple method for handlng mssng values Ignore the tuple Fll n the mssng value manually Use a global constant to fll n the mssng value Use the attrbute mean to fll n the mssng value Use the most probable value to fll n the mssng value may be determned wth regresson, nterpolaton, nference-based tools usng a Bayesan formalsm, or decson tree nducton (e.g., Shaffer) General Problems wth Mssng Values There may be some nformaton content mssng Example: Credt applcaton may warn and dentfy that certan useful nformaton appears as a result of certan felds not completed by an applcant The mssng value s necessary for computaton Example: Age s mportant for estmatng relablty Problem Defnton Create and nsert some replacement value for the mssng value The objectve s to nsert a value that nether adds nor subtracts nformaton from the data set Note that for age ths s trcky (older typcally ncreases relablty) and we mght decde not to fll n values Soluton Use approxmaton or nterpolaton to fnd the mssng values 9

10 Problem Defnton We have a gven set of n ponts (x, y ) wth y = f(x ) for = 1,..., n The problem that we are tryng to solve s to fnd a functon y = f (x) for whch Problem Defnton Informaton s carred n the relatonshp between the values wthn a sngle varable (ts dstrbuton) and the relatonshp to other varables y = f(x) for an arbtrary x (or a specfc subset) There may be several such functons or even possbly no smple ones that we can deal wth Insttute for Vsualzaton and Percepton Research Approxmaton and Interpolaton Approxmaton vs Interpolaton For approxmaton we want f (x) f (x ) ε for small ε > 0 For nterpolaton we want f (x) f (x ) = 0 nformaton vsualzed knowledge dscovered decsons made Note that approxmaton s less strngent Clmate Example Clmate Example Problem Spatally random dstrbuted automatc weather statons Temperature data approxmaton based on trangulaton 10

Approxmaton vs Interpolaton Approxmaton Regresson (lnear, quadratc, ) Interpolaton Polynomal (Lagrange bass, Newton form) Pecewse polynomal (cubc splnes, ) Orthogonal polynomals (Legendre, )

11 Approxmaton vs Interpolaton Approxmaton Regresson (lnear, quadratc, ) Interpolaton Polynomal (Lagrange bass, Newton form) Pecewse polynomal (cubc splnes, ) Orthogonal polynomals (Legendre, ) Trgonometrc functons Approxmaton Clmate data approxmaton based on trangulaton (a), (f) temperature (b) ar pressure (c) humdty (d) sea surface temperature (e) vapor pressure Insttute for Vsualzaton and Percepton Research Lnear Regresson Lnear Regresson Lnear regresson tres to dscover the parameters of the straght lne equaton that best fts the data pont The expresson descrbng a straght lne s y = b 1 x + b 0 where b 0 s a constant that ndcates where the straght lne crosses the y-axs n state space (the y-ntercept) and b 1 represents the slope of the lne nformaton vsualzed knowledge dscovered decsons made Lnear Regresson Lnear Regresson mnmzes the least square error : y yˆ : n #( y " yˆ )! mn = 1 Actual y 2 Estmated y value value Lnear Regresson Soluton Step 1 Determne b 1 ( xy) " (! x)( y) 2 n x " ( x) n!!!! b 1 = 2 11

Lnear Regresson Soluton Two-Varable Lnear

Source: Data from BP Research Two-Varable

12 Lnear Regresson Soluton Two-Varable Lnear Model Step 2 Determne b 0 once b 1 s known b 0 = y - b 1 x x y s the mean value of x s the mean value of y Measurements on 48 rock samples from a petroleum reservor Area: area of pores space, n pxels out of 256 by 256 Permeter: permeter of pores n pxels Source: Data from BP Research Two-Varable Lnear Model Lnear Regresson e Lnear Regresson Insttute for Vsualzaton and Percepton Research Splne Interpolaton nformaton vsualzed knowledge dscovered decsons made 12

13 Interpolatng Splne: Pecewse Polynomal Interpolaton Set of data ponts {( x, y ),..., ( x, y 0 0 n n) } Pass a sngle polynomal through the ponts note that ths can sometmes lead to oscllatons n the nterpolatng polynomal The nterpolatng polynomal lnkng the data pont s most often selected to be a cubc Cubcs are dfferentable and provde second order contnuty (the dervatves of neghbourng cubcs can be matched) Polynomal Interpolaton Pecewse Polynomal (Splne) Interpolaton Pecewse Polynomal (Splne) Interpolaton k 2 ( x) = s + s ( x! x ) + s ( x! x ) + s ( x x ) 3 s! x [ x k x ]!, k +1 k, 0 k,1 k k,2 k k,3 k subject to the followng constrants: s k( xk ) = yk Interpolaton of ( x k, y k ) sk( xk + 1 ) = sk + 1 ( xk + 1 ) Contnuty of nterpolant ' ' sk( xk + 1) = sk + 1( xk + 1 ) Contnuty of frst dervatves '' '' s x s x Contnuty of second dervatves k ( ) ( ) = k+ 1 k+ 1 k+ 1 Insttute for Vsualzaton and Percepton Research Normalzaton Normalzaton Problem Defnton Lnear Square Root Logarthmc Quantle nformaton vsualzed knowledge dscovered decsons made 13

14 Normalzaton Example Normalzaton Example (con t) LA-County Chcago-County Chcago-County LA-County Problem Defnton Normalzaton (lnear) Normalzaton Example (lnear) Normalzaton (square root) 14

15 Normalzaton Example (square root) Normalzaton (logarthmc) Normalzaton Example (logarthmc) Normalzaton Functons Normalzaton (quantle) 3.3 Data Preprocessng Quantle Normalzaton Normalzaton 15

3.3 Data Preprocessng Normalzaton Example (quantle) 3.3.3 Normalzaton Normalzaton vs.

nformaton vsualzed knowledge dscovered decsons made Quantle Normalzaton Segmentaton Manual/Automatc Segmentaton Manual/Automatc Segmentaton Problem

16 3.3 Data Preprocessng Normalzaton Example (quantle) Normalzaton Normalzaton vs. Contnuous Data Streams Insttute for Vsualzaton and Percepton Research Standard Normalzaton Square Root Normalzaton Logarthmc Normalzaton Segmentaton nformaton vsualzed knowledge dscovered decsons made Quantle Normalzaton Segmentaton Manual/Automatc Segmentaton Manual/Automatc Segmentaton Problem Defnton k-means Lnkage -based Methods Kernel Densty Estmaton Manual Segmentaton based upon Attrbute values/ranges Topologcal propertes Automatc Segmentaton Algorthms (Clusterng Algorthms) k-means Kernel Densty Estmaton 16

Problem Defnton Gven: A data set wth N d-dmensonal data tems Task: Determne a (natural) parttonng of the data set nto a number of clusters (k) and a nose parameter Problem Defnton Effectve and

17 Problem Defnton Gven: A data set wth N d-dmensonal data tems Task: Determne a (natural) parttonng of the data set nto a number of clusters (k) and a nose parameter Problem Defnton Effectve and effcent clusterng algorthms for large hgh-dmensonal data sets wth hgh nose level Requres Scalablty wth respect to the number of data ponts (N) the number of dmensons (d) the nose level k-means Gven a data set Determne k prototypes (p) of the data Assgn data ponts to nearest prototype Mnmze dstance crteron: k-means 2 Iteratve Algorthm Shft the prototypes towards the mean of ther pont set Re-assgn the data ponts to the nearest prototype k N "" = 1 j= 1 d( p, x j )! mn k-means Algorthm Input: D ={p 1,...,p n }(Ponts to be clustered), k (number of clusters) Output: C={c 1,...,c k } (cluster centrods) m: D {1,...,k} (cluster membershp) kmeans(dataset D,nteger k) Set C to ntal value (e.g. random selecton of D) For each p D m( p ) = arg mn dstance(p,c j) End j! { 1,..., k} K-means Whle m has changed For each {1,...,k} Recompute c as the centrod of {p m(p)=} End For each p D End End m ( p j j! { 1,..., k} ) = arg mn dstance(p,c ) 17

k-means Example: Lnkage-based method Herarchcal clusterng 5 1 1 1 2 4 3 6 5 8 9 7 5 1 2 3 4 5 6 7 8 9 2 1 0 dstance between clusters Types of herarchcal methods Bottom-up

Methods Method of Wshart Reduce data set Apply sngle lnkage Kernel Densty Estmaton Kernel Densty Estmaton Influence Functon The nfluence of a data pont y at a pont x n the

18 k-means Example: Lnkage-based method Herarchcal clusterng dstance between clusters Types of herarchcal methods Bottom-up constructon of dendrogram (agglomeratve) Top-down constructon of dendrogram (dvsve) Lnkage-based Methods Sngle Lnkage Connected components for dstance d Lnkage-based Methods Method of Wshart Reduce data set Apply sngle lnkage Kernel Densty Estmaton Kernel Densty Estmaton Influence Functon The nfluence of a data pont y at a pont x n the data space s modeled by a functon Densty Functon Influence Functon: Influence of a data pont n ts neghborhood Densty Functon: Sum of the nfluences of all data ponts e.g., y x 18

Kernel Densty Estmaton Kernel Densty Estmaton Densty Functon The densty at a pont x n the data space s defned as the sum of the nfluences of all data ponts x Insttute for Vsualzaton and Percepton

Reducton of number of dmensons Remove rrelevant, weakly relevant, or redundant (hhghly correlated) attrbutes or dmensons Compress data th PCA, ICA, Subsettng Start wth a set of data tems and generate

19 Kernel Densty Estmaton Kernel Densty Estmaton Densty Functon The densty at a pont x n the data space s defned as the sum of the nfluences of all data ponts x Insttute for Vsualzaton and Percepton Research Data Reducton Samplng Dmensonal reducton Data Reducton Subsettng Set of Data Items Subset of Data Items Samplng Random Samplng Queryng SQL nformaton vsualzed knowledge dscovered decsons made Reducton of number of dmensons Remove rrelevant, weakly relevant, or redundant (hhghly correlated) attrbutes or dmensons Compress data th PCA, ICA, Subsettng Start wth a set of data tems and generate a subset of these data tems Samplng Random samplng Queryng SQL Samplng Motvaton data set s much larger than possble (tme- and/or space-wse) to work on Example: voters of an electon too large to study all of them, so use a representatve sample Important The selected subset must be selected such that t represents some well defned characterstcs of the whole data set especally those we re nterested n 19

Types of Samplng Non-probablstc samples Sample selected on some non-random bass (such as volunteers, accdental,

) Probablstc samples Sample selected on the bass of random selecton so that every element of the data set has an equal

samplng Cluster random samplng Based samplng Smple Random Samplng Systematc random samplng Elements are numbered 1 to N

Every k-th n element s selected startng wth a randomly chosen number between 1 and k A random samplng strategy s the

Usng ths method, locatons are determned by generatng a lst of random coordnates and placng the ponts at those coordnates.

20 Types of Samplng Non-probablstc samples Sample selected on some non-random bass (such as volunteers, accdental, convenence, selfselected, etc.) Probablstc samples Sample selected on the bass of random selecton so that every element of the data set has an equal chance of beng selected Types of Probablstc Samplng Smple random samplng Systematc random samplng Stratfed random samplng Cluster random samplng Based samplng Smple Random Samplng Systematc random samplng Elements are numbered 1 to N n some order N k! Every k-th n element s selected startng wth a randomly chosen number between 1 and k A random samplng strategy s the least based samplng method. Usng ths method, locatons are determned by generatng a lst of random coordnates and placng the ponts at those coordnates. Stratfed random samplng The data set s dvded nto non-overlappng subsets called strata Samplng from the strata s smple random Cluster random samplng The sample conssts of a selecton from randomly chosen groups of neghbourng elements (clusters) Clusters need not necessarly be natural aggregates, but can smply be artfcal dvde the populaton nto populaton clusters based on geographcal locaton (dstrcts, countes, states,...) 20

21 Dmensonal Reducton Dmenson Reducton - How? What s the problem? Large number of features represent an object The data s dffcult to vsualze, especally when some of the features are not dscrmnatory Irrelevant features may cause a reducton n the accuracy of the analyss algorthms Problem Defnton Concept Identfy the most mportant features of an object to smplfy the processng wthout loss of qualty to drectly vsualze the two/three most mportant features Problem Defnton Soluton The smplest approach s to dentfy mportant attrbutes based on nput from doman experts Another common approach s Prncpal Component Analyss (PCA) whch defnes new attrbutes (prncpal components or PCs) as mutuallyorthogonal lnear combnatons of the orgnal attrbutes Prncpal Component Analyss Goal to dscover the key hdden factors that explan the data PCA to reduce the dmensonalty of the data Smlar to cluster centrods 21

22 PCA PCA (con t) Computng the Egenvalues The Egenvalues 22

23 Egenvalues (con t) Egenvalues (con t) PCA Dmensonal Reducton Data can be projected onto a subspace spanned by the most mportant egenvectors X PCA = C X where the m! k matrx C contans the k egenvectors correspondng to the k largest egenvectors PCA Dmensonal Reducton PCA s an optmal way to project data n the mean square sense the squared error ntroduced n the projecton s mnmzed over all projectons onto a k dmensonal dpace But the egenvalue decomposton of the data covarance matrx (sze m x n for m- dmensonal data) s very expensve to compute SVD Dmensonal Reducton Sngular value decomposton SVD Dmensonal Reducton Data can be projected onto a subspace spanned by the left sngular vectors correspondng to the k largest sngular values where the orthogonal matrces U and V contan the left & rght sngular vectors of X and the dagonal matrx S contans the sngular values of X where the m x k matrx U k contans these k sngular vectors 23

24 Insttute for Vsualzaton and Percepton Research Questons? nformaton vsualzed knowledge dscovered decsons made 24

Subspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points;

Subspace clustering. Clustering. Fundamental to all clustering techniques is the choice of distance measure between data points; Subspace clusterng Clusterng Fundamental to all clusterng technques s the choce of dstance measure between data ponts; D q ( ) ( ) 2 x x = x x, j k = 1 k jk Squared Eucldean dstance Assumpton: All features