Discriminant Analysis: LDA, QDA, k-nn, Bayes, PLSDA, cart, Random forests

Transcription

1 enote 9 1 enote 9 Diriminant Analyi: LDA, QDA, k-nn, Baye, PLSDA, art, Random foret

2 enote 9 INDHOLD 2 Indhold 9 Diriminant Analyi: LDA, QDA, k-nn, Baye, PLSDA, art, Random foret Reading material LDA, QDA, k-nn, Baye Claifiation tree (CART) and random foret Example: Iri data Linear Diriminant Analyi, LDA Quadrati Diriminant Analyi, QDA Prediting NEW data Baye method k-nearet neighbourgh PLS-DA Random foret foretfloor Exerie Reading material LDA, QDA, k-nn, Baye Read in the Varmuza book: (not oering CARTS and random foret) Setion 5.1, Intro, 2.5 page Setion 5.2, Linear Method, 12 page Setion 5.3.3, Nearet Neighbourg (k-nn), 3 page Setion 5.7, Ealuation of laifiation, 3 page

3 enote READING MATERIAL 3 Alternatiely read in the Wehren book: (not oering CARTS and random foret) 7.1 Diriminant Analyi Linear Diriminant Analyi Croalidation Fiher LDA Quadrati Diriminant Analyi Model-Baed Diriminant Analyi Regularized Form of Diriminant Analyi Nearet-Neighbour Approahe Dirimination with Fat Data Matrie PCDA PLSDA Claifiation tree (CART) and random foret Read in the Varmuza book about laifiation (and regreion) tree: Setion 5.4 Claifiation Tree Setion Claifiation Tree (Setion Regreion Tree) Read in the Wehren book: 7.3 Tree-Baed Approahe Integrated Modelling and Validation 195 (9.7.1 Bagging 196) Random Foret 197 (9.7.3 Booting 202)

4 enote EXAMPLE: IRIS DATA Example: Iri data # we ue the iri data: data(iri3) #library(mass) Iri <- data.frame(rbind(iri3[,,1], iri3[,,2], iri3[,,3]), Sp = rep(("","",""), rep(50,3))) # We make a tet and training data: et.eed(4897) train <- ample(1:150, 75) tet <- (1:150)[-train] Iri_train <- Iri[train,] Iri_tet <- Iri[tet,] # Ditribution in three lae in training data: table(iri_train$sp) Linear Diriminant Analyi, LDA We ue the lda funtion from the MASS pakage: # PART 1: LDA library(mass) lda_train_loo <- lda(sp ~ Sepal.L. + Sepal.W. + Petal.L. + Petal.W., Iri_train, prior = (1, 1, 1)/3, CV = TRUE) The Speie fator ariable i expreed a the repone in a uual model expreion with the four meaurement ariable a the x. The CV=TRUE option hoie perform full LOO ro alidation. The prior option work a:

5 enote EXAMPLE: IRIS DATA 5 the prior probabilitie of la memberhip. If unpeified, the la proportion for the training et are ued. If preent, the probabilitie hould be peified in the order of the fator leel. Firt we mut ae the auray of the predition baed on the ro alidation error, whih i quantified imply a relatie frequenie of errornou la predition, either in total or detailed on the lae: # Ae the auray of the predition # perent orret for eah ategory: t <- table(iri_train$sp, lda_train_loo$la) diag(prop.table(t, 1)) # total perent orret um(diag(prop.table(t))) [1] So the oeral CV baed error rate i = 2.7%. Som nie plotting only work without the CV-tuff uing the klar-pakage: library(klar) partimat(sp ~ Sepal.L. + Sepal.W. + Petal.L. + Petal.W., data = Iri_train, method = "lda", prior=(1, 1, 1)/3)

6 enote EXAMPLE: IRIS DATA Sepal.W. Sepal.L. app. error rate: Petal.L. Sepal.L. app. error rate: Petal.L. Sepal.W. app. error rate: Petal.W. Sepal.L. app. error rate: Petal.W. Sepal.W. app. error rate: Petal.W. Petal.L. app. error rate: 0.04 Partition Plot Quadrati Diriminant Analyi, QDA It goe ery muh like aboe: # PART 2: QDA # Mot tuff from LDA an be reued: qda_train_loo <- qda(sp ~ Sepal.L. + Sepal.W. + Petal.L. + Petal.W., Iri_train, prior = (1, 1, 1)/3, CV = TRUE) # Ae the auray of the predition # perent orret for eah ategory: t <- table(iri_train$sp, qda_train_loo$la) t

7 enote EXAMPLE: IRIS DATA diag(prop.table(t, 1)) # total perent orret um(diag(prop.table(t))) [1] 0.96 For thi example the QDA perform lightly wore than the LDA. partimat(sp ~ Sepal.L. + Sepal.W. + Petal.L. + Petal.W., data = Iri_train, method = "qda", prior = (1, 1, 1)/3)

8 enote EXAMPLE: IRIS DATA Sepal.W. Sepal.L. app. error rate: Petal.L. Sepal.L. app. error rate: Petal.L. Sepal.W. app. error rate: Petal.W. Sepal.L. app. error rate: Petal.W. Sepal.W. app. error rate: Petal.W. Petal.L. app. error rate: 0.04 Partition Plot Prediting NEW data # And now prediting NEW data: predit(qda_train, Iri_tet)$la Error in predit(qda train, Iri tet): objet qda train not found # Find onfuion table: t <- table(iri_tet$sp, predit(qda_train, Iri_tet)$la) Error in predit(qda train, Iri tet): objet qda train not found t

9 enote EXAMPLE: IRIS DATA diag(prop.table(t, 1)) # total perent orret um(diag(prop.table(t))) [1] Baye method We an fit a denity to the obered data and ue that intead of the normal ditribution impliitly ued in LDA: # Part 3: (Naie) Baye method # With "uekernel=true" it fit a denity to the obered data and ue that # intead of the normal ditribution # Let explore the reult firt: # (Can take a while dur to the fitting of denitie) partimat(sp ~ Sepal.L. + Sepal.W. + Petal.L. + Petal.W., data = Iri_train, method = "naiebaye", prior=(1, 1, 1)/3, uekernel = TRUE)

10 enote EXAMPLE: IRIS DATA Sepal.W. Sepal.L. app. error rate: Petal.L. Sepal.L. app. error rate: Petal.L. Sepal.W. app. error rate: Petal.W. Sepal.L. app. error rate: Petal.W. Sepal.W. app. error rate: Petal.W. Petal.L. app. error rate: 0.04 Partition Plot baye_fit <- upprewarning(naiebaye(sp ~ Sepal.L. + Sepal.W. + Petal.L. + Petal.W., data = Iri_train, method = "naiebaye", prior = (1, 1, 1)/3, uekernel = TRUE)) par(mfrow = (2, 2)) plot(baye_fit)

11 enote EXAMPLE: IRIS DATA 11 Naie Baye Plot Naie Baye Plot Denity Denity Denity Sepal.L. Naie Baye Plot Denity Sepal.W. Naie Baye Plot Petal.L. Petal.W. # And now prediting NEW data: bayepred <- predit(baye_fit, Iri_tet)$la # Find onfuion table: t <- table(iri_tet$sp, bayepred) t bayepred diag(prop.table(t, 1))

12 enote EXAMPLE: IRIS DATA 12 # total perent orret um(diag(prop.table(t))) [1] k-nearet neighbourgh # PART 4: k-nearet neighbourgh: # Exploratie plot WARNING: THIS may take ome time to produe!! partimat(sp ~ Sepal.L. + Sepal.W. + Petal.L. + Petal.W., data = Iri[train,], method = "knn", kn = 3) Sepal.W. Sepal.L. app. error rate: Petal.L. Sepal.L. app. error rate: Petal.L. Sepal.W. app. error rate: Petal.W. Sepal.L. app. error rate: Petal.W. Sepal.W. app. error rate: Petal.W. Petal.L. app. error rate: 0.04 Partition Plot

13 enote EXAMPLE: IRIS DATA 13 knn_fit_5 <- knn(sp ~ Sepal.L. + Sepal.W. + Petal.L. + Petal.W., data = Iri_train, method = "knn", kn = 5) # And now prediting NEW data: knn_5_pred <- predit(knn_fit_5, Iri_tet)$la # Find onfuion table: t <- table(iri_tet$sp, knn_5_pred) t knn_5_pred diag(prop.table(t, 1)) # total perent orret um(diag(prop.table(t))) [1] PLS-DA # PART 5: PLS-DA # We hae to ue the "uual" PLS-funtion # Define the repone etor (2 lae) OR matrix (>2) lae:

14 enote EXAMPLE: IRIS DATA 14 # Let try with K=2: Group 1:, Group -1: and Iri_train$Y <- -1 Iri_train$Y[Iri_train$Sp == ""] <- 1 table(iri_train$y) Iri_train$X <- a.matrix(iri_train[, 1:4]) # From here ue tandard PLS1 predition, e.g.: library(pl) # Pl: mod_pl <- plr(y ~X, nomp = 4, data =Iri_train, alidation = "LOO", ale = TRUE, jakknife = TRUE) # Initial et of plot: par(mfrow = (2, 2)) plot(mod_pl, label = rowname(iri_train), whih = "alidation") plot(mod_pl, "alidation", etimate = ("train", "CV"), legendpo = "topright") plot(mod_pl, "alidation", etimate = ("train", "CV"), al.type = "R2", legendpo = "bottomright") pl::oreplot(mod_pl, label = Iri_train$Sp)

15 enote EXAMPLE: IRIS DATA 15 predited Y, 4 omp, alidation RMSEP Y train CV R meaured Y train CV Comp 2 (25 %) number of omponent number of omponent Comp 1 (71 %) Be arefull about interpretation due to the binary etting You hould do a CV baed onfuion table for eah omponent, really: pred <- array(dim = (length(iri_train[, 1]), 4)) for (i in 1:4) pred[, i] <- predit(mod_pl, nomp = i) pred[pred<0] <- -1 pred[pred>0] <- 1 # Look at the reult from eah of the omponent: for (i in 1:4) { t <- table(iri_train$y, pred[,1]) CV_error <- 1-um(diag(prop.table(t))) print(cv_error) } [1] 0

16 enote EXAMPLE: IRIS DATA 16 [1] 0 [1] 0 [1] 0 The predition of new data would be handled imilarly (not hown). # WHAT if in thi ae K=3 lae: # Look at Varmuza, Se K=3 Iri_train$Y=matrix(rep(1,length(Iri_train[,1])*K),nol=K) Iri_train$Y[Iri_train$Sp!="",1]=-1 Iri_train$Y[Iri_train$Sp!="",2]=-1 Iri_train$Y[Iri_train$Sp!="",3]=-1 mod_pl <- plr(y ~ X, nomp = 4, data = Iri_train, alidation="loo", ale = TRUE, jakknife = TRUE) # Initial et of plot: par(mfrow=(1, 1)) pl::oreplot(mod_pl, label = Iri_train$Sp)

17 enote EXAMPLE: IRIS DATA Comp 1 (72 %) Comp 2 (25 %) plot(mod_pl, label = rowname(iri_train), whih = "alidation")

18 enote EXAMPLE: IRIS DATA 18 Y1, 4 omp, alidation Y2, 4 omp, alidation Y3, 4 omp, alidation predited meaured plot(mod_pl, "alidation", etimate = ("train", "CV"), legendpo = "topright")

19 enote EXAMPLE: IRIS DATA 19 Y1 Y2 Y3 train CV train CV train CV RMSEP number of omponent plot(mod_pl, "alidation", etimate = ("train", "CV"), al.type = "R2", legendpo = "bottomright")

20 enote EXAMPLE: IRIS DATA 20 Y1 Y2 Y3 R train CV train CV train CV number of omponent # Predition from PLS need to be tranformed to atual laifiation: # Selet the larget one: pred <- array(dim = (length(iri_train[, 1]), 4)) for (i in 1:4) pred[,i]<- apply(predit(mod_pl, nomp = i), 1, whih.max) pred2 <- array(dim = (length(iri_train[, 1]), 4)) pred2[pred==1] <- "" pred2[pred==2] <- "" pred2[pred==3] <- "" # Look at the reult from eah of the omponent: for (i in 1:4) { t <- table(iri_train$sp, pred2[, i]) CV_error <- 1 - um(diag(prop.table(t))) print(cv_error) }

21 enote EXAMPLE: IRIS DATA 21 [1] 0.96 [1] 0.24 [1] 0.2 [1] Random foret Wellome to example of randomforet and foretfloor. Random foret i one of many deiion tree enemble method, whih an be ued for non-linear uperied learning. In R, the tandard ran pakage i randomforet and the main funtion i alo alled randomforet. Randomforet an be een a a blak-box algorithm whih output a model-fit(the foret) when inputted a training data et and a et of paremter. The foret, an take an input of feature and output predition. Eah diiion tree in the foret will make a predition, and the ote by eah tree will be ombined in an enemble. The predited repone i either a number/alar (regreion model, mean of eah tree) or one label out of K lae (laifiation, majority ote) or a etor of K length with peudo probability of any K la memberhip(laifiation, pluraliti ote). library(randomforet) Warning: pakke randomforet ble bygget under R erion randomforet Type rfnew() to ee new feature/hange/bug fixe. #a imple dataet of normal ditributed feature ob = 2000 ar = 6 X = data.frame(repliate(ar,rnorm(ob,mean=0,d=1))) #the target/repone/y depend of X by ome unknown hidden funtion hidden.funtion = funtion(x,snr=4) { yignal <<- with(x,.5 * X1^2 + in(x2*pi) + X3 * X4) #y = x1^2 + in(x2) + x3 * x4 ynoie = rnorm(dim(x)[1],mean=0,d=d(yignal)/snr) at("atual ignal to noie ratio, SNR: \n", round(d(yignal)/d(ynoie),2)) y = yignal + ynoie

22 enote EXAMPLE: IRIS DATA } y = hidden.funtion(x) atual ignal to noie ratio, SNR: 4.06 #atter plot of X y relation plot(data.frame(y,x[,1:4]),ol="# ") y 2 0 X X2 2 4 X3 0 X #no noteworthy linear relationhip print(or(x,y)) [,1] X X

23 enote EXAMPLE: IRIS DATA 23 X X X X #RF i the foret_objet, randomforet i the algorithm, X & y the training data RF = randomforet(x=x,y=y, importane=true, #extra diagnoti keep.inbag=true, #trak of boottrap proe ntree=500, #how many tree replae=false) #boottrap with replaement? print(rf) Call: randomforet(x = X, y = y, ntree = 500, replae = FALSE, importane = TRUE, Type of random foret: regreion Number of tree: 500 No. of ariable tried at eah plit: 2 keep Mean of quared reidual: % Var explained: The modelfit print ome information. Thi i regreion random foret, 500 tree were grown and in eah plit 2 ariable we tried (mtry=ar/3). Next the model automatially doe out-of-bag ro-alidation (OOB-CV). The reult obtained from OOB-CV i imilar to 5-fold ro alidation. But to perform an atual 5-fold-CV, 5 random foret mut be trained. That would ompute 5 time lower. The RF objet i a lit of la randomforet, whih ontain the foret and alot of pre-made diagnoti. Let try undertand the mot dentral tatiti. #RF predited i the OOB-CV predition par(mfrow=(1,2)) plot(rf$predited,y, ol="# ", main="predition plot", xlab="predited repone", ylab="atual repone") abline(lm(y~yhat,data.frame(y=rf$y,yhat=rf$predited)))

24 enote EXAMPLE: IRIS DATA 24 #thi performane mut be een in the light of the ignal-to-noie-ratio, SNR plot(y,yignal, ol="# ", main="ignal and noie plot", xlab="repone + noie", ylab="noie-le repone") predition plot ignal and noie plot atual repone noie le repone predited repone repone + noie at("explained ariane, (peudo R 2 ) = 1- SSmodel/SStotal = \n", round(1-mean((rf$pred-y)^2)/ar(y),3)) explained ariane, (peudo R 2 ) = 1- SSmodel/SStotal = at("model orrelation, (Pearon R 2 ) = ",round(or(rf$pred,y)^2,2)) model orrelation, (Pearon R 2 ) = 0.69

25 enote EXAMPLE: IRIS DATA 25 at("ignal orrelation, (Pearon R 2 ) = ",round(or(y,yignal)^2,2)) ignal orrelation, (Pearon R 2 ) = 0.94 The random foret will neer make a perfet fit. But if the number ample and tree goe toward a uffiient large number, the model fit will in pratie be good enough. If the true model funtion ha a omplex urature, ignal-to-noie leel i low and relatiely few ample i ued for training (e.g. 100), the model fit will be highly biaed (omplex urature i inhibited) in order to retain ome tability/robutne. RF$rq ontain a etor of ntree element. Eah element i the OOB-CV (peudo R 2 ) if only thi many tree had been grown. Often it i enough to grow 50 tree. 10 Tree i rarely good enough. In theory predition performane will inreae aymtoti to a maximum leel. In pratie predition performane an dereae, but uh inident are mall, random and annot be reprodued. More tree i neer wore, only wate of time. par(mfrow=(1,1)) plot(rf$rq,type="l",log="x", ylab="predition performane, peudo R 2 ", xlab="n tree ued")

26 enote EXAMPLE: IRIS DATA 26 predition performane, peudo R² n tree ued The RF objet ontain a matrix alled importane(rf$importane). Firt olumn (when importane=true) i ariable permutation importane (VI). VI i a alue for eah ariable. It i the hange of OOB-CV mean quare error% if thi ariable wa permuted(huffled) after the model wa trained. Permuting a ariable make it random and with no ueful information. If the model deteriorated the mot by permuting a gien ariable, thi ariable i the mot important ariable. Thu without trying to undertand the model itelf, it i poible to ruin the ariable and meaure how muh CV performane deteriorate. Sometime i the purpoe of random foret modeling not the predition itelf, but to obtain the ariable importane. E.g. for eleting gene aoiated with aner, one an ue ariable importane to hooe important gene for further examiniation. ##ariable importane RF$importane %InMSE InNodePurity X e X e

27 enote EXAMPLE: IRIS DATA 27 X e X e X e X e #hort ut to plot ariable importane arimpplot(rf) RF X1 X2 X2 X1 X4 X3 X3 X4 X6 X6 X5 X %InMSE InNodePurity Notie ariable 5 and 6 are rather unimportant. Gini-importane, the eond importane term, i ompletely inferrior, and hould not be ued. The RF objet an be ued to make new predition. #let try to predit a tet et of 800 ample X2 = data.frame(repliate(ar,rnorm(800))) true.y = hidden.funtion(x2)

28 enote EXAMPLE: IRIS DATA 28 atual ignal to noie ratio, SNR: 4.12 pred.y = predit(rf,x2) par(mfrow=(1,1)) plot(pred.y,true.y, main= "tet-et predition", xlab= "predit repone", ylab= "true repone", ol=rgb(0,0,ab(pred.y/max(pred.y)),alpha=.9)) abline(lm(true.y~pred.y,data.frame(true.y,pred.y))) tet et predition true repone predit repone

29 enote EXAMPLE: IRIS DATA foretfloor Random foret i regularly only ued to either ariable eletion with ariable importane or to build a blak-box foreat mahine. The pakage foretfloor an be ued to explore and iulize the urature of a RF model-fit. The urature of multiariate model are high dimenional and annot be plotted properly. Multi linear regreion i fairly eay a it only omprie om oeffiient deribing a hyper-plane. For non-linear model, the uer often hae not the lightet idea of the atual urature of the model. -If a non-linear model predit a gien problem well, it urature might inpire and elaborate the ientit undertanding of the problem. #get pakge from rforge. Maybe additional pakage uh trimtree, mlbenh, rgl and #mut be intalled firt. #intall.pakage("foretfloor", repo=" library(foretfloor, warn.onflit=false, quietly = TRUE) #the funtion foretfloor output a foretfloor-objet, here alled ff ff = foretfloor(rf,x) print(ff) thi i a foretfloor( x ) objet thi objet an be plotted in 2D with plot(x), ee help(plot.foretfloor) thi objet an be plotted in 3D with how3d(x), ee help(how3d) x ontain following internal element: FCmatrix imp_ind importane X Y #ff an be plotted in different way. A our data hae ix dimenion we annot #plot eerything at one. Let tart with one-way foretfloor plot plot(ff)

30 enote EXAMPLE: IRIS DATA X X X X X X5 We an ee that the ontribution from ariable X1 and X2 to the repone look a lot like X 2 and in(x). The y-axi i alled feature ontribution(f). Eah ample ha one feature ontribution for eah ariable, thu eah ample i omewhere in eah figure. The x-axi i the alue of any ample by a gien ariable/feature. If ample i, at one time wa plaed in a node with repone mean of 1.0 and wa plit by ariable j, and ent to a new node of repone mean -4.2, then ample i experiened a loal inrement of -5.2 by ariable j. A feature ontribution i the um of tep (loal inrement) one ample took by a gien ariable through the foret diided by number of tree. Feature ontribution are tored in matrix, alled F or FCmatrix, of ame ize a X. Thu one feature ontribution for eah ombination of ariable and ample. F denote a OOB- CV erion feature ontribution only ummed oer thoe tree where the i th ample i OOB dieded with how may time i th ample wa OOB. F i more robut than F. The following equation applie, ỹ i = j j=i F ij + y i Thi mean that any gien OOB-CV predition ỹ i i equal to the um of it feature on-

31 enote EXAMPLE: IRIS DATA 31 tribution (one for eah ariable in the model) plu the i it boottrap mean. The boot trap mean i for any pratial onern ery ery loe to grand mean, mean(y). It deiate lightly from grand mean a eah tree tart out with lightly different boottrap mean and that eah ample i OOB in different tree. The equation aboe tate in pratie, that any predition i equal to the um of the tep taking in the proe plu the tarting point, the grand mean. #following line demontrate the aboe equation #thi part an be kipped... #thi i the tarting predition of eah tree tree.boottrap.mean = apply(rf$inbag,2,funtion(x) um(x*y/um(x))) #thi i the indiidual OOB boottrap.mean for any oberation y.boottrap.mean = apply(rf$inbag,1,funtion(x) { um((x==0) * tree.boottrap.mean)/um(x==0) }) #OOB predition = FC.rowum + boottrap.mean yhat = apply(ff$fcmatrix,1,um) + y.boottrap.mean #yhat and RF predited are almot the ame, only limited by double preiion. ##plot(yhat-rf predited) #try plot thi Next ariable 3 and 4 do not look well explained in the one-way foretfloor plot. A x3 and x4 interat, neither ariable alone an explain it influene on the predited repone. We an ue a olour-gradient to add another dimenion to the plot. Firt we need to learn the funtion fol(). #fol output a olouretor a funtion of a matrix or foretfloor objet #firt input, what to olour by, 2nd input, what olumn to ue par(mfrow=(2,2)) pair(x[,1:3],ol=fol(x,ol=2),main="ingle ariable gradient, \n X2")

32 enote EXAMPLE: IRIS DATA ingle ariable gradient, X X1 X X2 0 pair(x[,1:3],ol=fol(x,ol=(1:2)),main="double ariable gradient, \n X1 & X2") double ariable gradient, X1 & X X1 X X2 0 pair(x[,1:3],ol=fol(x,ol=(1:3)),main="triple ariable gradient, \n X1 & X2 & X3"

33 enote EXAMPLE: IRIS DATA triple ariable gradient, X1 & X2 & X X1 X X2 0 pair(x[,1:5],ol=fol(x,ol=(1:5)),main="multi ariable gradient, \n PCA of X1...X5 multi ariable gradient, PCA of X1...X5 3 2 X1 X2 4 X3 X5 3 4 X4 2 #the latter gradient ue PCA to hoe three linear orthogonal gradient, # whih bet an ummarize the high dimenional ariane of X. #A ariable of X i unorrelated, thi i not eay. For ollinear data, # multiariable gradient work great.

34 enote EXAMPLE: IRIS DATA 34 #print fol output olour defined a RGB+alpha(tranpareny) Col = fol(x,ol=(1:3)) print(col[1:5]) [1] "#B " "#0683AA33" "#1FBB5B33" "#B62D1933" "#C4FD3233" #firt two hexidemimal deribe red, then green, then blue, then alpha. # two hexideimal i equal to a 10bae number from 1 to 256 #now we apply a olor gradient along ariable x3 Col = fol(ff,3,order=f) plot(ff,ol=col) X2 X1 X4 partial.ontribution partial.ontribution partial.ontribution phyial.alue X phyial.alue X phyial.alue X5 partial.ontribution partial.ontribution partial.ontribution #one ample will hae ame olour in eery plot #ample with imilar olor hae imilar x3-alue We an ee that a horizontal olour gradient emerge in plot X3, a we hoe to olor by thi axi. Other plot eem randomly olored beaued the arible are unorrela-

35 enote EXERCISES 35 ted and do not interat within the model. But for ariable X4, the ame olur gradient emerge horiontal. Two undertand the olour pattern, it i eaier to ue a 3D-plot how3d_new(ff,3:4,ol=col) #one aweome 3D plot in a external window!! Thu now it i iulized that one-way foretfloor plot X3 and X4 imply are 2D projetion of af 3D addle point urature 9.3 Exerie Exerie 1 Exerie: Wine data Ue the wine data preiouly ued. a) Predit the wine-la uing the different method uggeted. b) Try alo a PCA baed erion of ome of the method. ) What are the reult? Make your own tet- and training data. Apply CV on the training data and predit the tet data. d) By looking at ariou plot (in addition to predition ability) onider whih model eem to be motly appropriate for the data.

36 enote EXERCISES 36 Exerie 2 Exerie: random foret protate train a model to predit protate aner ue ariable importane to identify gene related a) How well an protate aner be explained from gene expreion leel? b) What gene ode for protate aner? ) Thi model ha 100 ample and 6000 ariable. What doe the urature of a ery pare random foret model look like? Exerie 3 Exerie: Random foret exerie 2 - abelone a) How well an a random foret model predit the age of abelone b) What are the main effet (main trend) of the model? (ue one-way foretfloor) ) Are the any large interation beide main-effet and what ariable are impliated? (ue one way foretfloor + fol) d) Deribe the two-way foretfloor plot between hell.weight and hruked.weight: Why would the model ue uh a urature? (ue how3d new and fol)