STAC42 Multivariate Analysis Assignment 4

Transcription

1 STAC42 Multivariate Analysis Assignment 4 Due Thursday April 1st in class. 1. Two groups of students (of psychology and biology) were each given a list of 25 things or concepts. Each student was asked to rate each pair of items as similar or dissimilar. For each group, a dissimilarity matrix was obtained for the 25 items. SAS code for reading in the data and constructing a suitable SAS data set is available on the course web site under Things data. (a) Describe in words how the SAS data set is constructed. First the names of the concepts are read in. These are entered as variable item in a data set called Concepts. (The syntax $1-12 says to read in the first 12 characters on each line, regardless of spaces etc.) Then the dissimilarities for the psychology students are read in to a dataset Group1 ; next, the dissimilarities for the biology students are read in to a dataset Group2. Finally, everything is combined together into a dataset called Both, containing both matrices, the category labels, and an indicator showing which group is which. (b) Use individual differences scaling to produce a two-dimensional map of the concepts. How would you describe the two dimensions? Code like the following will work. You want to keep the names of the concepts (so that they will appear on the plot). proc mds model=indscal out=results; matrix group; id item; proc plot; plot dim2*dim1 $ item; Note the use of id. My plot is shown in Figure 1. Dimension 1 (horizontal scale) seems to have mammalrelated things on the right, and bird and flower-related things on the left. Dimension 2 has animals at the top and (mostly) plant-related things at the bottom. (c) By looking at the coordinates for the two groups, can you describe how the groups differ in terms of how they rate the items as similar or dissimilar? Print out the output data set and pick them off the bottom of the list: OBS GROUP _TYPE_ ITEM DIM1 DIM2 27 Psych DIAGCOEF Biolo DIAGCOEF Both groups have values close to 1 on the two dimensions, suggesting that the groups are very similar. The psychology students weight dimension 1 a little higher, and the biology students weight dimension 2 a little higher. 2. The International Foods dataset is described on page 263 of the text. In each of 16 countries a survey was carried out to estimate the percentage of households in which each of 20 types of food was present. (a) Figure out how to read this dataset into SAS, using the column 1

2 2 + > Chicken Frog < > Blood > Feathers > Bats 1 + >>> Dog Robin < > Bird D i > Animal m > Mammal e > Red n > Hairs s 0 + > Color i Living Thing < > Deer o n > Green 2 Hooves < > Tree > Antler -1 + > Rose Flower < > Plant > Leaves > Daisy > Cottonwood Figure 1: Individual differences scaling plot for question 1 names as the names of the variables. (Bear in mind that each row begins with the code for a country.) Paste the data file into your.sas file and edit it to look like this: data foods; input country $ GC IC TB SS BP SP ST IP FF VF AF OF FT JS CG BR ME OO YT CD; datalines; WG IT FR (b) Carry out a cluster analysis of countries, using Ward s method and get a tree diagram (arranging the output so that the tree fits on the paper and making sure that the country codes appear on the tree). I used the following code. The tree seemed to look better vertical: proc cluster method=ward out=clus; id country; proc tree; The tree itself is in Figure 2. (c) What is a sensible number of clusters for these data? how you made your choice. Explain The clusters don t change over the range 0.08 to 0.14 (of semipartial R-squared ), which is the widest range of non-change. So those 4 clusters look best: cluster 1 has Germany, Belgium, France, Switzerland, Luxembourg, Netherlands; cluster 2 has Britain and Ireland; cluster 3 has Sweden, Denmark, Norway and Finland; cluster 4 has Italy, Spain, Portugal, Austria. These clusters seem to correspond well to both geography and culture (apart perhaps from Austria). (d) Re-do your analysis using a different hierarchical clustering method (it does not matter which one). Compare the results from the two methods what is the biggest difference you see? I used single linkage as my other method. The tree (in Figure 3) is very different; indeed, it s very difficult to 2

3 W B F S L N G I S D N F I S P A G M R D G S B D W K Y D T P L A S XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX e XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXX m 0.2 +XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXX i XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXX - XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXX P XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXX a XXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXX XXXXXXXXXX r XXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXX XXXXXXXXXX t XXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXX XXXXXXXXXX i XXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXX XXXXXXXXXX a XXXXXXXXXXXXXXXX XXXX XXXXXXXXXX XXXXXXXXXX l XXXXXXXXXXXXXXXX XXXX XXXXXXXXXX XXXXXXXXXX XXXXXXXXXXXXXXXX XXXX XXXXXXXXXX XXXXXXXXXX R XXXXXXXXXXXXXXXX XXXX XXXXXXXXXX XXXXXXXXXX XXXXXXXXXXXXXXXX XXXX XXXXXXXXXX XXXXXXXXXX S XXXXXXXXXXXXXXXX XXXX XXXXXXXXXX XXXXXXXXXX q XXXXXXXXXXXXXXXX XXXX XXXXXXXXXX XXXXXXXXXX u XXXXXXXXXXXXXXXX XXXX XXXXXXXXXX XXXXXXXXXX a XXXXXXXXXXXXX. XXXX. XXXXXXX XXXXXXXXXX r XXXXXXXXXXXXX. XXXX. XXXXXXX XXXXXXXXXX e XXXXXXXXXXXXX. XXXX. XXXXXXX XXXXXXXXXX d XXXX XXXXXXX.... XXXXXXX XXXXXXXXXX XXXX XXXX..... XXXXXXX XXXX XXXX XXXX XXXX..... XXXXXXX XXXX.. XXXX XXXX XXXX XXXX W B D N F S F I S P A N L G S I G M K Y D D R T P L A S G B W D M 1 + i n XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX i XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX. m XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.. u XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX... m 0.8 +XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX... XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.... D XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..... i XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..... s XXXXXXXXXXXXXXXXXXX XXXXXXX t XXXXXXXXXXXXXXXXXXX XXXX a 0.6 +XXXXXXXXXXXXXXXX. XXXX n XXXXXXXXXXXXX.. XXXX c XXXX XXXXXXX.. XXXX e XXXX. XXXX.. XXXX XXXX. XXXX B... XXXX e XXXX t w e e n C l u s t e Figure 2: Tree diagram for Ward s method, question 2 Figure 3: Question 2 single linkage 3

4 decide from this picture how many clusters we might use. This might be the nature of single-linkage: the clusters found here may be long and skinny, so that each country joins onto the end of one of the previous clusters instead of forming a new cluster. (If you used something other than single linkage, you might have gotten a clearer picture.) 3. Figure out how to read in the data of the previous question so that you can do a cluster analysis of foods. You might find PROC TRANSPOSE helpful for this. This sort of code is all you need: proc transpose out=trans; proc cluster method=ward out=clus; id _name_; The food codes got turned into a variable called _name_, so that s what needs to go onto the tree. When you have the data read in, use Ward s method to carry out a cluster analysis of the foods. How many clusters do you get, and which foods are in each? From the output in Figure 4, it seems clear that there are 2 clusters of foods: Ground coffee, margarine, tea, butter, biscuits (cookies), apples, oranges, instant coffee, canned fruit, packaged soup, jam, garlic, olive oil Sugarless sweet, frozen vegetables, frozen fish, instant potatoes, canned soup, yogurt, crispbread. It s hard to judge what the items in each cluster have in common; it might be worth looking back at the data. For instance, canned soup is often found in northern European countries, but rarely in southern European countries. G M T B B A O I F S J C O S V F I S Y C C E B R P F F C T P S G O S F F P T T D XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX S XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX e XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX m 0.4 +XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX i XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX - XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX P XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX a XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX r XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX t 0.3 +XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX i XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX a XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX l XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX R XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX S XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX q XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX u XXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXX a XXXXXXXXXXXXX XXXXXXXXXXX XXXXXXXXXXXXX r XXXXXXXXXXXXX XXXXXXXXXXX XXXXXXXXXXXXX e d 0.1 +XXXXXXXXXXXXX XXXXXXXXXXX XXXXXXXXXXXXX XXXXXXXXXXXXX XXXXXXX XXX XXXXXXXXXXXXX XXXXXXXXXXXXX XXXXXXX XXX XXXXXXXXXXXXX XXXXXXXXXXXXX XXXXXXX XXX XXXXXXXXXXXXX XXXXXXXXXXXXX XXXXXXX XXX XXXXXXXXXXXXX.. XXX XXXXX XXXXXXX.. XXXXXXXXXXX XXX XXXXXXX... Figure 4: Question 3: Ward s method 4

5 4. Exercise 12.7, page 470. Part (a) asks you to do a hypothesis test (do this in SAS); part (b) asks you to run a two-group discriminant analysis (assume that the two groups have equal var/cov matrices) and to obtain a hit/miss table. My code looks like this. Note that I want to give my variables meaningful names, and I also want to exclude id. proc discrim manova; var useful content nostart dread; class group; The MANOVA table shows that the two groups do indeed differ significantly: Statistic Value F Num DF Den DF Pr > F Wilks Lambda Pillai s Trace Hotelling-Lawley Trace Roy s Greatest Root so that the discriminant analysis is worthwhile. The hit/miss table is as shown: Number of Observations and Percent Classified into GROUP: From GROUP 1 2 Total Total Percent This is pretty good: of the 15 depressed patients, 14 are correctly classified as depressed, while 25 of the 30 healthy patients are correctly classified as healthy. (If you look at the data, you ll find that the depressed patients almost all answered 1 and 2 to the questions, while the healthy patients had some 3 s and 4 s.) 5