2. Getting started with MLwiN

Transcription

1 2. Getting started with MLwiN Introduction This chapter aims to provide you with some practice with MLwiN commands before you begin to fit multilevel models. It is may be helpful if you have already read the previous chapter and you appreciated some of the design concepts, particularly the worksheet, which is covered there. We are going to cover some of the basic procedures that are needed before you begin modeling. In particular, we will show you how to: input data to the program; save and retrieve worksheets; sort data by higher level units; transform data and calculate new columns; describe and cross-tabulate data; produce histograms and scatterplots; handle categorical variables in tables and plots. Throughout we will be using a single running example of house-price data. This chapter is best followed by working hands-on at the computer, using the GUI, as we proceed. Use the help option at any time.. We assume that you have a working knowledge of Microsoft Windows as the program shares many features common to other applications such as word processors and some statistical packages. Thus, file opening and saving is standard, as is the copying and pasting. The opening screen On entry to the program you will see a rather unappealing grey screen. At the top, highlighted in blue to show that it is the active window is the MLwiN icon, and the usual minimize, re-size and close icons. Below this is a row of headings running from File to Help; this is the main menu. Below this are some buttons START, MORE, STOP which control the estimation process. In this session we want to concentrate on getting to know what can be achieved by the features listed on the main menu. In particular, if you place your cursor over the following headings, you will get some appreciation of what can be achieved. Three headings give standard Windows operations: EDIT provides the usual facilities for Cut, Copy and Paste 17

2 WINDOW allows windows to cascade, and the closure of all windows HELP gives access to the help system (described briefly in Chapter 1) We are going to use some of the facilities under the following headings: FILE DATA MANIPULATION BASIC STATISTICS GRAPHS importing and exporting data; saving and retrieving files calculation, sorting of data averages, cross tabulations, correlations customize graphs for histograms and scatterplots; trellis graphs when a categorical variable is involved. OPTIONS controls the size of the worksheet, the amount of decimal points to be displayed, and the location of directories for acquiring macros There are two headings we are not going to look at all in this chapter, these are, MODEL ESTIMATION facilitates the specification of the multilevel models; the next chapter will focus on these commands controls which estimation process is used. Data Input: copying data from an Excel spreadsheet In an Excel spreadsheet enter the following 20 values with their headings (this page and over) House District Price Size Type Det Terr Semi Terr Det Semi Terr Semi Terr Terr Terr Terr Semi 18

3 Semi Terr Semi Semi Semi Terr Semi These values have been taken from a larger file, which we will be using later. As is common, the data are a mixture of categorical (ie alphabetical) and numerical values. Highlight all five columns and all values and copy them (CTR-c should work) 1 Switch across into MLwiN and on the Edit button on the Main Menu choose Paste, this should bring up the dialog box that follows: 1 For this procedure to work properly, the data should be organized in rows and columns of equal length, with each column separated or delimited by white space or tabs. 19

4 Tick the box for Use first row as names Tick the box for Free columns This simply means that the values 134, 8, 2 etc are ready to be pasted into a previously free column c1 and we are going to name this variable House. The values 7,1,1 etc are going to be put into column 2 and this variable is going to be called District and so on. You will see at the top of the screen that you can set a particular value (eg -9.9 etc) to be missing. If you have missing data it is a good idea to use the same missing value code for all variables. 2 If everything is correct, click on the Paste button at the bottom of the screen. The Paste View window should close to be replaced by the Names window 2 By default, a value of System Unknown (a very large negative number, E+29) is in force. There is much more on importing data into MLwiN at 20

5 If you are working on a networked computer you may hit a problem as during the pasting process, MLwiN will need to write a small file and the current directory may be a read only one. If you encounter this problem, proceed as follows: Options on Main menu Directories Browse to get My documents 3 as the current directory as follows Let us take another look at the Names worksheet (it is a good idea to keep this window always open as it gives the structure of the worksheet). The bottom of the main window gives you a Tab for each open window; clicking on the relevant tab will make that window active. If you have closed the Names window, then Data manipulation on main menu Click on Names To give again the following summary of the worksheet 3 Or some other directory to which you have permission to write. 21

6 There are 5 columns, each with 20 observations. Notice that you are provided for each named column a minimum and maximum, a count of the total number of missing vales; and that the text variable Type is automatically given a categorical coding indicated by the status True. Across the top of the Names window, you will see the following functionality which all works by clicking on a column or groups of columns Edit name Data Toggle Categorical Categories Description Copy, Paste, Delete this edits the name of a column; at the outset, columns are named c1, c2 etc. Consequently, it is good practice to edit these and replace the name with something memorable as you soon develop a lot of columns. allows you to see the values in the column, we will do this soon. A toggle is a command that is like an on-off switch; if you highlight a column and it is categorical (ie True) it changes to numeric (ie False); if it is already numeric(ie False), the column will become categorical (True). This allows you to set the labels of a categorical variable. You can add a long distinctive Description to each column to fully document your worksheet. Not surprisingly these commands allow you to Copy from a column(s), Paste to a column(s), and Delete a column(s). 4 There are two other aspects of functionality Help This gives help on the Names window; it is currently out of date and refers to an old version of the software. A tickbox for Used columns Ticking on this box results in the display only of columns with values them; this can be useful if you have stored data in high columns; eg the estimates are automatically stored in c1096-c1099; see Chapter 1. To view the data: In the Names window Highlight House to Type until all 5 variables turn dark blue Click on Data in the Names Menu This will bring up the following data 5 4 If anything goes wrong and the incorrect data has been pasted, highlight columns c1-c5 and click Delete on the Names window, and then re-paste having copied the correct data from Excel 5 You could also have used Data Manipulation on Main Menu View or Edit the data Click on View and highlight the columns you wish to see Drag the right-hand window to see all the five columns; 22

7 This shows the actual values for all 20 observations. As the Show value Labels box is ticked, you will see text for categories. You can use the slider to move up and down the observations. It is clear from this display that the data are not sorted by district by house. Having got the data into MLwiN, is is a good idea to save the worksheet. Appropriate sorting must be completed before modeling. Saving and retrieving the worksheet To save the worksheet File on the Main Menu Save worksheet as C:\Training manual\house.wsz That *.wsz extension is as a MLwiN 2.11 compressed worksheet. 6 Of course, the exact path and filename depend on your preferences; but make a note where you have stored it and the name you have used To retrieve a previously saved worksheet: 7 6 Using this extension means that the worksheet is zipped as it is stored, making a huge reduction in storage, typically over 95% on a non-compressed file. The worksheet can also be stored in Minitab, SPSS and Stata format; see the Manual Supplement; section 6 7 It is also possible to read data files in Minitab, SPSS and Stata format, but we strongly urge you to reduce the number of columns to those that you are going use in the modeling, before importing the data. 23

8 File on the Main Menu Open worksheet C:\Training manual\house.wsz Data manipulation Data manipulation window under the main menu offers a range of options and we are going to explore some of them in this chapter. The data must be sorted appropriately before the data can be modeled correctly. House within the same district must be next to each other in the worksheet. The sequence for sorting is as follows: Data Manipulation on Main Menu Sort Increase number of keys to 2 [because houses nested in ditricts] Choose District as the highest key [slowest changing] Choose House as the lowest key [fastest changing] Highlight House to Type (ie all 5 columns) Click on Same as Input Click on Add to Action List Click on Execute The graphic below shows the Sort window just before the execution button is pressed. It is worth spending a little time on this window as it has a number of design features that are common to several data manipulation windows. The window is divided into two panes; on the left pane, actions are specified and columns chosen, while on the right pane there are buttons for actually carrying out the actions. The common elements buttons are as follows: Free columns Same as input Add to action list Execute Remove all Remove selects the first set of free or empty columns for the output; they will not necessarily be next to each other. the input columns are effectively overwritten by the output from the action. this adds the chosen action to the list on the right-hand pane of the window, the actions are not yet executed, merely selected. executes all the actions selected on the right hand pane, a * will appear in the action executed column at the top of the right-hand pane. removes all actions not yet executed. remove any highlighted action from the Action list, but only if it has not been executed. 24

9 Undo undoes all the executed actions, that is those on the action list with a * Viewing the revised data (use Names or Data manipulation, then View or Edit Data or click on the Data tab if the window is still open) it is clear that the data has been sorted so that all the houses in District 1 are adjacent to each other in the worksheet, and then houses in District 2 and so on. This is a fundamental requirement that has to be done before modeling; if it is not done the results obtained are very unlikely to be the correct ones. When you are satisfied that the data are as required and that nothing has been overwritten, you can save the worksheet in the usual way. Closing a window means that the action, in this case Sorting, cannot be undone. 25

10 Recoding variables A useful procedure is to be able to recode a variable. Here we will recode the numerical size variable into two groups: houses with more or less than 5 rooms. 8 The sequence is: Data Manipulation on Main Menu Click on Recode Choose by Range (as it is a numerical value) Values in range 1 to 5 to New Value 0 Highlight input column Size Free column [highlights c6] MUST Add to Action List Values in range 6 to 10 to New Value 1 Add to Action List [as Size & C6 already highlighted] Execute 8 If the letter 'm' is entered in the values in range box this is expanded to 'missing' and allows you to operate on the current missing value code as a condition. 26

11 We can now use the Names window to re-name column 6 to largep (with the Edit name function); change from numeric to categorical (with Toggle categorical to True function), and give a more descriptive categorical textual label to the categories than the default labels (with the Categories function). As you can see we have used the labels Small for those houses coded 0, and Large for those code 1. Here is the data after recoding, renaming, and re-labelling Calculations and transformations The next type of data manipulation that we will consider is calculations involving columns of data. A straightforward calculation is to calculate the price per room for every house. The sequence is: 27

12 Data Manipulation on Main Menu Calculate Highlight c7 in left-hand pane and click across arrow Click = or type on keyboard Highlight Price and click across arrow Click or type / [that is divide] Highlight Size and click across arrow Click Calculate The price of house has been divided by the number of rooms and the result has been stored in a new column 7. The calculate facility can also be used to perform transformations. For example, we may wish to model the logarithm (base 10) of price. The sequence is now: Clear the previous sequence by deleting it Highlight c8 in left-hand pane and click across arrow Click = or type on keyboard Highlight LOGTen in list of functions, click up arrow Complete the equation so that it reads C8 = LOGTen( Price ) After naming the new columns Cost and LogPrice, viewing the data gives the following results: Again save your updated worksheet. Clearly, the arithmetic expression may be entered in three different ways: using the keyboard to enter numbers, column names (which must be enclosed in quotes ), column numbers, and the arithmetic operators +, -, * (multiplication), / (division), ^ (exponentiation, that is raised to a power), or (unary, which has the effect of multiplying the following term by 1); using the visual keypad in the lower right hand side of the window to 28

13 enter numbers or arithmetic operators; using the drop down menu box listing arithmetic and matrix transformations, and the upwards pointing arrow to place it at the cursor position; columns which are to be operated upon must be enclosed within round brackets. The 'missing value' button allows the missing value code to be included in operations. The calculator follows standard mathematical practice, but it is a good idea to separate the elements of a complicated expression with round brackets. The calculate window also allows logical expressions to be evaluated, returning the value 1 if true, 0 if false. The following relational operators are allowed: < is less than > is greater than <= is less than or equal to >= is greater than or equal to!= is not equal to == is equal to as are the following Logical operators & and or! not As an example we could have recoded the size variable into 0 for below 5, and 1 for above 5 in a single command: C10=( Size >=6) To produce a code in C10 that identifies properties more than 5 rooms and over 70 thousand pounds, we could use the following: Calculate can also operate on matrices. C10=( Size >=6 & Price > 70) Data Description The next stage is to undertake some statistical description. We begin with averages and correlations. which can be found under Basic Statistics on the Main Menu. For example we can describe the two numeric variables Price and Size 29

14 Results go to an Output window which automatically pops up when you click the Calculate window; this also shows the commands that were used in the back-end to derive the results. AVERage 2 price size N Missing Mean s.d. price size The output in the output window can be cut and pasted in to a word-processor. It is also possible to calculate a weighted average. For example, if the relatively-rare detached properties had been over sampled by a factor of 2 to get a more reliable estimate we could attach weights which are 0.5 for detached properties; 1.0 otherwise. [Bug: This does not appear to work currently] To select more than one column hold down the control key while highlighting. Correlations between variables are derived in a similar fashion. Note the means and standard deviations will also be produced automatically. CORRelate 3 price size logprice 20 observations Means price size logprice S.D.'s price size logprice Correlations price size logprice price size logprice

15 We next proceed to deal with categorical data. The simplest procedure is to produce a table or a tally for a single variable using Basic Statistics and Tabulate. Here we have chosen a single variable for the columns of the table; that is Type; we have also checked Counts, but not Means. All the other column names in the table (the two instances of house) are not highlighted. We have also asked for the percentage of row totals. The results from the output window are TABUlate 1 'Type' Det Terr Semi TOTALS N % There are three types of house in our sample of 20; 10% of houses are detached. The next graphic shows the choice of decision to cross-tabulate type on the columns with Largep on the rows (row box checked); and Percentage of the Grand Total checked. 31

16 Columns are levels of Type Rows are levels of Largep Det Terr Semi TOTALS Small N TOTAL % Large N TOTAL % TOTALS TOTAL % The revised output window shows that 5 of the 9 terraces are small properties; whereas both detached properties are large. The final example shows the results when we cross-tabulate type and property size again, but this time we request descriptive statistics (via the Means option) for the continuous variable Price The distinctive feature here is that the Variate column contains the continuous variable. The results show that mean price changes with both property size and type. Large detached houses having the highest price of 125k; while small terrace properties only average around 54k. Variable tabulated is Price Columns are levels of Type Rows are levels of Largep Det Terr Semi TOTALS 0 N MEANS * SD'S * N MEANS SD'S TOTALS MEANS SD'S As can be seen from the windows there are several features we have not used: there is an option to filter on other variables; so there results could be derived for houses only in district 1. you may also alternatively or additionally request percentages of the row, 32

17 column or grand total and a chi squared statistic for testing row and column independence; the latter also outputs the contribution to the overall chi squared for each cell of the table. the cells of a tabulation can also be stored in a column by checking the box for Store. Graphical display This final section of this chapter now uses the graphical procedures to produce some histograms and scatterplots. There is a lot of fine tuning available in MLwiN; here we are keen to limit discussion to the basics. There are three levels of structure in the graphs there are ten graph display sets, D1-D10 which are accessed using the top left drop down menu; a display is what can be shown on a computer screen at one time; everyone of these 10, can be subdivided into up to 25 sub-graphs using the Layout button, each of which is displayed in its own panel with its own X and Y axis; for each 25 sub-graph we can have up to 49 datasets plotted (indicated by ds #), each one consisting of a set of X and Y coordinates held in worksheet columns which can be drawn on the same or on different sub-graphs. This sound horribly complicated but an example should help. A single graphic display (D1 say) could consist of 3 subgraphs; one for a histogram of Price; one for a scatterplot of Price against Size, and one for Price against size together with Cost against Size on the same subgraph. The first two subgraphs would consist of a single data set (Price for the histogram; Price and Size for the simple scatterplot), but the third graph would have two datasets (one for Price and Size; the other for Cost and Size). Let us start by sticking with graph display D1, and drawing a histogram for the Price data. Graphs on main menu Customised graphs In the customized graphics window; highlight Price for y axis; and choose Histogram for plot type. The completed window should then look like: 33

18 To draw the plot, click on the Apply button which will open a new window in which the histogram will be displayed. If you click inside the Graph Display window, a Graph Options window will pop up; complete the Titles tab as follows, and use the Scale tab to choose nice values for the vertical and horizontal axes. Clicking on the Apply button brings up the graph with the required titles and scales. Not including the border, gives the cut and paste graph as follows. 34

19 Our next aim is to plot a scatterplot of Price against size for all houses. Click on Del Data set on the Customized graph window to remove the histogram. Choose Point on the Plot Type box on the Plot What? tab and choose Price for the Y axis and Size for the X axis as shown below. Clicking on the Apply button will produce the scatterplot, but to get the graph below you will have to alter the Titles and choose Autoscale for both axes 35

20 Clicking inside the graphical display and as close as possible to the most expensive property, and choosing the Identify point on the Graph Display tab will bring up the following: The identified point is a 8 roomed house costs 157k, and is in the 16 th row of the worksheet. Having got some feel for plotting, let us look at the Customized Graph Display window in a little more detail. On the far left there is a table grid which lists the data sets included in the current graph and there is a further column (accessed by scrolling to the right of this grid) which displays 36

21 the order in which the graphs are to be plotted. This can be changed from the Other tab. Clicking on a row in the table grid will result in the right-hand side window displaying, for modification, the details of the particular data set and associated graph. After a data set has been modified or specified you must click the Apply button for these changes to take effect. The modifications to a graph are based on a set of tabs: The Plot what? tab allows you, via drop down menus, to specify the following: Y and X variables: Grouping variable: Filtering variable: Plot type: Row and columns both for a scatter plot; Y for a histogram; a group is a higher-level identifier such as district, we will see later how to produce a varying relations plot with one line per district using this command; as the name suggests this is used to select only those points for plotting where the value in an associated filter column is equal to 1; we have already seen histogram and point, other options are Line - which draws lines between adjacent points in the order they appear in the column, and Line + Point which draws the points and joins adjacent points in the order they appear in the column with straight lines; this allow you to plot what are called trellis graphics whereby a set of graphs are plotted for each category of a categorical variable; for example if we choose Type for col codes We get three graphs; a scatterplot of Price and Size for each type of property. You will have to re-size the graphs to get all the labels on each graph. 37

22 Row codes result in graphs being placed one above each other; column codes result in the graphs being placed alongside each other. These category settings can be used in combination with each other to see the effect of more than one set of categorical variables. The Plot style tab allows you to specify the following: Symbol type and size: Line type and thickness: Colour: there are 14 different symbol types available with type 1 (triangle) as the default; any integer symbol size can be specified, with size 25 (points) as the default. there are 6 line types (default is 1 for a continuous line), and 5 line thicknesses (default is 1 for the thinnest); there are 16 colours available for symbols and lines, the default being blue); option 16 allows the colours to be rotated to show different groups; The Position tab allows you to position the current graph within a 5 x 5 matrix of subgraphs; if each subgraph is placed in the same position the sub-graphs will be superimposed on each other; The error bars tab allows the plotting of confidence limits; The Other tab allows you to produce a key to the plot with labels for subsets of points or lines in the graph. To construct labels you have a choice of using either the variable used to group data or any other variable (click on check box above and enter name of variable). This tab also allows you to change the plot order for the currently selected graph so that particular symbols or lines can appear to be superimposed on others. Its final feature concerns highlights. This is particularly useful when there are several related subgraphs on a display. You may highlight an outlier in one graph with a distinctive colour and then this point will show up as a highlight on 38

23 the other graphs. The Labels button deals with how the labels and legends are placed on each graph. The final item on the display is the check box Autosort with the tick signifying that it is on it as the default. This ensures that the data are plotted in ascending order of the horizontal variable. This is important if a grouping variable is specified. Playing around with the options we can produce the following: Two graphs on the same plot (via Layout) with Log of Price and Price being plotted against number of rooms; we used Plot Style to change the nature and size of the symbols Two scatter plots on the same graph; one showing price by number of rooms; the other showing cost per room by number of rooms. 39

24 The command window and the output window The command interface is where old-styled command can be given directly. On the command screen, if the user box is unchecked then, in addition to user specified commands, all commands issued by the MLwiN front end, that is system generated commands will also be displayed. Clicking on Output will open a window where all the commands and associated output and results will be displayed. This window will contain all user generated commands and associated output since the start of the MLwiN session. A separate check box at the bottom of the output window specifies whether MLwiN front end generated commands and associated output will be displayed but these will only be displayed following the checking of the box, not from the start of the session. This is useful, for example, if the user wishes to monitor error diagnostics occurring, say an iteration is proceeding very slowly, then stop after the iteration, open up the command interface output window, check this box and restart the iteration, leaving the window open. You will then see any error diagnostics being displayed. Creating a log activities It is often useful to keep a log of activities and to print out parameter estimates and values for variables. A log can be turned on in the following manner Data manipulation on main menu Command window Logo c:\templog.txt Print price size Fixe Rand Logo turns the log on prints out the values of variables lists fixed parameters estimates lists random parameters estimates turns the log off The file c:\templog.txt can now be read into a favoured word-processor ; we recommend a nonproportional font (for example Courier) and a small size font (such as 8 or 9 point) so as to preserve column alignment. Copying text When only small amounts of output need to be transferred to a word-processor, you can select text and copy to the clipboard for input to other software, from where it may be printed etc. Use the copy item in the edit menu. To select one individual line in these screens, simply click on it. To select multiple individual lines hold down the keyboard control key and click on each line you wish to select. To select a range of lines click on the first one, hold down the shift key and click on the last one of the range. 40