Spatial Data Analysis

Transcription

1 Spatial Data Analysis Laboratory Exercises by Luc Anselin University of Illinois, Urbana-Champaign Prepared for the Workshop in Spatial Analysis Wharton School University of Pennsylvania August 29-31, 2001 Copyright 2001, Luc Anselin, All Rights Reserved

2 Introduction The exercises that follow are a fairly straightforward demonstration of the three aspects of spatial data analysis that were covered in the lecture part of the workshop. These are: (1) exploring spatial data; (2) assessing and visualizing spatial autocorrelation; and (3) spatial regression specification search. These techniques are illustrated by means of the Columbus data set that is available on the SpaceStat web site, and is the same one used to illustrate the various spatial econometric methods in the SpaceStat Tutorial. A more extensive set of exercises is available in 90+ page Workbook, Spatial Data Analysis with SpaceStat and ArcView (Anselin 1999), also available from You are encouraged to download the workbook and accompanying sample data sets from the SpaceStat site at any time, to further delve into hands on spatial data analysis. Getting Started ArcView preliminaries: Make sure you have a working directory that contains the Columbus data sets. There should be the three files for the ArcView shapefiles (Columbus.shp, Columbus.shx, Columbus.dbf), two files with the Columbus SpaceStat data set (Columbus.dat and Columbus.dht) and two files with spatial weights (rook.gal and queen.gal). Open ArcView 3.2. Make sure the DynESDA and SpaceStat Extensions are installed and active. Click on File > Extensions and make sure the two extensions are checked, as shown below:

3 Create a new View with the Columbus shape file. In ArcView, click on the View icon followed by the New button and an empty view will open as a window. Next use the menu for View > Add Theme or use the corresponding button to add the Columbus.shp theme, as shown below (the drive and directory path may be different in your case, but the shape file should be the same). Next check the button next to Columbus.shp in View1 and the outlines of the Columbus neighborhoods will appear.

4 Create a simple choropleth map of Columbus Neighborhood Crime. Double click on the rectangle below Columbus.shp to start the legend editor, and select Graduated Color as the Legend Type, with Crime as the Classification Field. You should get the following map (colors may differ). If you are familiar with ArcView functionality, feel free to use a different type of Classification for the legend (click on Classify ) DynESDA preliminaries: Make sure the View with the Columbus map is active and launch the DynESDA extension by clicking on the button. A toolbar will open on your screen, with menus and buttons for each of the DynESDA functions. SpaceStat preliminaries: Launch SpaceStat by clicking on its shortcut. Press any key to get beyond the opening screen. Enter the path for your current working directory at the prompt. In the example below, this is D:\demo\colshape, that is, the same directory as for the shape files. Make sure you enter the proper path for your workspace and that both the shape files and the SpaceStat data sets are in the same directory. Your screen should be as below. Minimize the SpaceStat window. You will work with ArcView and the DynESDA extension first. Rearrange the ArcView window on your screen so that it is moved to one side, with about half the screen available to be used by the windows from the exploratory analysis.

5 Exploring Spatial Data Linking Windows Select the histogram option in DynESDA, either by using the menu of the toolbar (Explore > Histogram) as illustrated below, or by clicking on the histogram button (the left most button). Scroll down the list of variables (illustrated below) until you find Crime, double click (or click and then click on OK); a new window will open with a standard histogram of the crime variable. Adjust the histogram to be a bit more informative by changing the number of intervals from the default 6 to 12, by means of the Tools > Intervals menu item. Type in the desired number (12) or move up the list and then click OK (you can also test various intervals by means of the apply button; these changes are not permanent, once you click OK, they are). Resize and/or move the histogram window until you can see it along with the choropleth map.

6 Now select the boxplot option in the toolbar (select Explore > Boxplot or click on the boxplot button). Select a variable as for the histogram, but now double click on Hoval. A boxplot window will open, with two points as outliers [the box plot has the familiar form, with the dark part corresponding to the quartiles around the median the blue dot in the middle the thick lines below and above the box are the fences ; observations outside the fences are referred to as outliers]. You can change the definition of the fences from 1.5 interquartile range (the default) to 3 times the IQR with the Tools > Hinges menu item (notice how this affects the impression of outliers). Resize and/or move the window so that you can see all three relevant views of the data (the map, the histogram and the box plot). IMPORTANT: make sure to always close the DynESDA functions (File > Quit or close the floating toolbar) BEFORE you exit ArcView. Otherwise, strange things can happen Also, the toolbar is connected with a particular view. If you want to close that view, you need to close the toolbar first. So far, all you have is three different ways to summarize the distribution of the data. Now, start by highlighting (clicking) on the lowermost category in the histogram (if you selected 12 categories, it should have 2 elements in it. Once you click on the histogram, the corresponding points are also highlighted (in yellow) or linked on the two other views. You can link any observation in any view of the data to the other views by clicking on it. For example, use the select tool in Arcview to select the easternmost neighborhoods. The matching items highlighted in the other graphs show the spatial distribution for this subregion. For crime this is pretty much the same as the overall pattern. Now use the circle select tool in the View and select the neighborhoods nearest to the center of town. Assess the extent to which the pattern is still the same as overall. This tool can be used to assess the extent to which the distribution of subregions matches the overall pattern, or, as a visual approach to assess spatial heterogeneity. In the box plot, you can select individual observations by clicking on them, and add unselected observations (or unselect selected observations) with SHIFT-Click. Unselect the neighborhoods in the map in ArcView. Move to the box plot to select the outliers for Hoval by drawing a box around these observation points to select them. You can switch the selection by double clicking in the graph. Assuming you selected some points in the previous step, double click in the histogram and observe what happens (this is the equivalent of select unselected in ArcView). Double click again to restore the selection. The easiest way to clear all selected observations is to do it in the View (click the selection tool anywhere outside the map); you can also select a single point and then deselect it with SHIFT-click, or select all the points and double click to unselect. So far, there is no command to clear the selection. Try the various combinations to make sure you have the mechanics down. Practice: Use the linking and brushing methods to assess the type of relationship between crime and hoval and crime and inc. Also assess where the outliers are. Check your findings with the slope in a scatterplot of crime on inc and crime on hoval

7 respectively [in the scatterplot function, you select the variable for the x-axis first, then the variable for the y-axis] Brushing A brush is a small rectangle that can be moved over the data in a graph to investigate how changing subsets of observations have similar or different associations in a multivariate setting. Here, you will look at the regression slope in the scatterplots of crime against inc and hoval how it changes with changing subsets of the data. Make sure you have the two scatterplots from the previous practice session. If not, create them by clicking on the scatterplot button, select inc as the x-axis variable, and crime as the y-axis variable, and for the second scatterplot, use hoval and crime. You brush a scatterplot by moving to one of the two scatterplots and making a long and narrow rectangle around the highest observation on the x-axis (or, alternatively, on the y-axis). Click in the left-most corner and drag to form the rectangle, followed by pressing the CTRL key (the rectangle will flash for a brief moment). Now move the brush from high values of the covariate (or, alternatively, high values of the dependent variable) to low values: different sets of observations are selected in the three graphs (two scatterplots and one map).

8 You can use this same procedure to brush a box plot as well (try it out if you are curious). The scatterplots can adjust to the selected data in the sense that the regression slope can be recomputed for the unselected points as the selection changes. In one of the scatterplots, use Tools > Exclude Selected. There are now two slopes reported in each scatterplot: one pertaining to the full set (in blue) and one only pertaining to the unselected points (in brown). To get the slope for the selected points, switch the selection with a double click. Adjust the selection of points and observe how the slope changes. You can use this to assess the leverage of particular observations, or to identify visually observations that have high influence on the regression slope. Practice: Use the select button in the View to assess the degree of spatial heterogeneity in the slopes of the two scatterplots. For example, use the circle select tool to pick out neighborhoods in the core. You can see the slopes for each of the subsets by double clicking on the scatterplot. The brown slope is the one for each subset. Compare the core-periphery regimes to an east-west or north-south pattern (or any subset you may want to select). You can extend this (provided you have sufficient screen real estate) by including other graphs, such as histograms for the individual variables, etc. Spatial Autocorrelation Moran Scatterplot You will continue to use the DynESDA extension to compute a Moran s I spatial autocorrelation statistic, visualize it in a Moran scatterplot and assess its significance. Click on the Moran scatterplot button, or use Explore > Moran Scatterplot on the DynESDA menu to launch the variable selection dialog: double click on crime. Repeat this for hoval and inc. You will have three scatterplots of the spatially lagged variable against the original variable (in standardized units, so each unit on the graph equals one standard deviation) The slope in each of the plots equals Moran s I. This value is also listed at the top of the graph. Select one of the graphs and click on the Tools > Randomization menu item, pick 99 as the number of permutations (see below). A new window opens that shows the values of Moran s I for all the permuted data sets and compares the observed statistic to that reference distribution. It also lists a pseudo-significance level. Close the window and repeat this for the two other variables (each time selecting 99 as the number of permutations). Note how the significance is the same for all, even though the slopes are quite different. Now repeat this analysis, but this time use 999 for the number of permutations. Do you still get the same result? You can assess the variation of spatial autocorrelation in subsets of the data (a form of spatial heterogeneity, or spatial non-stationarity) by combining the Tools > Exclude Selected feature with a Moran scatterplot. This can be used to compare the value for the complete data set to that obtained when selected observations (such as boundary locations) are excluded. It can also be used to compare Moran s I in two subsets of the data (using the double-click to switch the selection). The brown value

9 for Moran s I shows the value for the subset, the blue one is for the overall data set. Try this for one of the variables in your set. Practice: Use the Moran scatterplots and the select tools in the View to assess the extent to which the degree of spatial autocorrelation differs between the east side and west side (or center periphery) for each of the variables. Can you discover some really interesting patterns (such as halving of the coefficient, switches in sign, etc.). LISA Maps To study local patterns of spatial association, you will use the combination of SpaceStat as the analytical engine and ArcView as the visualization device, by means of the SpaceStat extension. Make sure this extension is active (see the getting started section). SpaceStat Options: in SpaceStat, make sure that the options are set as shown below (except for the exact path of the working directory). Specifically, the Report Format should be set to comma-delimited and the indicator variable should be set to polyid. You invoke the options by CTRL-O in SpaceStat and return to the SpaceStat menu by pressing the ESC key. NOTE that the mouse is of no use to you in SpaceStat (the interface is keyboard driven only). Change the options by moving the cursor (using up and down arrows) or by pressing the option number (4 for indicator variable, 5 for report format). Enter the proper response to the prompt (polyid and 2 for comma delimited).

10 In the SpaceStat main menu, select Explore (type E) > 3 Moran > 9 Local Moran- Permutation. Select interactive as the way to formulate your problem (option 2) and press return to waive saving a problem file (once you are more familiar with SpaceStat s operations, it makes more sense to save a file that contains the selected data set, weights, variables, etc.). In response to the next set of prompts, enter the following: o Data set: Columbus followed by return o Spatial Weights: rook followed by two returns o Variable Name: crime followed by two returns Next, you will get a series of result screens, which we will ignore for now since we will be using ArcView to look at the results. Make sure you press return after each screen to get back to the Explore Moran menu. In ArcView, make sure that a View with the Columbus shapefile is active, select the SpaceStat menu and click on LISA Local Moran Map. Select lm_rook.txt as the File in the first dialog and S_crime as the variable in the second dialog, as shown below. A new view will open with a LISA map of the locations for which the Local Moran statistic is significant, with different colors for 0.05, 0.01 and

11 With the LISA Map view active, click on the Moran scatterplot button in the DynESDA toolbar [make sure you close the toolbar first and then reopen it if it was active for another view]. A new window will open with the matching Moran scatterplot. By selecting any of the significant locations in the LISA Map, you can find out in which quadrant (high-high, low-low, etc.) it belongs to, which indicates the type of local spatial autocorrelation (clustering or outliers). You can obtain the same result, but only for significance at 0.05 by selecting SpaceStat > Moran Significance Map: select lm_rook.txt at the first prompt and M_crime at the second. A new View will open with color codes matching the quadrants in the Moran scatterplot (you can verify this by opening a Moran scatterplot using the DynESDA toolbar and selecting locations in the View). Practice: Repeat the exercise for crime, but now with the queen spatial weights. Compare the resulting indications of local association. Where are the differences? You can also carry this out for multiple variables as well as multiple spatial weights. The file names of the report files are all lm_weight.txt, where weight is a shortened version of the spatial weights file name. Spatial Regression Spatial Specification Search From now on, you will be using SpaceStat exclusively for the analysis. Set the Output File Option to output.txt so you can save the results of your analysis (CTRL-O > 2 enter the file name at the prompt, and ESC to get back to the main menu).

12 Select the Regress module (type R) > 1 Classic Model > 1 OLS. The next interface is identical to the one for the Local Moran in the previous exercise. Select 2 for Interactive (skip the problem file) and enter the following in response to the prompts: o Data Set: Columbus (followed by return) o Spatial Weights: rook (followed by two returns) o Model Specification: 1 Generic Regression o Constant term: Return o Variables: crime (return), inc (return), hoval (two returns)

13 o Heteroskedastic variables: return After you enter this information, the program starts estimating the model and a series of result screens appear. You move from screen to screen by pressing the return key. If you have the Output File set then these results will be dumped into that file (otherwise, they are lost forever ). First is the screen with the estimates, as below: Note in particular the value of LIK, the maximized likelihood: you will need to compare this fit to that obtained in the ML estimation of the spatial models. The next screen contains multicollinearity diagnostics and a test against normality, followed by a series of tests against heteroskedasticity, and lastly a screen with the results of the spatial tests, as below:

14 Both the Lagrange Multiplier (error) and the Lagrange Multiplier (lag) tests are significant, suggesting an inconclusive result at first sight. However, of the two Robust versions of the test, only the LM-lag one is (somewhat) significant. Hence you should conclude that the proper alternative is a lag model. Practice: Repeat this analysis for the queen weights and/or for other specifications of the model (e.g., include a spatial trend) and assess the degree to which the diagnostics are affected. Estimating Spatial Models Since the specification tests suggested that a lag model is the proper alternative, you will now estimate this using both a maximum likelihood method as well as instrumental variables. In SpaceStat, start the Regress module (type R) > 4 Space Lag Model > 1 SAR-ML. Use 2 Interactive for the problem description, select Columbus as the data set and W1 as the weights file (IMPORTANT: do NOT select rook or queen for the weights file; maximum likelihood estimation requires this to be a full matrix and not the sparse format used for rook and queen). Enter the remainder of the problem description in exactly the same way as as for the OLS model (see previous exercise). The first screen of results contains the estimates and measures of fit. Note how the significance of the estimates has changed and the improved fit over OLS (LIK).

15 The second screen shows various diagnostics. A test for remaining heteroskedasticity (clearly still a problem here), a Likelihood Ratio test on the spatial autoregressive parameter (this is one of the three classic tests on the significance of spatial lag dependence) and a Lagrange Multiplier test on remaining error autocorrelation. The latter is a diagnostic for misspecification (i.e., not all the autocorrelation has been taken care of), whereas the LR test is a significance test on the ρ parameter. An alternative estimator for the lag model is based on instrumental variables. Select R > 4 Spatial Lag Model > 3 SAR-IV(2SLS). Enter the same information as before in response to the prompts (NOTE: here you can use either W1 or its sparse counterpart rook as the weights file, IV estimation is implemented for sparse spatial weights). Note a slight difference in the sequence of prompts for the variable names. o Dependent and Endogenous Variables: enter crime (followed by return) o Exogenous variables: enter inc (return) and hoval (followed by two returns) After this follows the screen with estimation results. Note that you do NOT need to enter the instruments. SpaceStat computes these for you (as the spatial lags of the exogenous variables in the model). Compare the estimate for the spatial autoregressive coefficient to that obtained with ML estimation. Also, compare the standard errors of the estimates: what do you observe? (there is also a screen with diagnostics, which we will skip for now). Practice: Compare the results above (lag model) to those for an ML estimation for the spatial error model. How does the fit compare? How are the standard errors affected? If you are adventurous, experiment with the other model specifications in SpaceStat (see the Workbook and the SpaceStat Tutorial for more extensive guidelines)