316 Appendix Interaction Data: Classroom Activities Adam Dennett University of Leeds, UK ABSTRACT This appendix contains a set of three activities that allow you to gain some familiarity with handling migration and commuting data and producing flow maps with three different GIS/mapping systems: MapInfo/MapBasic, Flowmap and Postgis. In all of the exercises you will be using data obtained from the Web-based Interface to Census Interaction Data (WICID). To access WICID, go to the CIDER website: http://cider.census.ac.uk/, click on the WICID logo and follow the login instructions. If you have not already done so you will need to register with the Census Portal before being able to access the data. You may also need to agree to some special conditions to access all features on the site. ACTIVITY 1: flow MAPPING Of STUDENT MIGRATIONS IN ENGLAND In this exercise you will select an area of England, download the flows of students to this area from all other areas and produce a flow map representing the volume of these flows. To complete this exercise fully, you will need to have MapInfo installed on your computer. Part 1: Data Extraction from WICID 1.1 Log into WICID. Click on continue and you should be taken to the general query interface to start building your query. 1.2 Start by clicking on the Data tab. When selecting data in this exercise, we will Select by dataset and table > Migration data > 2000-01, and use the 2001 SMS level 1 dataset > Table 5 Economic Activity by Sex. 1.3 In this table, you should select the Total economically active full-time students as well Copyright 2010, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.
as the Total economically inactive students (cells 22 and 31). Once ticked, click the Add selected cells button. 1.4 At this point, we are going to derive a new variable from the two just selected. Click the Derive new variables button, select both items and continue. Give your new aggregate variable the name Total students and Remove the old variables from the list of selected items before clicking continue. You are now ready to select your origin and destination geographies. 1.5 Firstly, click on Geography > Select or edit origins to select the origin geography you wish to use. 1.6 For this task, choose List selection and then select CIDS 1991/2001 common geography - 100 zones. 1.7 Confirm that you wish to use this geography and then click the Add all areas button as we are going to examine the numbers of students migrating from all 100 zones to our zone of choice. 1.8 Now you will need to select a destination zone. Click select Destinations > List selection and Confirm that you wish to proceed using the same geography. 1.9 Now you may select any destination that interests you. Once you have selected your destination zone by ticking the box next to it, click Add chosen areas to add the area to your destination selection. 1.10 At this point you may wish to click on the Query button at the very top of the page to view the summary of your current query. You should see that 100 origins are selected along with the one destination you have chosen. There should also be a green light on the Geography tab, indicating that you can proceed with data extraction. 1.11 If you are happy with your query, you should click the Run tab to extract your data and then Continue to output pages selecting Tabular output. 1.12 On the tabular output plan page, you should select Origin-destination pair list as the output layout and Comma separated values as the output format. Click on Preview output and download file to download your data selection. Call your file StudentMigrations. csv before clicking the button to download the file to a specified directory of your choice. 1.13 Once downloaded, open the file and delete the first four rows replacing the headings in the first row above the data with the words Origin, Destination and Total students. Scroll to the bottom of your file and delete the notes and source information as it is not required. Save the file as an excel (.xls) file and close it. Part 2: Mapping the Information Using MapInfo 2.1 Start MapInfo and Open the file UK- 100Zones.TAB which can be found here: http://cider.census.ac.uk/cider/training/ uk100zones.zip. Also open your Student- Migrations.xls file and save a copy in Map- Info as StudentMigrations.TAB (in newer versions of MapInfo the.tab file is created automatically when you open the excel file, so you may not have to save a copy). Obtaining Zone Centroid Coordinates 2.2 For this mapping exercise you will need to find the centroid of each of the 100 Zones. To do this, you will need to select Tools > Coordinate extractor > Extract coordinates (if the coordinate extractor is not already listed, you will need to go into the Tool manager and tick the box next to coordinate extractor to add it to your list). 2.3 You will need to extract the coordinates from the UK100Zone table. Check that the projection is set to British National Grid 317
and you should click the Create new columns to hold coordinates button and call the new table columns OriginX and OriginY. Once you click OK, you should be able to view the table with the X and Y coordinates created for each of the 100 Zones. Adding the Attribute Data to the Boundary Data 2.4 To combine your attribute and boundary data into one mappable table, you will need to use the SQL select function in MapInfo. Go to Query > SQL Select. When the module opens you should select all columns (*) from the tables StudentMigrations and UK100Zones where condition column StudentMigrations. origin = column UK100Zones.Cname. When you click OK your two tables should now be joined together into a single table. 2.5 Save the query as a new table called StudentMigrationFlows.TAB and then close all tables. Creating Destination Zone Coordinates 2.6 Open your new StudentMigrationFlows. TAB file. Also open the UK100Zones file. To begin with, you will need to add two new columns to your StudentMigrationFlows table. To do this, go to Table > Maintenance > Table Structure and select the StudentMigrationFlows table. Add two fields named DestX and DestY of the type Float. Click OK and then re-open the table. You will see that two new empty columns have been added to the end of your table. 2.7 To create the destination coordinates in these columns, do the following: Select Table > Update column. Firstly, update your new DestX column, and get the value from your UK100Zones table. You will need to join your tables where destination in your StudentMigrationFlows table matches Cname in the UK100Zones table. In this case, you are copying the value of OriginX from where your destination appears in the origin column, so select calculate the value of OriginX, before clicking OK. You should now see your DestX column updated with the X coordinates from your selected destination. To fill your DestY column with coordinates, follow the previous step substituting OriginX for OriginY where appropriate. Once done, save your table. Mapping the Migration Flows 2.8 Your StudentMigrationFlows table should now contain the UK100Zones boundaries, the data you wish to map and the coordinates of the zone centroids for both the origins and destination you want to map. You now will need to make a copy of StudentMigrationFlows, and call it something like StudentMigrationFlows1. To do this, use the Save copy as option in the file menu. You are doing this as it is this version of the table you are going to make the flow lines from. 2.9 Close all tables and open your new Student- MigrationFlows1 table. Go to the Options menu and select Show MapBasic Window. Once the MapBasic window has opened, to create the flow lines, type the following: Update StudentMigrationFlows1 set obj = CreateLine(OriginX, OriginY, DestX, DestY) 2.10 Once you hit the return button, you should see lines created from the centroids of each of the 100Zones origins to the centroid of your chosen destination. You may find that the map window remains blank. If this occurs, it is because the Map Window settings are incorrect. You can solve this problem by going to the Options menu > Preferences > Map Window and setting both the Table 318
projection and Session Projection to British Coordinate Systems and British National Grid. Then run your MapBasic command again and the flows should appear in the Map Window. 2.11 Currently you will only see the lines displayed on your map. To display the UK100Zones boundaries with your map, you should reopen either the StudentMigrationFlows table or the UK100Zones table. Doing this should automatically add the zone boundaries to your map with flows lines. 2.12 This next stage is to produce a thematic map so that the flow lines are displayed of differing thickness depending on the flow size. Go to the Map menu and select Create thematic map. You should choose Line Ranges, Varying Width style. In step 2, select StudentMigrationFlows1 and choose a variable you wish to see mapped in this case your only choice is likely to be Total students. Click Next to go to step 3 where you can customise your map (for example you may wish to alter the colour of some of the lines to add additional clarity to your map, or change the legend). Click OK and a flow map will appear which allows easy interpretation of from where students are migrating from, to your chosen area. Figure A.1 shows the distribution of student flows to Leeds from 100 zones in the rest of England in 2000-01. Figure A.1. An example flow map showing student flows to Leeds from 100 zones areas in England 319
ACTIVITY 2: ANALYSING REGIONAL MIGRATION flows IN BRITAIN USING flowmap SOfTWARE Activity 1 made use of proprietary software in the form of MapInfo to analyse flow data. We recognise that not every user of interaction data has access to commercially developed, GIS packages which are sometimes expensive. Therefore the next two activities make use of freely available software to analyse flow data. In this activity, we make use of Flowmap a GIS package designed specifically to handle data related to origin and destination pairs. Flowmap can be downloaded from: http://flowmap.geog. uu.nl/. Download and install Flowmap on your computer if you wish to attempt this activity. Part 1: Data Extraction from WICID For this exercise you can choose any migration variable you wish. 1.1 Log into WICID. Click on continue and you should be taken to the general query interface to start building your query. 1.2 Start by clicking on the Data tab. When selecting data in this exercise you can try the Select by variable option. This facility allows you to select variables that fall under any of the categories listed. By clicking on one of the categories, the list will be expanded to include all of the datasets and tables that feature variables in this category. 1.3 You will see that datasets are identified by the census years they are associated with, as well as by an SMS, SWS or STS identifier, indicating whether the data are for migration or commuting. In this exercise, we will be using post-1999 Government Office Regions as our geography, so choose a 2001 data set. We will also be looking at migration flows, so choose an SMS set (although if you wanted to look at inter-regional commuting, this would be possible), remembering that level 1 data are likely to be more accurate than level 2 or 3 data. Click the table you wish to take data from. 1.4 Once you are on the variable selection screen for your chosen table, choose one variable you wish to analyse (or alternatively a new variable derived from a number of variables see Activity 1 for instructions on deriving new variables). Once you have selected your variable, click Add select cells and then click the Geography tab to begin selecting your origins and destinations. 1.5 Firstly, click Select or edit origins to select the origin geography you wish to use. 1.6 For this task choose List selection and then select UK Government Office Regions (1999-) 1.7 Confirm that you wish to use this geography and then tick areas 1-11 (Government Office Regions in England plus Scotland and Wales) before clicking the Add chosen areas button. 1.8 Now you will need to select destination zones. Click select Destinations > Copy selection and click the Origins -> Destinations button to select your destinations to equal your origins. 1.9 At this point you may wish to click on the Query button at the very top of the page to view the summary of your current query. You should see that 11 origins are selected along with the same 11 destinations. Your data item will also be shown. There should now be a green light on the Geography tab, indicating that you can proceed with data extraction. 1.11 If you are happy with your query, you should click the Run tab to extract your data and then Continue to output pages selecting tabular output. 320
1.12 On the tabular output plan page you should select Origin-destination pair list as the output layout and comma separated values as the output format. Click on Preview output and download to download your data selection. Call your file RegFlow2.csv before clicking the button to download the file to a specified directory of your choice. 1.13 Once downloaded, open the file and delete the first four rows replacing the headings in the first row above the data with the word label1, label2 and score. Scroll to the bottom of your file and delete the notes and source information as it is not required. 1.14 You should now save your file in.dbf format (dbf III if you have a choice) with the same file name (e.g. RegFlow2.dbf). Notes (i) (ii) If you are using Excel 2007, you will not be able to save in.dbf format. To get around this you will need to save your data as an Excel (.xls or.xlsx) file, and open this file in MS Access. In Access you will be able to save or export the file as a.dbf. For Flowmap to recognise the.dbf data file, the file name must be in an exact format. It MUST consist of 7 characters no more, no less and end in 2.dbf. Any file name not in this precise format will not be recognised by the program. Part 2: Mapping the Data in flowmap Preparing the Files 2.1 First you will need to place a number of files into a project directory for Flowmap to read. These additional files can be found on the CIDER website at: http://cider.census.ac.uk/ cider/training/training.php under Additional data for use with Flowmap practical. 2.2 Download these files and unzip them into a directory of your choice (preferably the default C:\Program Files\FLOWMAP directory, but any will suffice). The unzipped file should contain two files: GB_Rgion.BNA and Region01.DBF. To the directory where these two files are now located, also add the RegFlow2.dbf file you created in the last section. 2.3 Start Flowmap. The first thing that you will need to do is create a map file from the.bna import file you have just unzipped. Go to File > Convert Files > BNA > BNA -> Flowmap and locate the GB_Rgion.BNA file. Flowmap should then give you a default file name for the new map file. This will probably be GB_Rgion.006. Click Save. It may ask you to select vector type. Choose Polygon. It may also find some topological errors in the file. This is not a problem for this exercise and you should just choose the Continue regardless option. 2.4 Select File > New Project. The New Project window (Figure A.2) will open. You might need to change your workspace directory so it matches the directory you have just unzipped the additional files into. 2.5 You should now add the files you have created and downloaded to the project. As above, the map file will be GB_Rgion.006; the origin and destination files will be Region01. dbf; and the flow file will be RegFlow2.dbf (with origin set as LABEL1, destination as LABEL2 and flow size as SCORE). You should also tick the three boxes under View Settings and click the Set button. Click Save as and save your project. Mapping the Data 2.6 Now a map of your flow data can be created. To draw the base map go to Maps > Advanced Display > Draw Map File > Draw Edges/Lines > Uniform Drawing and select 321
Figure A.2. New Project window in Flowmap an appropriate line style. This should draw your base map. 2.7 Before drawing the flow lines, it is useful to adjust the symbology so that larger flows are more clearly distinguished from smaller flows. To do this go to File > Edit Symbology Settings > Customise Flow Symbology settings and click on one of the triangle symbols. Set the thickness to Max and click Ok. 2.8 To draw the flows go to Maps > Advanced Display > Draw Desire Lines from Flow File and select the symbol you have just adjusted. You may also wish to Set intrazonal interaction as symbols to display intra-zonal flows. Your finished flowmap should resemble something similar to the example in Figure A.3. ACTIVITY 3: DISPLAYING INTERACTION DATA USING OPEN SOURCE DATABASE AND GIS SOfTWARE: MAPPING THE PREfERRED METHOD Of TRAVEL TO WORK for OUTPUT AREAS IN LONDON In this activity, you will be introduced to two very powerful pieces of free (open source) software: PostgreSQL and Quantum GIS. PostgreSQL is a relational database management system which importantly has a spatial extension known as PostGIS. For this exercise you will need to install PostgreSQL with the PostGIS extension. PostgreSQL along with PostGIS can be downloaded and installed from the PostgreSQL website: http:// www.postgresql.org/ Note: When you install the program, if you have not used it before it is recommended that you 322
Figure A.3. Intra- and inter-region flows depicted using Flowmap database tables. QGIS can be downloaded from http://www.qgis.org/ - the version used in this example is version 0.11.0. Newer versions of the software may now be available. Part 1: Downloading your Data from WICID and UKBORDERS This section will go through, in detail, the process of selecting output area boundary data from UKBORDERS for all output areas in London. It will also describe extracting from WICID, 2001 commuting data by all methods of transport (with car passenger and car driver aggregated into one variable) from all output areas as an origin and the London region as a destination (downloaded as a paired list. If this is a familiar process, attempt it yourself and move straight to Part 2, otherwise follow these instructions: install to the host localhost, on the default port 5432, with a username postgres and a password postgres. Quantum GIS (QGIS) is a user friendly, graphical interfaced Geographical Information System developed by a community of open source developers. QGIS supports vector, raster and database formats and, importantly for this exercise, has direct support for PostGIS spatially enabled 1.1 Customised boundary data downloads can take some time from UKBORDERS, so we will go through this process first in order to get the boundaries by the end of Part 1. Go to the UKBORDERS website http://edina. ac.uk/ukborders/ and log in. 1.2 Choose the Boundary Data Selector and select England > Census boundaries > Post 1999 and click Find. Click English output areas > List areas click Greater London > Expand. You now see that the search summary has your target geography as English Output Areas and the selected area as Greater London. Check that the data format is ESRI Shapefile. If it is not, use the Format tab at the top of the page to change the format. Click Extract Boundary Data. You will now be taken to the data extraction screen. It is normal that because of the large number of boundaries being extracted you will be taken to the UKBORDERS bookmarking facility. Bookmark this page in your browser and return to it at the end of Part 1. 323
1.3 Log into WICID. Click on continue and you should be taken to the general query interface to start building your query. 1.4 Start by clicking on the Data tab. When selecting data in this exercise, we will Select by dataset and table > Commuting data > 2001, and use the 2001 SWS level 3 dataset > Table 1 Method of travel to work. 1.5 We want to select all persons and all methods of travel, so check boxes, 4, 7, 10, 13.34. Once checked, click the Add selected cells button. The status at the top of the page should inform you that eleven data items have been selected. Two of these items (car driver and car passenger) will need to be aggregated into a single item, however, before we continue. 1.6 At the bottom of the page click on the Derive new variables link. This will take you to a new page where the two variables can be selected. Tick the boxes next to car passenger and car driver and click Continue. Call your new variable car and check the option to remove the old variables from the list before clicking Continue. 1.7 You are now ready to select you geographies. Click the Geography tab and select origins first. Choose List selection > All geographies > UK Output Areas 2001 > Select a higher level geography > UK Government Office Regions (1999-) > Confirm that you wish to proceed with this geography. Now, by ticking London and clicking the Add chosen areas button you will select all 24,140 output areas within London. 1.8 You can now select your destination. Click Select destinations > List selection > All geographies > UK Government Office Regions (1999-) > Confirm and check London before clicking Add chosen areas. This should select just the one area, London, as your destination. 1.9 You can now click the Run tab to execute your query. Once the query is complete select Continue > Tabular output. On the tabular output planner page make sure that your output layout is a pair list and the format is Comma separated values. Download your file and save it to an appropriate location. 1.10 If you return to your UKBORDERS bookmark, your data should now have extracted and you should be able to download the zip file containing your boundary data. Download this to the same location as your commuting data file. Part 2: Preparing the Data for Mapping Preparing Your Data from the Raw Download File 2.1 Open the.csv file you downloaded from WICID. Column A should contain the codes of the output area origins, B will just contain London as a destination and columns C to L should contain counts of commuters by their mode of travel for each output area. 2.2 We now want to convert this data so that for each output area, we know the most popular method of travel to work. To do this we need the column reference of the method of travel with the highest count for each output area. Assuming that C5 is the first cell with numeric data type the following formula into cell M5: =IF(C5=MAX($C5:$L5),COLUMN(0,0) This formula returns a numeric column reference for the maximum value in the data between cells C5 and L5. The $ symbols fix the range from C to L enabling the formula to be copied across and down to other cells. 2.3 Copy this formula to each cell until cell V5. You should find that each cell contains a value 0 except for T5 which contains 20. Highlight this new row of values from M5 324
to V5. Double click the small black square that appears in the bottom right corner of the square and the formulas should be copied all the way down to the last row of data. 2.4 The next empty cell you have should be cell W5. In this cell type the following formula: =MAX(M5:V5) In the same way you copied the last formula down all rows by clicking on the small black square in the corner of the cell, copy this formula to all rows in column W. Column W will now contain a reference to the most popular method of travel to work for all output areas in London. You might want to make a note of the method each number relates to e.g. 13 Home worker, 14 Underground et cetera. 2.5 Open up a new excel spreadsheet and copy and paste the output area reference codes values from column W into columns A and B in the new spreadsheet. Do not include a header row. You should now have values in every row down to 24140. Go to Save As and save a copy of the file as Text (Tab delimited) (*.txt) under the name london_commuting. txt (click yes if any pop-ups appear). Adding your Data to PostGIS / PostgreSQL 2.6 You now need to set up a PostGIS enabled PostgreSQL database to store this data in. Go to pgadmin to start Postgres and double-click on the localhost server you set up when you installed the program. You may be prompted for a password if you set up the installation as recommended at the beginning of this exercise, this will be postgres. You will now need to set up a new database on this localhost server. Click Edit > New object > New database. Call your database commuting but make sure you use postgis as a template. Leave the encoding as SQL_ASCII. 2.7 After a moment pgadmin will have created a new database into which you will be able to put both the interaction data you have just prepared and the boundary data downloaded earlier. Click on commuting and then the + symbol. Next + Schemas + Public + Tables - this should reveal the six tables including a table called geometry_columns. If this table exists then your database is spatially enabled. 2.8 We will now copy the london_commuting. txt data into a table called london_commuting in the commuting database. Click the SQL icon in pgadmin to open up an SQL window. In the box type the following: CREATE TABLE london _ commuting ( orig _ label text, method _ travel integer ); COPY london _ commuting FROM C:/london _ commuting.txt ; Now click the green play button to run the query. Note: You will probably need to change the drive and the path to london_commuting.txt to the location it is stored on your computer. If any error messages appear, check that the original london_commuting.txt file only has two columns of data and there are no column headers in this file. If you click the refresh button in pgadmin you will see you new table has appeared. Right clicking on this table you should be able to View data > View all rows to see the table. 325
Adding your Boundary Data 2.9 We can now add the boundary data downloaded from UKBORDERS to our new commuting PostGIS database. If you have not already done so, un-zip the file downloaded from UKBORDERS. Start Quantum GIS. 2.10 Once QGIS has started, go to the Plugins > Plugin manager and check the box next to SPIT and click OK. You are now able to click on the SPIT icon in QGIS to bring up the box shown in Figure A.4. Click New and create a new PostGIS connection using the options shown in Figure A.4. 2.11 Once you have created a connection to your PostGIS database you will need to click the Connect button to set up SPIT for importing the shapefile. After clicking Connect, click Add and navigate to the shapefile containing the boundaries of output areas in London. SPIT will scan the file and then add it to the selection window. Click OK to import the file to your commuting PostGIS database. Note: This may take a few minutes. 2.12 If you click the refresh button in pgadmin you should now see your new boundary data table. This table can be mapped in QGIS, but at the moment your london_commuting data cannot. To map this data you will need to create a new table which combines the two tables together using a common identifier in this case the output area code. The quick way to do this is to input the following into a new SQL window: create table london _ commuting _ mappable WITH OIDS AS select * from london _ commuting, england _ oa _ 2001 where london _ commuting.orig _ label = england _ oa _ 2001.ons _ label; Click the run button. If you refresh the tables list in pgadmin, your new table should appear. Figure A.4. Shapefile to Postgis Import Tool in QGIS 326
Note: The WITH OIDS is key to this new table as QGIS will not be able to map a table without an index. Part 3: Mapping your Data 3.1 Now you have set up your data, you are able to produce a map using QGIS. Go to Layer > Add a PostGIS layer. A connection to commuting should have already been set up when you imported the shapefile in the last section. Select this database from the dropdown list and click connect. Click on the london_commuting_mappable layer and then click Add. A plain map of all output areas in London should appear. 3.2 To create a thematic map of your data, rightclick the layer in the Legend window and select Properties. A box similar to the one in Figure A.5 should appear. In the Symbology tab change legend type to Unique value and classification field to Method_travel. Click Classify and the seven unique values relating to the seven types of transport most popular for output areas should appear. Hold down Shift and highlight all seven before changing Outline style in the Style options to No Pen. Click Apply and OK and your thematic map should now be drawn. 3.3 You can now edit your map and legend add additional layers et cetera so that an end product similar to the one shown in Figure A.6 can be produced. Figure A.5. Layer Properties window in QGIS 327
Figure A.6. Preferred method of travel by output area in London, 2001 328