Working with International Census Data in GIS

Transcription

1 Working with International Census Data in GIS Updated February 23, 2015 Table of Contents SETTING UP... 1 DATA SOURCES... 1 TABULAR CENSUS DATA Tabular Census Data... 5 GEOGRAPHIC DATA... 5 JOINING DATA... 5 DEFINING SYMBOLOGY... 9 ADDITIONAL CENSUS DATA EXPLORATION... 9 MAP LAYOUT... 9 Setting Up 1. If they are not already mapped, map out M and S drives from the GIS Center. Go to My Computer. From the menu bar, select Map Network Drive. 2. In the drop down menu labeled Drive: choose the drive letter M 3. In the Folder: drop down menu manually enter \\rstore2\gis$\datasets 4. Leave the box checked Reconnect at logon. 5. Click Finish. 6. Repeat the same step to map the S: drive except enter \\rstore2\gisprojects$ Navigate to S:\classes\DHP_P207\ 7. Leave the box checked Reconnect at logon. 8. Click Finish. Data Sources International Data 1. The Tufts GIS website provides connections to many data sources, including international ones. Start at and click on Online GIS Data Sources under the DATA SOURCES menu. 2. Under Geographic Region, click on International then on the link for Census Dates for Countries and Areas of the World: 1945 to 2014 this will take you to the US Census Bureau s resources on International census data. 3. This page provides general international population and census data for review and download, as well as links to additional resources. Note when this site was last updated. Click on the link for Africa under Tables 1

2 of census dates by region. Here you can see the dates that censuses were conducted in each of the African countries listed. Find South Africa and determine when their most recent census was. 4. Return to the Tufts GIS website, International GIS Data Sources page ( and select United Nations Statistics Division, 2010 World Population and Housing Census Programme. This is another site that details the dates of censuses. 5. Return to the Tufts GIS website ( and select Index of International Statistical Agencies this time. 6. We re going to look at South Africa, so find and click the link for this country s data in the list this will take you to the Statistics South Africa page. To analyze census information in GIS, we need to gather both geographic and tabular data. We will start with tabular data. Tabular Census Data 1. Unlike the geographic boundary files, tabular census data is available for direct download from the Statistics South Africa website. 2. Return to the agency s homepage ( At the bottom of the page, under Tools, click on SuperWeb. 3. In the menu on the left, expand Community Profiles, Census 2011, and Family. 4. Click on the lowest level Family to bring up a new menu of fields. This is where you will build a table of data of interest for download. 5. We will look at monthly household incomes at the municipality level. Expand Geography so that the provinces are visible. Now expand each of the provinces so that the districts are visible and further expand each district to show the municipalities. 2

3 Province District Municipality Once ALL the municipalities in each province are visible, scroll back up to the top and check the box beside Geography. This selects all the boxes that are open under Geography. 6. Each row in the table will represent a municipality, so click the Row button at the bottom left to populate the table with your municipalities. If you have made any errors, you can use the Clear Table button or use the row/column swap button in the lower right. Swap row/column 3

4 7. Next, minimize Geography and uncheck the box next to it. Expand Individual monthly income. This shows all the different categories of income and we will want to add all of them to the table as columns. Note you can click the (i) icon beside many of the data types in order to get more detailed information about the census data. 8. Check the box next to income and then click the Column button in the lower left to add it to the table. You should now have a table on the right that has geography and income. Save your table as an Excel file by clicking on Save As in the top right of the page and selecting the proper format. Then open the file in Excel and take a look at it. 4

5 2001 Tabular Census Data 1. For the remainder of this exercise we are going to use 2001 census data. Some 2001 census data on household income has been previously downloaded from this service and placed in S:\Classes\DHP_P207\Census Exercise Materials\. Copy the folder and paste it to your H: drive. In your H drive, right click on the folder and open the properties. Uncheck read only. 2. Open the file SAfrica_HH_income.xlsx and familiarize yourself with the data. What data is here? Is there metadata? Geographic Data 1. The spatial data that we need, the geographic census boundaries of South Africa, are not posted on the Statistics South Africa website. This data was previously requested by Tufts and is saved on the M: drive. It took about three months to receive this data. 2. Go to M:\Country\SouthAfrica. Here you will find several data layers for South Africa which were requested and received from Statistics South Africa. 3. Find and open the text file South_Africa_GIS_metadata.txt. This file contains information about the contents of the shapefiles in this folder. 4. From your H drive Census Folder, open the ArcMap file SAfrica_census_tutorial.mxd. Several of the data files you just browsed on the M: drive have been added to this map and were renamed to be more descriptive of their contents. Each of the polygon layers contains information for different geographic units in South Africa. 5. Take a few minutes to explore the data layers by turning on the map layers and viewing the Attribute Tables. 6. Use the Attribute Tables and data layers to determine the relative scales of the polygon layers and reorder them in the Table of Contents so that they are arranged from smallest to largest geographic unit. Joining Data 1. Back in ArcMap, turn on (check) the Municipalities_SA layer in the Table of Contents. Open the Attribute table to view all the data contained in this layer. 5

6 This layer contains measurements and identifiers for geographic municipal data. To include more detailed information in our map, we will join this layer of geographic information with the income data file. 2. In order to join data to an ArcMap session it must be clean and in a specific format. First, copy the excel table information and paste it into a new excel workbook. All titles, table footnotes and extraneous data should be removed from the Excel file. The first row should be the column field name for the data you are importing. Column field names must be unique. Column field names can be no longer than 8 characters. Column field names cannot be empty and cannot contain spaces. Column field names cannot contain strange characters such as -,*, $, etc. Underscores are okay. NOTE: The first 8 rows of data values will determine field type when brought into ArcGIS. If there is a mix of numbers and text, the field type will become TEXT in ArcGIS. 3. Edit the Excel file per the rules above. A suggestion for the income table is to re-label the fields as follows: Name the first column muni No income no_inc R1 R400 R400 R401 R800 R800 R or more more Collective living quarters coll_liv Make sure there are no leading or trailing spaces in your column field names Save your new Excel file as SA_HH_clean.xls. 4. In order to join the shapefile and Excel data, the files need to share a common unique identifier field. In looking at the municipality shapefile and the Excel tabular file, what common fields exist in the two disparate datasets? Note that the field names can be different, but the contents need to be identical. Write these down. 5. We find that the MN_MDB_C field in the Municipalities_SA.shp Attribute Table and the muni field in the SA_HH_clean.xls match. Note these two fields and close the Excel file and Attribute Table. Turn off all layers in ArcMap except for Municipalities_SA to speed up processing. Note: You cannot have ANY tabular file open in Excel when attempting a join in ArcMap. Make sure to close Excel. 6. Right-click again on Municipalities_SA. Select Joins and Relates Join. 6

7 7. Select the fields and data in the Join Data window as shown below. (1) The field is the unique ID from the layer attribute table. (2) Click on the yellow folder to the right as illustrated in the second image below. Navigate to the Excel file in your H: drive and select the sheet within the excel workbook (usually Sheet 1). (3) This is the unique ID in the Excel file. (4) Select Keep All Records and click OK 8. Reopen the Attribute Table for Municipalities_SA.shp. Scrolling to the right, you should now see all of the data that was in the Excel files appended to the Attribute Table. What data did not join? 9. As it is right now, ArcMap is pulling the income data from the excel file. To preserve the join and make the excel data remain permanently in the Attribute Table, right-click on the layer and go to Data Export Data. 10. In the window that pops up, select this layer s source data. Navigate to your H: drive and save the output as a shapefile named Municipalities_SA_join in the census folder. Click OK. 7

8 11. When asked if you want to add the new layer to the map, select Yes and Municipalities_SA_join will appear in your Table of Contents. 12. The income data that we have only provides the number of households in each income category. To make the mapping analysis more useful, we need the total number of households in each municipality so that we can normalize the data. To determine this, open the Attribute Table of Municipalities_SA_join and click the Table Options Dropdown Menu on the upper left of the table and click Add Field. In the window that opens, name the field tot_hh and select Long Integer, click OK. 13. Scroll all the way to the right of your Attribute Table and you will see a new field with the heading tot_hh at the end. Right-click on this field heading and select Field Calculator. Click yes if an error pops up. 14. This window allows us to calculate values based on the other fields already populated in the Attribute Table. Sum all of the income range fields by adding them to the box under tot_hh. Double-click on the fields and plus sign to add them to the query box. 8

9 15. Click OK, the window will close and the tot_hh column will populate with the sums of the income fields for each municipality. Defining Symbology 1. Right-click on Municipalities_SA_join and select Properties. 2. We want to map the percentage of all high income households of each municipality. Navigate to the Symbology tab, select Quantities Graduated colors. 3. Under the Fields section, select More in the Value dropdown menu. For Normalization select tot_hh. This will allow us to create a map where each census municipality is colored according to the percent of households with incomes greater than R 204,800 per month. 4. Click OK. You can refer back to the Learning ArcGIS Basics (India) for an additional example of setting symbology. Additional Census Data Exploration 1. Now that you ve gone through the process of finding and mapping one category of census data, let s look at another one. A second table of census data, called SAfrica_HH_goods.xls has been downloaded from the Stats SA website and placed in the folder you copied to your H: drive. 2. Open this file. What formatting needs to be done to join this file to the Municipalities_SA shapefile? 3. Formatting this data is more complex than the income data because the codes and name of the municipality are in the same field. We need them to be separated in order to have a unique ID, which will then match a unique ID field in the shapefile. 4. Open the SAfrica_HH_goods_clean.xlsx file to review a version of the file formatted properly for uploading to ArcMap. 5. Close the excel file. 6. In ArcMap, right click on Municipalities_SA Joins and Relates Remove Joins sheet1$. This removes the previous join, which is no longer needed since it is part of the attribute table in Municipalities_SA_join. 7. Follow the previously outlined steps to join and map the household goods data to Municipalities_SA in a similar manner to the income data. 8. Use symbology to display information in this attribute table. Take some time looking at how the data is displayed. Map Layout 1. Now that we have the data that we need in the map, we can edit the layout. Choose View - Layout View from the main menu. 2. The view changes to show your data frame on a page layout and a new toolbar appears - the Layout toolbar. 9

10 3. The first thing you should do is to set up your Page properties. Choose File Page and Print Setup. 4. In the Page Setup dialog box, make sure that the page size is set to Letter. Also check either Portrait or Landscape (which would be better for the map you want to create?) Press OK to return to the map. 5. Arrange the data frame on the map so it fills out the whole page. 6. Add a Title, Legend, North Arrow and Scale Bar to your map. These are all usually required elements on a map. You access them by going to the Insert item on the main menu bar. See the chart below for some details on adding map elements. Insert Scale Bar Insert Legend Modifying an element after insertion The scale bar you select will be in the display units of your data frame (e.g., feet). If you want a different unit, click on Properties in the Scale Bar Selector dialog box, and select a different Division Unit. There are lots of ways to improve your legend - we will work on a Legend Tips section of its own. For now, just experiment with changing the number of columns so that the legend box fits well on your map.you may not need to list all of your map layers in the legend. For example, water is usually identifiable on its own. Roads may be as well; however, if they are symbolized by type (ie highway versus street) they need to be included in the legend. Note that in the Legend Wizard you can control what items go into the legend, and the order in which they are listed. You can modify a legend by double-clicking on it to bring up the legend properties. You can usually modify an element after you have inserted it by selecting it and then double-clicking on it. The relevant dialog box will appear for that element Again, the Learning ArcGIS Basics (India) tutorial provides additional details regarding set up of map layouts. 10