GIS Virtual Workshop: Geocoding

Transcription

1 This module will deal with the concept of geocoding of data. Geocoding is placing a data point which does not have a X, Y reference (no Latitude and Longitude) and automatically mapping it by knowing the address of the information. The process of geocoding is relatively easy if the input data is composed properly. What is more time consuming is fixing the data file to meet the standards of ArcMap and manually coding data points that are not automatically coded. The initial data can have many different formats, such as a text file with commas as the delimiter or a Microsoft Excel file. If the information is presented as an Excel file it cannot be in 2007 format, but needs to be in an early format. You can directly use the spreadsheet or convert it into a text file either way will work. None of the field names can have a space between words, if they do they will need to be modified, it suggested to replace the space with an underline symbol. Most of this module will deal with managing the data more than geocoding. This module will not deal with the construction of a locator file, but instead will use the StreetMap USA file from ESRI. This file should be part of the software from ESRI, it will not be distributed as part of this module, other data files can be downloaded on the module website. Make sure StreetMap USA is loaded on your computer harddrive. This module assumes that you have developed basic GIS skills and that you can generate a map like the one displayed to the left. The starting point map is the same as the one used in the buffering module, if you need to reference the buffer module please do so before continuing with this module. It is also assumed that you can do basic Microsoft Excel. From the web component of this module you can download the data set displayed to the left. This is a map of the Kentucky Counties, this data set can also be used from the ArcIMS server located at: If you need assistant in loading data from a server please see the module on using online data or the module on using the catalog functions. As with many operations in ArcMap there usually are more than one way to complete the operation, in this module we will only discuss one way in which to do each operation. 1 3/18/2008

2 First we will need to work with the data that will be geocoded. Data is provided on the website for data to be used in this module. If you wish to use data other than that provided, the data must be in Microsoft Excel format version or a comma delimitated text file. For this module we will assume it in Excel format. The data must include at least a name, address, city, state and five digit zip code. The data that is included with this module is a directory of doctors in Kentucky, this data may not be used for any purpose other than this module. The name of the fields must have no spaces, if there are spaces in the field names they must be eliminated. The file cannot be in Excel 2007 format. If the zip code is nine digit format they must be changed to five digit zip codes. The screen shots in this exercise will be from Excel 2007, but saved in 2003 format. If you are using a version of ArcMap of version 9.1 or older, you should use text files and not Excel. This workshop will assume that you are using version 9.2. The files are available to you for the state of Kentucky on the website. Some of the files are also available on the ArcIMS server including the Kentucky county file and other Kentucky file as well as similar files for other states. The next step is to open a state map with counties. Besides the state shapefile with counties, load zip code boundaries, state boundaries, and major roads. While some of theses layers will not be used directly they may be useful as you extend beyond the scope of this module. Other layers can also be loaded that you may have created in other modules or is appropriate for your data set if you are not using the information contained on the website for this module. Turn off all layers except the county shaepfile. When completed you should have a map that looks like the one on this page. It is assumed that you are using the information provided on the website, if you are using other data of course your map will look different, this distinction will not be referenced in the future. Therefore, you should have a state map showing counties at this point. 2 3/18/2008

3 The next steps will be to geocode the data file against the StreetMap USA file. Geocoding works by comparing the datafile containing street address, city, state and zip code to a locator file. Once the street is located in the locator file, the location of the individual is determined as a ratio of the street address compared with the length of the street and address range. For example if the address is 445 W. Maple St in a given city, if this street is found in the database with a length of.2 mile and an address range 401 to 451 odd numbers, it will assume that 445 is 90% of the way down the street and place the residence on the proper side of the street. 90% down the street would be.18 mile from the 401 end of the street. In the gecoding process you can set the tolerance of the coding that will be displayed and you can also manually match addresses. Note post office boxes cannot be coded. In general if you code to 85% you have done very well. Usually new streets do not code properly because they are not in the database, typographical errors are also a major source of error, as is putting apartment numbers in the address line. In general rural areas are harder to code than urban areas. Open the Excel and the spreadsheet so that you can explore what the data set looks like. This image represents only a part of the Excel file of doctors. Note an underline has been used instead of a space. You must replace all spaces or the operation will not work properly. Note some of the doctors in this screen shot have post office boxes and when we do the geocoding process they will not code properly. Once the Excel file has been save in format and closed we are ready to geocode the data. Open ArcMap to continue this module. 3 3/18/2008

4 Click on Tools and then on geocoding and select geocode addresses. 4 3/18/2008

5 You need to tell the program the location of the locator file, click on add and you will need to define the pathway to StreetMap USA. This file should be part of the install package with ArcMap. Once the locator file path has been added the window to the right will open. You need to select the location of the data file (Excel file). Use the open file folder symbol to locate the file. If you had added the file to the map you could use the pull down arrow. 5 3/18/2008

6 Make sure that the names for address, city, state and postal code are the proper ones from the Excel file, if they don t auto match use the pull down to located the proper name. In this example the address name had to manually identify. The output file must have an appropriate name and storage location. Use the lower open folder in the output section to select a proper name and storage location. At this point the program will begin the geocoding process. Depending on file size and the speed of the computer, will determine the amount of time for coding. The example file is a very large data set with several thousand of records. 6 3/18/2008

7 Once the process has been completed a results window will be shown, in this particular example 65% was geocoded with accuracy between 80 and 100, but 28% were not coded or another way to think of this is that 72% were coded and mapped. We will look at how to manually code as part of this module, but first lets see what the results looks like. Click on Done to show the map of the results. Note a new layer was created in this process As can be seen in the map below, there are areas of concentration of doctors and areas with few doctors, the map is a little misleading because many doctors offices contain multiple doctors and on the map they show up as a single point since they are plotted on top of each other. Click on tools, then geocoding and finally on Review/ Rematch Addresses and that will show the geocoding window. 7 3/18/2008

8 We can change the threshold for geocoding, the default is 70%. Depending on the purpose of the map you might be willing to accept a lower threshold. Click on Geocoding Options. Change the minimum match score to a lower percentage, in this exercise we will use 55%. Then click OK and it will display the Review/Rematch Addresses window, click Match Automatically. When complete the Review/Rematch Addresses window will appear. Compare the percentages in the window above to the newly displayed window. 8 3/18/2008

9 The completed results are displayed to the left. Note we now have only 21% of the data not matched or 79% matched. That is still 1559 candidates not matched. While those reporting post office boxes cannot be matched, there may be others that can be matched by looking at the addresses, you may find mistakes in spelling, suite numbers or apartment numbers included in the address line or listed on the wrong side of the street. This is a slow process at this point to match the data manually. Click Match Interactively. 9 3/18/2008

10 The window above shows how the interactive match window appears. In the image above we have scrolled down several records from the top. The address is 230 Yeager Ave, if look at the first line of data you wonder why the software didn t code this address since the range is from 200 to 301, look to the left side of the data window and you will see it says it is on the Right side, but it is an even address and should be on the left. Thus the score was only 48. Well if we are looking at state wide or county wide data that is most likely close enough and we should accept it, if we are looking at the street level than that may not be close enough and it maybe should be rejected. Click match if you wish to accept the point or just go to the next record. You can also modify data by changing it in the Address window or by using the modify button. This is important when the data contains apartment numbers or in this case suite numbers. 10 3/18/2008

11 Once you have completed the geocoding process you may want to classify the types of doctors which is also included in the data set. You may have done this in other modules, but maybe not with point data. Again remember the results will not show differences when one dot (doctor) is directly on top of another, i.e. doctors share the same office building. Therefore this map is not definitive map, but shows potential trends. Open the property window and click on the symbology tab. Select the Categories and Unique Value, next change the Value Field and select Specialty. Then you will need to select Add All Values, you will need to either accept the ramp color or change it and then apply it to your map. 11 3/18/2008

12 A part of the legend is visible to the left and the map has been zoomed to show more details, the dots are several different colors and not just one color as before. This completes the module on geocoding, you should be able to load an Excel spreadsheet, make modifications to the file, then use a standard locator file to code the data file. In addition you should understand threshold levels and how to manual recode information. 12 3/18/2008