Geospatial Day II Introduction to ArcGIS Editor for Open Street Map Geospatial Operations Support Team (GOST) Katie McWilliams kmcwilliams@worldbankgroup.org GOST@worldbank.org 0
Using OSM for Network Analysis Creating a topologically-correct road network is generally a time consuming and complicated task. Recent improvements in open source road data from Open Street Map (as well as open source analytical tools that utilize such data) have drastically cut down the amount of time and effort necessary to produce these networks. The following training guide will outline the steps necessary to build a network in ArcGIS using OSM data. To complete the exercise, you must have ArcGIS 10.3 or higher, access to the Network Analyst extension, and have installed the ArcGIS Editor for OSM toolbox. Open Street Map: OSM is a free, publicly available global data set. While it is primarily known for roads and transport data, it also includes points of interest, buildings, natural features, and land use information. Because the data is crowd-sourced, the coverage varies greatly from country to country and between urban and rural areas. While some places have incredible amounts of detail and accuracy, other locations do not. Always check for the level of coverage in your area of interest before using OSM for analytical purposes. https://www.openstreetmap.org/about ArcGIS Editor for OSM: a free, open source set of tools that can be added as a toolbox directly to ArcGIS. These tools load an OSM file and create a network without forcing the user to spend time cleaning up the raw data. For more information, please visit: http://www.esri.com/software/arcgis/extensions/openstreetmap Geofabrik: while data from Open Street Map is open source, downloading it directly from OSM s website is difficult. Geofabrik is a company that provides Open Street Map data available for download as either a native-format.osm or as a shapefile (available for most, but not all, countries). For more information, please visit: https://www.geofabrik.de/geofabrik/geofabrik.html Note: analytical results are only as good as the input data. Keep in mind that open-source data is never perfect, and will have flaws that can affect the results. It is always necessary to weigh the pros and cons of using such data. EXERCISE OBJECTIVES: 1. Download OSM data from Geofabrik 2. Load the OSM file and build a network using the OSM toolbox 3. Run a sample travel-time analysis PART 1: Downloading OSM data from Geofabrik 1. In your web browser, navigate to http://download.geofabrik.de/
From the home page, you can download the OSM file for an entire region (by clicking any of the links under the.osm.bz2 column). Note: raw OSM files are extremely large- downloading a file for an entire region is not a task for a standard Bank computer. 2. Click on the Central America sub region and note which countries are available. Then, click on the Belize.osm.bz2 file to initiate the download. 3. When prompted, save the zipped file to the desktop. (Or drag the file to the desktop from the downloads menu.) 4. On the desktop, right-click on the file Belize-latest.osm.bz2 and select WinRar > Extract To. A new folder will appear on the desktop with the unzipped OSM file. Note: Depending on the size of the country, downloading and unzipping the OSM file could take a while. Given time constraints, Belize was chosen for this lesson because of the small file size. PART 2: Loading the OSM file in ArcGIS and building the network 1. Open ArcGIS by going to the desktop Start menu and typing ArcMap into the search bar. Click on ArcMap 10.5.1 (or whatever version is installed) to open the program. 2. In the Getting Started dialogue box, click cancel to open a blank map. The OSM editor tools are dependent on the functionality of the Network Analyst extension within ArcGIS. Before we begin, make sure that the Network Analyst extension is enabled. 3. In the top menu bar, go to Customize > Extensions. Click the box next to Network Analyst to activate it. Then close out of the extensions screen.
Tools running in ArcGIS process in two ways: in the foreground and in the background. Background processing allows the user to continue working within the platform while a tool is executing. During foreground processing, however, the user must wait until the tool finishes before continuing with any other tasks. There is a known issue with the OSM editor tools that requires background processing to be disabled. If not, the platform will crash while the network is being built. Background processing must be disabled before beginning. 4. Under Geoprocessing in the top menu bar, select Geoprocessing Options 5. In the Background Processing section, uncheck the box next to Enable, then click OK to exit the menu. Now that Network Analyst is activated and Background Processing has been disabled, we are ready to access the OSM tools. 6. Open the ArcToolbox by clicking on the red box symbol in the top menu bar. 7. If the OSM toolbox is not already visible, it must be added manually. To do so, right-click on the ArcToolbox and click Add Toolbox.
8. Under the Look in drop-down menu, select Toolboxes, then double click on System Toolboxes. 9. Click ONCE on OpenStreetMap Toolbox.tbx. When it appears in the Name field at the bottom, click Open. The OSM toolbox should then appear in the list of active toolboxes on the ArcGIS display. The OSM toolbox contains a variety of tools that help process raw OSM data. To build a network from Geofabrik data, we only need two: Load OSM file and Create OSM Network Dataset. For more information on the rest of the tools, visit https://github.com/esri/arcgis-osmeditor/wiki/documentation. 10. Inside the OSM Toolbox, double click on Load OSM File. 11. Click on the folder to the right of the OSM File dialogue box, and navigate to the desktop. Then open the belize-latest.osm folder.
12. Double click on the belize-latest.osm file to add it to the dialogue box. 13. Make sure the Conserve Memory box is checked. An OSM file can only be loaded into a geodatabase. Before we can fill out the Target Feature Dataset box, we must create a geodatabase to use. 14. Leaving the Load OSM File tool open, click on the Catalog button in the top menu to open ArcCatalog. (Alternatively, go to Windows > Catalog.) 15. In the top menu bar of the Catalog window, click on the Connect to Folder button (a folder with a black plus sign). 16. In the dialogue box that opens, navigate to the desktop and click OK. 17. In the ArcCatalog window, click on the black plus sign next to Desktop under Folder Connections to expand it.
18. Right-click on the belize-latest.osm folder, and go to New > File Geodatabase. 19. Change the name of the new database to OSM_Network.gdb. 20. Close out of ArcCatalog.
Now that we have created the necessary geodatabase, we can resume loading our OSM file. 21. In the Load OSM File tool, click on the folder to the right of Target Feature Dataset. 22. Using the drop-down menu next to Look In, navigate to Desktop > Belize-latest.osm. 23. Double click on OSM_network.gdb. In the empty name field at the bottom, type OSM_Roads and click Save. 24. When your dialogue screen looks like the one below, click OK to begin run the tool. NOTE: The Belize road file has over 500,000 nodes (endpoints and points of intersection) and 67,000 ways (lines) that need to be loaded. Depending on the processing power of the computer, this step could take anywhere from 10 to 15 minutes (on average). Because we have disabled background processing, we must simply sit and wait for the tool to finish executing. 25. When the Load OSM file completes, click close to exit the tool. 26. In the table of contents on the left, uncheck every box except the one marked OSM_roads_osm_ln. (You might see two line layers with the same name- if so, just leave one visible.)
27. Right click on the layer name and select Zoom To Layer to fit it in your display. Note: Open Street Map includes other data beyond roads- notice the oddly placed lines extending well beyond the country borders, or other lines that clearly are not roads. These are maritime and administrative borders (feel free to use the Identify tool in the top menu bar to explore the outliers). Keep in mind that only lines classified as roads in the OSM data will be used in the network analysis. 28. If you d like, zoom in further to see the roads in closer detail, or play with the symbology to highlight different types of lines. (Symbolizing the highway column will allow you to see the different types of roads.) Now that the OSM file is loaded, we can use it to build the actual network.
29. Navigate to the Create OSM Network Dataset tool within the OSM toolbox, and double click to open it. 30. Click on the folder to the right of Input OSM Feature Dataset. In the drop-down menu, navigate to the OSM_Network geodatabase saved inside the Belize-latest.osm folder on your desktop. 31. Double-click on the geodatabase, then click once on the OSM_roads feature dataset. Click Add. 32. Click on the folder next to Network Configuration File. The Network Configuration File essentially defines the rules for which travel can be accomplished across the network. This includes speed limits, turning, etc. The OSM network comes with a native Configuration File- but you can also create a custom file. The instructions for doing so can be found here: https://github.com/esri/arcgis-osm-editor/wiki/generate-a-custom-network-configuration-file-script. For the sake of time, we will be using the native network configuration file. 33. Using the drop-down menu, navigate to the location where the toolbox itself was installed: C: > Program Files (x86) > ArcGIS > Desktop 10.5 > ArcToolbox > Toolboxes > ND_ConfigFiles. 34. Inside the folder, select DriveGeneric.xml and then click Open. (Note: your Desktop version of ArcGIS might be different.)
35. The final section, Output Network Dataset, should populate automatically. Make sure that the network will be saved in the OSM_Network.gdb, then click OK to run the tool. Note: Depending on processing power, it should take between 2 and 5 minutes to create the Network Dataset. Part 3: Running a travel-time analysis To test our OSM network, we will finish the exercise by performing a sample Service Area analysis, using the tools in the Network Analyst extension. NOTE: if you are familiar with the functionality of the Network Analyst tools, and prefer to run a different test (such as an OD Matrix) feel free! In the
following example we will use the road network to create polygons (drive time sheds) that show areas within 30, 60, and 90 minutes from 8 cities in Belize. 1. In the Table of Contents, right-click on the OSM_roads feature class to remove everything from the map. 2. Click the add data button in the top menu bar, and then navigate to the OSM_Network.gdb. 3. Double click on the OSM_Roads feature class, then click OSM_Roads_nd to add the network dataset to the display. Click yes when prompted. Note: Clicking yes adds all feature classes related to the network to the display (junctions, barriers, roads, etc.). If you prefer, uncheck the boxes next to every layer except Roads to then them off. Also notice how only the roads, and not the administrative/maritime borders from the original OSM file, are part of the network.
4. Click the add data button again and navigate to the BelizeCities folder on the desktop. 5. Add the Belize_Cities shapefile to the display. This shapefile contains location information for 8 of the largest cities in Belize. 6. Under the Network Analyst toolbar, select New Service Area. (If the toolbar is not visible, go to Customize > Toolbars > Network Analyst).
7. Make sure that the Network Analyst Window is visible to the left of the regular Table of Contents. (If it isn t, click on the button in the Network Analyst toolbar that looks like a table with a blue flag.)
8. Right-click on Facilities in the Network Analyst Window, and select Load Locations. 9. In the Load From drop-down menu, make sure Belize Cities is listed. 10. Click ok to load the cities as the target locations for the analysis. Note: When point locations are loaded into Network analyst, they are either located or unlocated along the network. Whether they are located or not is dependent on the distance threshold set in the Load Locations settings (see the red box in the screenshot above). In the example we just completed, the distance threshold was set to 5,000 meters- and since all eight cities were located, it means all cities are within 5,000 meters of a road. If you ever encounter locations that are unlocated, you can either a) change the distance threshold or b) edit the point shapefile to move the point closer to the road network. 11. In the main Table of Contents, right-click on Service Area to open the properties.
The Service Area properties dialogue box contains numerous tabs which allow you to customize your analysis. For this exercise, we will create polygons the outline 30, 60, and 90 minutes of drive time from each city. 12. In the Analysis Settings tab, make sure that the drop-down menu next to Impedance says DriveTime (minutes). (The other option is distance- this would create polygons that outline certain distances from the city points.) 13. In the Default Breaks box, enter 30, 60, 90. This will create polygons which show 30, 60, and 90 minutes of drive time from each city point. 14. In the Polygon Generation tab, make sure that Generalized and Trim Polygons: 100 meters is set, and under Multiple Facilities Options the Overlapping button is chosen.
15. Click OK to exit the properties. 16. To solve the network analysis, either chose the Solve button in the top menu bar, or right-click on Service Area and choose Solve. If the resulting polygons are not as pretty as you would prefer, go back to the Polygon Generation dialogue box and play with the options to adjust the output. Change a few settings and then click solve to see the new output. After you ve tried a few variations, end with Generalized polygons that are not trimmed (the box is unchecked).
Congratulations! You have now successfully downloaded an OSM file, used the OSM Network Tools to load and build a network, and ran a sample Service Area analysis. For the remainder of the session, feel free to play around with the data or try other Network Analyst tools. Suggestion: in the service area you just ran, change the polygon settings so that they are detailed and untrimmed. Solve the analysis again, then zoom in to some of the holes and other oddly-shaped polygon results. What causes this? What do these results say about the input OSM file?