Finding GIS Data and Preparing it for Use

Transcription

1 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 1 of 16 Getting Ready for the Tutorial Sign Up for the GIS-L Listserv Finding GIS Data and Preparing it for Use The Yale University GIS-L Listserv is an internal University message system, devoted to GIS use and issues at Yale. The list is the primary source for updates on software and license server issues, data acquisitions and workshop announcements. All GIS users are encourages to sign up for the list. The GIS-L list is very lightly used, so that you will not be overwhelmed with messages that are not relevant to you. 1. In your Web Browser, Go To the Map Collection Website, at 2. Under the Quick Links section on the right,, Click on the Sign up for the Yale GIS-L Mailing List link to go to the GIS-L registration page. 3. Enter your Address and Choose a Password. Note: We suggest that you not select the Daily Digest feature. 4. Click on the Subscribe Button to finish. You should receive a confirmation shortly afterward, with instruction on how to use the GIS-L list. Setting Permissions in Internet Explorer 1) First, if you are using Internet Explorer, you must add the census sites to your trusted sites list to allow downloads and pop-ups for this exercise. 2) In Internet Explorer, go to Tools>Internet Options and click on the Security Tab. 3) Highlight the green Trusted Sites icon, and click the Sites button. 4) Uncheck the Require server verification radio button. 5) Add *.census.gov *. usgs.gov *.esri.com to the Trusted Web Sites list 6) Click OK twice to exit the Security setting dialog. 7) On the Main Menu, go to Tools>Pop-Up Blocker>Turn Off Pop-Up Blocker.

2 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 2 of 16 ESRI Data & Templates Default Template Sets 1. Start ArcMap from the Start>Programs>ArcGIS Menu. 2. When prompted, Select Start Using ArcMap with: a Template. 3. Select the USA Tab in the resulting Template Selector Dialog. 4. Select NortheasternUSA.mxt. 5. Click Ok to Open the Template. 6. Select the Layout Zoom In Tool and use it to Zoom Into The State Of Connecticut. 7. Save your Map Document (.Mxd) to your C:\temp\your_initials\02- Finding_Data_and_Preparing_It\ Folder, naming it Finding_Data.mxd. 8. Switch to Data View, using the View Toolbar, at the bottom left corner of the Map Layout Display Window. 9. Right-Click on the Data Frame Title, Northeastern United States and Open the Properties Dialog Box. 10. Select the Coordinate System Tab and Use the Select A Coordinate System Panel to Browse to Predefined>Projected Coordinates Systems>State Plane>NAD 1983>NAD 1983 StatePlane Connecticut FIPS Click Ok to Apply the New Coordinate System to the Map Document. Note that this Coordinate System has been applied to the Map Document, rather than the Layers within the document, whose coordinate/projection systems remain. ArcMap will now Display these Layers, and all subsequently added Layers, using the State Plane Coordinate by applying projection on the fly.

3 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 3 of 16 Census Data Download and Import into ArcGIS Downloading Census Boundary Files from ESRI 1. Open your Web Browser and Go To Google.Com. 2. Enter Esri and Census as your Search Terms and Click the Search Button. 3. The first link returned should be Download Census 2000 Tiger/Line Data. Click on this link. 4. From the Select A State: Drop-Down, Select Connecticut and Click on the Submit Selection Button. 5. On the resulting page, Select New Haven from the Select By County Drop-Down and Click the Submit Selection Button. 6. On the resulting page, Select the Checkbox for Block Groups Scroll to the bottom of the page and click on the Proceed to Download Button. 8. On the resulting page, Click on the Download File Button. 9. Save the resulting file to your C:\temp\your_initials \Data\Shapefiles Folder. 10. Browse to the C:\temp\your_initials \Data\Shapefiles Folder. 11. Right-Click on the Downloaded *.Zip File and select Extract All from the Context Menu. 12. Accept the defaults for Unzipping the file to a folder called C:\temp\your_initials\Data\Shapefiles\. 13. This will result in another zip file, grp zip, which will have to be extracted. Use the same procedure to extract the files to the C:\temp\your_initials\Data\Shapefile Folder. 14. Return to ArcMap and Click on the Add Data Button. 15. Browse to the C:\temp\your_initiials\Data\Shapefiles\tgr09009grp00.shp File and Add it to the Map Document. 16. You will be presented with an error message indicating that the coordinate system of this file is not yet defined. Click Ok.

4 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 4 of Open ArcToolbox using the ArcToolbox Button. 18. Click on the Search Tab at the bottom of the ArcToolbox and Enter Define as your Search Term. 19. Click Search. 20. Double-Click on the Define Projection Tool from the Data Management Toolbox to Open it. 21. Select the tgr09009grp00 layer from the Input Dataset Drop-Down. 22. Click on the Open Properties Button to Open the Spatial Reference Properties Dialog Box. 23. Click on the Select Button and Browse to and Select the Geographic Coordinates Systems>North America>North American Datum 1983.Prj Projection File. 24. Click Add. 25. Click Ok. 26. Click Ok to Define The Projection for the tgr09009grp00.shp shapefile. 27. If you do not see the Tgr09009grp00 Layer overlay properly, try using the Refresh Button, found on the View Toolbar at the bottom left corner of the Map Display Window.

5 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 5 of 16 Downloading Census Attribute Tables from the American FactFinder Website Now we will use the American FactFinder to create a dataset with several variables of interest. 1. Open the IE browser and go to 2. GO to the American FactFinder Homepage 3. Under Getting Detailed Data Find the Decennial Census item and Click Get Data. 4. Scroll Down and Check the Census 2000 Summary File 3 (SF 3) - Sample Data Checkbox. 5. Click on the Detailed Table Link. 6. On the resulting page, Click the Geo Within Geo Tab. 7. Set the Selection Method to: Show me all: Block Groups Within: County Select a state Connecticut Select a county New Haven 8. In the last Menu, Select All Block Groups and Click Add. 9. Once All Block Groups have been added, Click Next. 10. From the list of available attributes, Select those of interest (use the CTRL key to select more than one). 11. Click Add. 12. Click Show Result. 13. At the top of the next page, Click on the Print/Download>Download Link. 14. A Pop-Up Window will be presented, with option for download (you will need to enable pop-ups for this function). 15. Scroll Down the page and find the item for Database compatible (data rows only) downloads. Check the Microsoft Excel (.Xls) Checkbox and Click the Download Button. 16. When prompted, browse to the C:\temp\Data\ folder, make a new folder called Tabular and Save the output.zip File in it. 17. Once the download has completed, Browse to the C:\temp\your_initials\02- Finding_Data_and_Preparing_It\Data\Tabular folder and Unzip the Contents of the file to the.\tabular Folder.

6 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 6 of 16 Preparing the Data to be Joined to the Boundary Files 1. Browse to the C:\temp\your_initials\Data\Tabular Folder and Open the dt_dec_2000_sf3_u_data1.xls file with Microsoft Excel. 2. On the Main Menu, Go To Insert>Worksheet. 3. Double-Click on the Sheet1 Tab, at the bottom of the New Worksheet, and Rename it Clean. 4. Return to the Original Worksheet, and Double- Click on the Tab and Rename it Orig. 5. Click on the Upper-left cell (directly above the row numbers) to select all records in the worksheet. 6. On the Main Menu, go to Edit>Copy, or use the Ctrl-C shortcut to copy the selected records. 7. Switch to the Clean worksheet, and click on the Upper left cell to select all cells in the worksheet. 8. On the Main Menu, go to Edit>Paste, or use the Ctrl-V shortcut, to paste the copied records to the Clean worksheet. 9. Right-Click on the 2 ant the far left of the second record in the Clean worksheet and select Delete to remove the entire row of descriptive names. 10. Save the changes to dt_dec_2000_sf3_u_data1.xls. 11. Close Excel.

7 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 7 of 16 Joining the Attributes Data to the Boundary File 12. Return to ArcMap. 13. Use the Add Data Button to Browse to the dt_dec_2000_sf3_u_data1.xls file. 14. Double-click on the dt_dec_2000_sf3_u_data1.xls file to Browse into it. 15. Select the Clean$ worksheet and Add it to ArcMap. 16. Right-click on the Tgr09009grp00 Layer and Select Joins and Relates>Join 17. In the Join Data Dialog, Set the following: Join attributes from a table based on: STFID table to join Clean$ field to base the join on GEO_ID2 18. Click OK. 19. Open the Attribute table for Tgr09009grp00 Layer to confirm the success of the Join. 20. Close the attribute table. 23. Browse to the C:\temp\your_initials\Data\Tabular Folder and save it as New_Haven_County_Block_Groups_Median_Household_Income_and_Race.shp. 24. Remove the Tgr09009grp00 Layer. 25. Remove the Clean$ Table. 26. Return to the Display Tab of the Table of Contents Panel. 21. Right-click on the Tgr09009grp00 Layer and go to Data>Export Data 22. Change the coordinate system as: to the data frame. 27. Right-click on the New_Haven_County_Block_Groups_Median_Household_Income_and_Race layer and Open its Attribute Table. 28. Confirm that the demographic variables successfully transferred to the new dataset. 29. Save you changes to the Map Document. You should now have successfully joined the downloaded Census attribute data to the boundary file downloaded from ESRI. If desired, you can apply Symbology to the resulting dataset for mapping the downloaded attributes.

8 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 8 of 16 Downloading Data from the USGS Seamless Data Distribution Website Downloading from the Seamless Data Distribution Website The Seamless Data Distribution Website is maintained by the USGS and is the primary clearinghouse for raster data produced by the USGS, including ortho-imagery, Digital Elevation data, mosaics of Landsat imagery and more. Once familiar with the interface for the Seamless site, you should be able to successfully navigate and use many other ArcIMS-based data sites, as they are based upon the same architecture. In this exercise you will download elevation data for the New Haven, CT area. 1. Go to the USGS Seamless Data Distribution Website at eamless/ and click on the View and Download United States Data, link at the right side of the page. Wait for the Map to load. 2. By default, the Zoom Tool will be active. Use it to zoom into New Haven, CT. This may take several Zooms. 3. On the right side of the Seamless Application, look for the Download tab and activate it. Zoom Define Download Area Download Tab 4. Scroll down the list (noting the available downloads), find the Elevation Group and expand it (if not already) by clicking on the triangle next to the word Elevation. 5. Make sure the 1 NED item is checked, and that no other items are enabled. Elevation NED is the National Elevation Dataset, which is a raster dataset that describes the elevation at any given point, at several different resolutions (in this case ~30 meters). This type of data is commonly referred to as a Digital Elevation Model, or DEM.

9 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 9 of Now click on the 1 NED layer name to open a description of the layer in a new window. 7. Enable the Define Download Area Tool, under the Downloads Menu and Drag a box across the city of New Haven to define area of interest (AOI). Note that the box will remain green unless the area you have defined becomes larger that the Seamless Site allows (you can request up to 1.6GB, in 100MB files at once). Once you release the mouse button, a new page will be opened in a new browser window (you may have to enable pop-ups for the site). This new window will provide some general info about the data you have requested (projection, bounding coordinates, cell size, etc ) as well as Download Link Buttons to begin downloading the data to your hard drive. 8. Click on the Download Button to post the data request to the server. Yet another window will be opened, which indicates the Current Status of your request on the server. 9. When the Save As dialog box opens, browse to the \Data\Raster folder to save the file. 10. When the download is complete, browse to find the ZIP file you just downloaded, right-click on it and Extract All. 11. The data will be extracted to a new subfolder called something like ned_ In the rest of this tutorial, this layer will be referred to as ned_#######. Applying an Appropriate Projection to You Data Data does not always come in a ready to use format. One common necessity is to apply a Projection to the data, in order to transform the spatial reference from angular Lat/Long coordinates to planar/linear units, such as feet or meters. Here, you will project your Digital Elevation Model to a projection that is appropriate for applying calculations that assume the data is recorded in linear units. 12. Open ArcMap. 13. Use the Add Data button to browse to the folder containing your elevation data. There should be two files there, one a polygon shapefile named METADATA.shp, (which contains the footprint of your elevation data, and its metadata), as well a raster layer with a name similar to the folder that contains it.

10 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 10 of Select the ned_######## layer and add it to ArcMap. 15. Right-Click on the ned_######## layer and select Zoom to Layer. 16. If necessary, Drag the ned_######## layer above you polygon layers, so that it is visible. 17. Right-click on the ned_######## layer and Open its Properties Dialog box. 18. Select the Source Tab and scroll down to the Spatial Reference information. 19. Note that there is no Linear Unit assigned to this data. Scroll back up to the top and look at the Cellsize (X,Y) item. 20. This data has a Cellsize of about 30 meters, but here it is notes as This is because the data has not been projected and is currently spatially referenced in latitude & longitude coordinates, which are angular units of measurement. 21. Click OK to close the Properties Dialog. t in the \Data\Raster folder. Change the Z factor option to 3 (this exaggerates the elevation for a better visual quality). Click OK to apply the tool. When the tool is finished running, you should see a new layer in your Map View window. However, the effect is has produced is not very attractive. The Hillshade layer we have produced is very dark, and the topography it has created seems far more extreme than we might have expected. These poor results are related to what we observed earlier in the Spatial Reference and Cellsize of our Digital Elevation Model. Creating a Hillshade involves calculations that assume that the input parameters being used are in linear units, rather than the angular units that we currently have. This same problem would be true if we were to calculate slope, aspect and many other mathematical operations we might want to apply to this elevation data. What is necessary is that we Project our dataset from the current Latitude & Longitude Coordinates, which locate features on the face of the oblate spheroid that is the earth, to a projection that records our data in linear measurements, as if the earth were flat. 22. Open the ArcToolbox and search for Hillshade. Open the Hillshade tool and use your ned_######## Layer as the Input Raster. Name the Output raster hillshade01 and place i

11 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 11 of You should find that you now have a much more pleasant looking result from the Hillshade Tool. 23. Return to ArcToolbox and Search for Project Raster. Open the tool and select your ned_####### layer as the Input raster, Browse to your \Data\Raster folder and save the Output raster as ned_proj. For the Output Coordinate System, Click the Properties Icon to open the Spatial Reference Properties Dialog Box. Click on the Select Button and Browse to Projected Coordinate Systems>State Plane>Nad 83> NAD 1983 StatePlane Connecticut FIPS 0600.prj. Click Add. Click OK. 24. ArcToolbox adds the new layer to our Map Document. Right-Click on the new ned_proj layer and Open the Properties. Select the Source Tab and inspect the changes to the Cellsize and Linear Units items. 25. Use the Hillshade tool again, using the new projected elevation layer, to produce a new Hillshade Layer, called hillshade_02. Be sure to set the Z Factor to 3, like before. State Departments of Environmental Protection One of the best resources for obtaining GIS data, within the U.S., is to check what is made available by state Departments of Environmental Protection, or their equivalent. Here you will download data from the Connecticut Department of Environmental Protect and prepare it for use in your ArcGIS project. 1. Open your web browser and go to 2. Click on the Maps & Spatial Data link, at the right side of the page. 3. Click on the Spatial Data link at the left side of the page. 4. Click on the GIS Data link at the left side of the page. On the resulting page you are presented with dozens of layers of spatial data that are downloadable and available for use in GIS analysis. Each of these datasets also includes a mage of Metadata for the datasets. Find the 5. Find the Digital Orthophoto Quarter Quadrangles datasets and click on the dataguide link.

12 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 12 of 16 The resulting page provides information about the source of this data, its publication and collection date, update frequency, scale etc All of these pieces of information are critical to your understanding of your data and its relevance to your analysis. Always examine the metadata of any GIS data you obtain from a third party. If the Metadata is not available, you must seriously question the accuracy of the data and verify its relevance to your analysis, independently. 6. Return to the previous page, using the Back Button on your browser. 7. Scroll down the page and examine the available layers. All of the datasets available on this page are either compressed using the *.zip or self-extracting *.exe format. These files can be downloaded and added to your Map Document without processing other than projection to a coordinate system or projection suitable for your analysis. Connecting to Networked Data Resources in ArcMap The Geography Network ArcMap has built-in support for networked data sources. In fact, there is already at least one networked data source already available in the ArcMap program, by default. Here, you will connect to the Geography Network and add data to your Map Document. 1. Return to ArcMap and click on the Add Data Button. 2. Click on the Look In: drop-down list and select the GIS Server Item. 3. Double-click on the Geography Network Services hosted by ESRI Icon. 4. Scroll across the list of Data Services and Select the EPA_Hazards_FS. Click Add to open the data service in ArcMap. 5. You will be warned about a coordinate system that does not match that used in the Map Document. This is fine, for now, and you can click close to go ahead and add the data. Note that there are several types of Data Service provided from the Geography Network server. The one you have added is a Feature Service. This type of service allows you to work with the data as if it were on your own machine. You can change the symbology; view the attribute table, etc One thing you cannot do is edit the data, however. For this reason, you should probably subset the feature of interest using selection methods, and export the data to a new data layer on your own machine. Th3e Geography Network also provides access to Image Services that serve geographic data that has been rendered at the server, then sent as a snapshot to ArcMap. This type of

13 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 13 of 16 service is useful for adding Canned data to your map document for use as background. Finally, the Geography Network also contains a Metadata service, which you can connect to using ArcCatalog to examine and download the Metadata for the services available. 6. Right-Click on the Hazardous Waste Generator (EPA) Layer and Open the Attribute Table. 7. Note that the Feature Service allows you to examine the data as if it were on your machine. 8. Close the Attribute Table. 9. On the Main Menu, go to Selection>Select by Location. 12. Right-click on the Hazardous Waste Generator (EPA) Layer and go to Data>Export. 13. Check the data frame to use the coordinate system of the data frame for export. 14. Browse to the C:\temp\your_initials\02- Finding_Data_and_Preparing_It\Data\Shapefile\ Folder and save the export file as Hazardous_Waste_Generator_EPA.shp. 15. When prompted, Add the exported data as a layer in the Map Document. 16. Turn off the visibility of the EPA_Hazards_FS Group by unchecking its checkbox. 17. Save your work. 10. Use the Selection Settings: select features from Hazardous Waster Generator (EPA) that: intersect features in this layer: New_Haven_County_Block_Groups_Median_Household_Inc ome_and_race layer Apply a buffer of: 500 Meters. 11. Click OK to close the Select by Location dialog and Apply the selection.

14 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 14 of 16 USGS Seamless Data Distribution In the same way the Geography Network makes data available over the Internet, other data sources provide data services that are accessible in ArcMap. One of the most useful is the USGS Seamless Data Distribution service. This service provides direct access to the data that is accessible through the web browser interface. 1. Turn off the visibility of all Polygon Layers, using their checkboxes. 2. Click on the Add Data Button. 3. Return to the GIS Servers Item 4. Double-Click on the Add ArcIMS Server Icon. 5. Add the URL of the USGS Seamless Website: 6. Click OK. 7. You will be returned to the Browse Window and the seamless.usgs.gov server will be added to the window. Doubleclick on the seamless.usgs.gov server to browse into its services. 8. Scroll across the services and select USGS_EDC_Ortho_Landsat7. 9. Click Add to Add the Landsat7 Layer to your Map Document. 10. Click Close when warned about the Coordinate System.

15 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 15 of 16 Searching for Data Most of the time, you will want to begin your GIS project by searching for the data that is available to you for your area of interest. The following points should help guide you in your search for data. Geodata.gov The Geodata.gov site is a clearinghouse for all type of Federal, State and local GIS data sources. It provides searchable Metadata, and the ability to examine discovered data in a built-in GIS data Browser. 1. Return to your Web Browser and go to Geodata.gov. 2. Click on Show Advanced Search Options. 3. Use the My Geography Map at the right, to Zoom into the state of Connecticut. 4. Check the Use My Geography checkbox. 5. Under Content Type, select Live Data & Maps. 6. Under Data Category, select Agriculture & Farming. 7. Click on Search to query the database. 8. The first result returned should be USDA-FSA-APFO NAIP UTM Zone 18 ArcIMS Image Service. Click on the View Metadata Button. 9. On the resulting Full Metadata, scroll to the bottom of the page and look for the ESRI Metadata section and the Server: item. This is the server address that you can add to ArcMap as an ArcIMS Server. Once added, you should be able to browse into the server s services and select the image for Zone 18 to add it to the Map Document. 10. Return to ArcMap and click on the Add Data Button. 11. Go to the Add ArcIMS Server item and Double-click on it. 12. Add the server to ArcMap. 13. Browse into the server, select and add the NAIP_UTM18 Service. 14. Remove or turn off the visibility of any layer obscuring the added data layer. Google Search Strategies for Finding GIS Data Google can be one of the more effective tools for searching for GIS data. Here are a few tips for effective GIS Data Googling: o Add GIS and Data-Specific keywords to your Google Search ( GIS Data, shapefile, cadastral, digital elevation model, etc )

16 02-Finding_Data_And_Preparing_It_For_GIS_Analysis_92.Doc Page 16 of 16 o o ALWAYS inspect metadata for data discoveries and pay particular attention to collection date, scale and update frequency. If these items are not available, contact the source of the data to confirm these Metadata items. Always obtain explicit permission to use data that is not expressly labeled as Public and Open.