SOMAP-OSGEO WORKSHOP Open GIS and Modern Webmapping Vienna, November 2014

Transcription

1 SOMAP-OSGEO WORKSHOP Open GIS and Modern Webmapping Vienna, November 2014 Barend Köbben November 12, 2014 Contents 1 A short introduction to Geo-webservices 5 2 Introduction to the OSGeoLive system 17 3 Webmap clients Using Leaflet to make a simple OpenStreetMap viewer 25 4 QGIS Open Source Desktop GIS Using the world of services in QGIS. 4.2 Using QGIS to create data 5 Serving data as a Web Map Service Using Geoserver. 5.2 SLD and cartographic coding in Geoserver. 6 WMS in a webmap client: Geoserver WMS and OpenStreetMap in a Leaflet application 49 ITC University of Twente, Faculty of Geo Information Science and Earth Observation. This document may be freely reproduced for educational use. It may not be edited or translated without the consent of the copyright holder.

2 ii Dept. of Geo Information Processing 2

3 Key points The hands-on workshop Open Source GIS and Modern Webmapping is a 2-day workshop on November 24th & 25th 2014 in Vienna, Austria. It is organized on behalf of the Commission on Map Production and Geobusiness and the Commission on Open Source Geospatial Technologies of the International Cartographic Association (ICA). Note that these exercises assume you are using the OSGEO-Live system, but the various softwares from that system can also be installed in any Windows, MacOSX or Linux based computers, and the exercises would work accordingly on those computers...! In many cases during these exercises, you will have to type code (HTML, JavaScript or MapServer configuration code). It s very easy to make mistakes in such code. HTML code and MapServer map files are not case sensitive, but JavaScript is: the variable mysomething is different than the variable MySomeThing! Also take care of the special character ( ) in the code examples we provide: this character means the line should be typed without interruption, the move to the next line in our example is only because it would not fit otherwise. So do not type a return or enter in this place! Typing the code of longer listings is usually not necessary: You can copy the code from the filefragments folder in the exercise data. In this folder you will find all code fragments from the exercises in text files with the same names as the listing. Do not copy from the PDF exercise description! ITC University of Twente, Faculty of Geo Information Science and Earth Observation 1 3

4 4

5 A SHORT INTRODUCTION TO GEO-WEBSERVICES Lecture note on the history and principles of geo-webservices Barend Köbben Version 2.0 May 7, 2012 Contents 1 From monolithic to distributed GIS architectures 2 2 Interoperable webservices XML: extensible Markup Language Webservices Geo-webservices Open Web Services specifications (OWS) The FOSS4G geo-webservices stack An application example: The Melka Kunture Virtual Museum ITC University of Twente, Faculty of Geo Information Science and Earth Observation. This document may be freely reproduced for educational use. It may not be edited or translated without the consent of the copyright holder. 5

6 ii Dept. of Geo Information Processing 6

7 Key points This document is intended to serve as a quick introduction to the notion of geo-webservices. It was originally written for students at the International Institute for Geo-Information Science and Earth Observation (ITC) that are fairly familiar with spatial data and GIS. It is based on the teaching materials (slides, exercises) for various modules on WebGIS, Webmapping, Spatial Data Infrastructures and the like, developed by Rob Lemmens, Andreas Wytzisk and Barend Köbben. ITC University of Twente, Faculty of Geo Information Science and Earth Observation 1 7

8 1 From monolithic to distributed GIS architectures Interoperable geo webservices are the latest in a long line of software for handling geographic, or spatial, information. In 1962 Roger Tomlinson built the Canada Geographic Information System to determine the land capability for rural Canada by mapping information about soils, agriculture, land use, etc. He coined the term GIS for software that is used to gather, visualize and analyse geo-information. Tomlinsons GIS software was running on a large mainframe computer and its architecture was what we call monolithic, with the presentation logic, application logic, and data management layers combined in one software tier. This might still be the typical design for simple desktop applications, but nowadays larger GISs, like other software systems, have their logical layers separated. Separate logical layers, a depicted in Figure 1 improve the interoperability between Geographic Information Systems. Figure 1: A typical setup of layers in a GIS. One or more database layers take care of the data storage and retrieval, application layers are used to analyse the information, a mapping engine turns the information into maps, and separate client software gives the actual users access to all of this. The main advantage of such a setup is the flexibility: the different parts can be distributed over various computers and thus can easily be scaled, that is, adopted to changing conditions, such as an increasing number of users. Because the client software is separated from the application logic, the same back end can serve different client platforms (e.g. PCs, PDAs and mobile phones). Equally the data being used in the analysis part can come from various different information sources, from various providers, even from different places on the globe. 2 Dept. of Geo Information Processing 8

9 2 Interoperable webservices Probably the best known example of such a distributed system approach is the World Wide Web. The WWW consist of many distributed servers that are responding to individual requests of a sheer endless amount of distributed clients. This kind of system is only possible when the different components are guaranteed to work together, because they are interoperable. The International Standards Organisation (ISO) defines interoperability as: The capability to communicate, execute programs, or transfer data among various functional units in a manner that requires the user to have little or no knowledge of the unique characteristics of those units. Interoperability in computer systems, as shown in Figure 2, means in the first place that the systems are able to transfer data seamlessly. Figure 2: Interoperable Geographic Information Systems. This can be achieved by having the data encoded in a standardized, platform and application independent manner. Nowadays, the popular encoding scheme used for that purpose is the extensible Markup Language (XML). 2.1 XML: extensible Markup Language The extensible Markup Language (XML) is a computer language that defines a set of rules for encoding documents in a format that is both human-readable and machine-readable. The design goals of XML emphasize simplicity, generality, and usability over the Internet. It is a textual data format and widely used for the representation of arbitrary data structures, for example in web services [wikipedia.org]. The structure of XML uses a strict separation of content from presenta- ITC University of Twente, Faculty of Geo Information Science and Earth Observation 3 9

10 tion. XML is self descriptive, it provides a way to encode both structure and content of data. In any XML-language, the XML standard just sets the syntax, and an XML schema is needed to explain what the XML means. One can compare it to a set of languages that share exactly the same grammar. Th big advantage is that only translation of individual words is needed to translate one language to another... Figure 3: An example of XML. 2.2 Webservices Interoperability also means that two information systems should be able to access distributed functionality in a seamless manner. This would for example mean that one Geographic Information System could use the geo processing tools of another GIS. For that to work regardless of operating system, computer platform or software used, we have to specify and set up an infrastructure of interoperable software services. Serviceoriented software differs from traditional software in that it fully encapsulates its own functionalities and makes it accessible only via well specified and standardised interfaces. These interfaces publicise the methods that a software component implements, and its calling conventions. In other words, you do not know, nor have to know, how the service actually works, only what input it can receive and what output you can expect back. There are many ways of setting up such Service Oriented Architectures (or SOA s), but by far the most used distribution platform is the WWW and the services implemented on that are logically called webservices. Webservices are components that can be described, published, located and invoked over the Internet. They are defined as loosely coupled components that communicate via XML-based interfaces. Loosely coupled means that they are independent of computer platform and can be exchanged by similar components, e.g., when one service fails or is too 4 Dept. of Geo Information Processing 10

11 busy, the system can find a similar service elsewhere on the Web. The XML-based interfaces are human readable and self describing, which allows for automated discovery of their functionality. A simple example of such a Webservice is a currency convertor, for example the one at If an application (such as the Demo form on the webpage) is used to send a request to this service formatted as a standardized XML message <FromCurrency>USD</FromCurrency> <ToCurrency>EUR</ToCurrency> the service returns a standardized XML response: <double> </double> If webservices have spatial functionality, for example if they use geographic data, can output maps or find routes, we call them geo webservices. 3 Geo-webservices There are many such geo webservices available, the best know proponent of it probably is Google Maps, and other examples are Bing Maps and Yahoo Local Maps. These can be used by anybody, as their interfaces are publicly available, but they are still proprietary in the sense that they are defined, developed and owned by commercial companies. Alternatively, Open Standards are created in an open, participatory process, where everyone interested can influence the standard. The resulting specifications are non proprietary, that is, owned in common. That means free rights of distribution (no royalty or other fee) and a free, public, and open access to interface specifications that are also technology neutral. There is a set of such well defined Open Standards for geo webservices: the Open Web Services (OWS) of the Open Geospatial Consortium (OGC). The OGC was founded in 1994 as a not-for-profit, international voluntary consensus standards organization that develops Open Standards for geospatial and location based services. Their core mission is to deliver interface specifications that are openly available for global use. The Open Web Services specifications are the basis of many high-profile projects (e.g., the European Community INSPIRE initiative). 3.1 Open Web Services specifications (OWS) There are OWS specifications for most parts of the spatial data storage, analysis and delivery process: for geographic data encoding: the complete and complicated Geographic Markup Language (GML), and the simpler, more limited ITC University of Twente, Faculty of Geo Information Science and Earth Observation 5 11

12 Keyhole Markup Language (KML); for spatial data delivery: the Web Coverage Service (WCS) and Web Feature Service (WFS), for querying and retrieving raster and vector data respectively; for processing of spatial data: the Web Processing Service (WPS). Note that this specification does not standardize the analysis or processing methods themselves, but rather defines a standardized interface that lets you publish geospatial processes, and lets client software find those processes and employ them. for data visualisation in the form of maps there is the Web Map Service (WMS). This is by far the most mature and widest adopted OWS specification. There are numerous open source as well as commercial solutions offering WMS functionality. Related to WMS are the Styled Layer Descriptor (SLD) specification, for map styling, and the Web Map Context Documents (WMCD) specification, for map setup and layout; for describing and finding spatial data, there is a set of metadata specifications in the Catalog Service Web (CSW). We will use WMS as an example to show the working of OGC specs in a bit more detail, see the OGC website for more details on the other specifications. The WMS specification defines four interfaces: GetCapabilities, GetMap, GetLegendGraphic and GetFeatureInfo. The GetCapabilities interface is used by client software to ask for the capabilities of the service: what layers are available, what projection systems can the maps be delivered in, what output formats can be requested, etcetera. So if you point your browser to the URL: cgi-bin/mapserv.exe?map=d:/inetpub/mapserver/config.map&service=wms&version= 1.1.1&REQUEST=GetCapabilities the service will return an XML file describing the capabilities. It is human-readable text, but not really meant for reading: Software (GIS or mapping clients) will parse it to be able to show layers, zoom to the extent, know what projection to use, etcetera... Based on the GetCapabilities result, the GetMap request is issued to ask for an actual map. Because the client knows the possibilities of the service from the GetCapabilities response, it can issue its request with specific parameters asking for example for a one or more layers of information (LAYERS=borders,forest), in a certain part of the world (BBOX=x1,y1,x2,y2). It will also request that the output is returned in a format that the client can handle (e.g., a web browser 6 Dept. of Geo Information Processing 12

13 will request FORMAT=image/png), and in a specific size in pixels (e.g., WIDTH=250&HEIGHT=300). Furthermore, the map output will have to be be requested in a specific projection (SRS=EPSG: ). Thus, if you type in the URL: config.map&service=wms&version=1.1.1&request=getmap&layers=forest, railroad,airports&styles=&srs=epsg:4326&bbox=97.35,5.61,105.64,20. 47&WIDTH=400&HEIGHT=600&FORMAT=image/png in a web browser, the WMS at that site will return the graphic shown in Figure 4. The GetLegendGraphic request is used in a similar fashion to get a pictorial rendering of a legend item separately. If the user of a mapping client triggers a zoom command, the client will issue a new GetMap request, now with a smaller BBOX. This process will be repeated over and over again while the user interacts with the map. Figure 4: Result of the WMS request in the URL on page 7 The GetLegendGraphic request is used in a similar fashion to get a pictorial rendering of a legend item separately. 1 the SRS, or Spatial Reference System is usually expressed as an EPSG code. EPSG is the European Petroleum Standards Group, and their standardized descriptions of datums and projections are used worldwide. See for an overview. ITC University of Twente, Faculty of Geo Information Science and Earth Observation 7 13

14 In WMS services that advertise layers of data as queryable, the GetFeatureInfo request can be used to find attribute values in the underlying data. The client software reports the location of a request in pixel coordinates within the map graphic, and requests the WMS to report on the attribute values of the location in specific layers. The WMS thus has to take into account the size in pixels and in real world coordinates of the graphic it has supplied, plus any re projections is has done, to calculate the request location in the coordinate space of the original data. It then finds data objects at that location, and does a lookup in the original data store (a vector or raster file, a database, etcetera) to find appropriate data attributes. It then reports these back to the client in the format requested (plain text, HTML or XML). You can experiment with the WMS and other OWS services at our website An easy way to see how the different parameters of the interfaces work is the testing form at The FOSS4G geo-webservices stack We use geo webservices at ITC in in teaching, research and projects. Students and staff build services and clients using a software and data stack that is employing so called Free and Open Source Solutions for Geospatial (or FOSS4G for short). This Open Source stack can be set up using a wide choice of software. We use a set-up that we call SDI light. The term SDI for Spatial Data Infrastructure may be usually connected with (very) large regional or national spatial data warehouses, but the principles of SDI can also be applied in more simple and cost effective ways. The down to earth approach of SDI light provides researchers and students alike with a proof of concept platform for relatively simple, low cost, yet powerful ways of sharing data amongst various distributed offices and institutions as well as the general public. To achieve this, we use open standards whenever available, open source solutions where possible and commercial software where necessary 2. In general, the main building blocks we use are: A spatial database back end that stores the spatial data using the Open Geospatial Consortium Simple Features specifications. As a platform, the PostGIS extension to the object relational DBMS PostgreSQL is a logical choice. First, because PostgreSQL is a solid DBMS that has a reasonably gentle learning curve, yet is wonderfully appropriate for advanced database applications, and 2 more about SDI light in B. Köbben, R. de By, et al. (2010). 8 Dept. of Geo Information Processing 14

15 Figure 5: The Open Source stack used in ITC s SDI light. its documentation is very transparent. PostGIS in addition, is the leading open standards implementation of spatial vector management, and enjoys a lively and supportive user/developer community. A set of interoperable middleware web applications that interface with the database back end and with each other, and fulfil tasks such as delivering maps for visualisation purposes or providing data and processing services. We use existing open source solutions, mainly MapServer and GeoServer. Simple (thin) browser-based clients enabling access to the maps and data. We employ the OpenLayers API and Scalable Vector Graphics (SVG). There are also more established stand alone (thick) clients: We use QuantumGIS for map viewing, data analysis and editing. 3.3 An application example: The Melka Kunture Virtual Museum One website that uses this stack is the Melka Kunture Virtual Museum at Melka Kunture lies in the upper Awash valley, 50 km south of Addis Abeba (Ethiopia). It is the site of very important archeological finds and 70 archaeological levels have been discovered so far, the oldest dating back to about 1.7 million years. In 2003 the Museum of Melka Kunture was built, and in 2007 a Virtual Museum was put on the WWW. It represents not only the four buildings and the Open Air Museum, but also gives access (in the For Scientists section) to the vast amount of spatial data of the excavations and various finds. All elements have been stored in a spatial database. We use MapServer to provide WMSes for the excavation data and the OpenLayers script of- ITC University of Twente, Faculty of Geo Information Science and Earth Observation 9 15

16 Figure 6: Screendump of the interactive map part of the Melka Kunture Virtual Museum. fers these as interactive map views in the web pages. Custom Javascript on client- and server-side provides further links from the spatial excavation data to the content of the museum exhibition panels, and the other way around. In this way, users can click on finds in the map, find more data (and sometimes drawings and photos) of that element, as well as links to the appropriate museum sections. Further Reading OGC website. This is the official homepage of the OpenGeospatial Consortium, where you can find all relevant standards and whitepapers and other information about them. Barend Köbben, Rolf de By, Theodor Foerster, et al. (2010). Using the SDIlight approach in teaching a Geoinformatics Master. Transactions in GIS, 14 : s1, pp [LINK]. This further explains the ITC FOSS4G software stack and its use in education, research and projects. 10 Dept. of Geo Information Processing 16

17 INTRODUCTION TO THE OSGEO LIVE SYSTEM Barend Köbben Version 1.0 November 11, 2014 Contents 1 The OSGEO-Live system Installing OSGeoLive as a Virtual Machine Trying out OSGeoLive without installation ITC University of Twente, Faculty of Geo Information Science and Earth Observation. This document may be freely reproduced for educational use. It may not be edited or translated without the consent of the copyright holder. 17

18 ii Dept. of Geo Information Processing 18

19 Key points This document gives a short introduction to the OSGEO- Live system. In various workshops we are using free and open source GIS and webmapping applications. In many cases it is most convenient to use these softwares as installed in the so called OSGEO-Live system (the current version is OSGeoLive 8). This is a self-contained bootable DVD, USB drive or Virtual Machine based on the Linux Ubuntu operating system, that allows you to try a wide variety of open source geospatial software without installing anything. It is composed entirely of free software, allowing it to be freely distributed, duplicated and passed around. 1 The OSGEO-Live system The OSGeoLive system is created, maintained and offered (for free) by OSGEO. The Open Source Geospatial Foundation, or OSGEO, is a not-for-profit organization whose mission is to support and promote the collaborative development of open geospatial technologies and data. The foundation provides financial, organizational and legal support to the broader open source geospatial community. OSGeo also serves as an outreach and advocacy organization for the open source geospatial community, and provides a common forum and shared infrastructure for improving cross-project collaboration. The foundation s projects are all freely available and useable under an OSI-certified open source license. More information at the website The OSGeoLive system provides pre-configured applications for a range of geospatial use cases, including storage, publishing, viewing, analysis and manipulation of data. It also contains sample datasets and documentation. There are several ways you can use the OSGeoLive system: 1. Try out OSGeoLive without installing, by booting your computer from the OSGeoLive DVD or USB. The advantage of this is that you leave your computer untouched, and it works regardless of the specifications of your computer. It works however rather slow. ITC University of Twente, Faculty of Geo Information Science and Earth Observation 1 19

20 2. Run OSGeo-Live within a VirtualBox virtual machine (VM). The advantage of this method is that the response speed of a VM is much faster than on the DVD, and typically comparable with installing directly on the host machine. All configuration changes, software updates, and saved files are permanent, just like on any regular computer. Also, you can backup your virtual machine installation. If at all possible, we advice you to use the second method, of running in a VM, described below in section 1.1. If for some reason that can not be done, refer to section 1.2 for running of the DVD or USB. Both methods should result in your computer running OSGEOLive, and a Linux desktop such as seen in figure 1 below... Figure 1: The OSGEOLive desktop, in this case version 8, running on Lubuntu Linux. 1.1 Installing OSGeoLive as a Virtual Machine The Lubuntu system runs well with 1 GB of RAM, and you ll need to keep at least the same amount of memory for your host system. So a total of 2 GB will be needed for smooth operation. The virtual disk file (vmdk) from live.osgeo.org unzips to almost 10 GB. And you ll want more space to allow some room to work on the virtual machine. So 20 GB is a good recommendation. Most machines produced in the last 4 5 years will meet these requirements. 2 Dept. of Geo Information Processing 20

21 Download Virtual Box for your platform, and install the software. On windows run the installer, or on Ubuntu-like linux systems do the following: apt-get install virtualbox-ose. In addition download the OSGeo Virtual disk file (.vmdk) from your data source (USB stick, website of the workshop or live.osgeo.org. Then unzip the downloaded file (using 7zip, installer provided if needed). Start the VirtualBox application and click on the New button to create a new VM, and then Next. Enter a name, such as OSGeo-Live, and choose Linux as the Operating system, and Ubuntu as the Version. In the next screen set the memory to 1024 MB (or more if your host computer has more than 4GB). Continue to the next screen and choose Use existing hard disk or existing vmdk file. Now click on the folder icon to browse to where you saved the osgeo-live-vm-6.0.vmdk file. Select this file, press Next and Create. Once the VM is created, click on the Settings button. In the General section, go to the Advanced tab, and click to select Show at top of screen for the Mini toolbar. Go to the Display section and increase video memory to 32 or 64 MB. In addition, move to the Shared Folders section, and click the Add folder (green + icon on the right) to find a directory on your host computer that you wish to share inside the VM. Once the Folder path and Folder name are defined, click OK, and again OK to finish and close the settings window. Now bootup the VM by clicking the Start (green arrow) button. ITC University of Twente, Faculty of Geo Information Science and Earth Observation 3 21

22 Once the OSGeo system comes up, you have the option to add the VirtualBox Guest Additions to improve video performance, and enable the shared folders option that was defined above. The guest addition installations are supplied as an ISO file together with the VirtualBox application from Oracle. This ISO is mounted within the VM as a CD, and the installation is run from there. Here s how it s done: In the VirtualBox menu (so actually outside the Linux window), open the Devices menu and click Install Guest Additions. This will mount the Guest Additions as a CD drive in your OSGeo Live VM. Once the CD folder appears, open a Linux terminal (by going to the menu Accessories > LXTerminal): In the Terminal, type and run (press enter) the following command: sudo apt-get update You will be asked for the password for the user called user. This, not surprisingly, is user... 4 Dept. of Geo Information Processing 22

23 Now run these commands: 1.2 Trying out OSGeoLive without installation sudo apt-get install build-essential linux-headers-generic cd /media/user/vboxadditions_4.1.18_78361/ sudo./vboxlinuxadditions.run This will complete after a few moments. Reboot your VM, and you will now be able to run in full screen mode, and mount your shared folders. In the above example, we defined a Shared Folder path on the host system and named it GIS in the VM Settings. To mount it within the VM, open a terminal window and run: mkdir GIS sudo mount -t vboxsf -o uid=user,rw GIS /home/user/gis Now files on your host machine in the chosen folder will also be available in the VM in the GIS folder in user s home directory. To try out OSGeoLive without installing anything on your computer, simply: 1. Insert the DVD or the USB stick in a computer or virtual machine; 2. Reboot the computer. In Microsoft Windows, you might have to verify or set the boot device order, to allow startup from a CD (mostly by pressing a key during start up, typically the F 12 key). On Apple OSX, to boot from CD, you hold the C key while restarting; 3. For CD booting: Wait for the boot: prompt, then press Enter to startup and login. For USB booting, you should see a menu where you choose the first option (start normally); 4. The Lubuntu OS should start up, and now you should have a desktop similar to figure 1 (depending on the version of OSGeoLive you are using). Many applications are also provided with installers for Apple OSX and Microsoft Windows. More information, including alternative disk images and installations, can be found on the website http: //live.osgeo.org. ITC University of Twente, Faculty of Geo Information Science and Earth Observation 5 23

24 24

25 EXERCISE: PROGRAMMING WEBMAP CLIENTS Using the Leaflet Javascript API to make a simple OpenStreetMap viewer Barend Köbben Version 1.0 OSM only November 12, 2014 Contents 1 Open Data: using OpenStreetMap 2 2 The Leaflet API 3 3 Making a Leaflet viewer for OpenStreetMap Enhancing the lay-out Changing the starting position and zoom ITC University of Twente, Faculty of Geo Information Science and Earth Observation. This document may be freely reproduced for educational use. It may not be edited or translated without the consent of the copyright holder. 25

26 ii Dept. of Geo Information Processing 26

27 Key points This is a reference for the use of Leaflet, a JavaScript API (Advanced Programming Interface). Leaflet can be used to build general WebMapping clients, among others to connect to OGC Web Map Services. In this exercise we will give a very brief introduction, where you will learn how to: 1. Use the Leaflet library in a web page; 2. Make a simple map using the OpenStreetMap background; 3. Controls the initial view;! In many cases during these exercises, you will have to type code (HTML, JavaScript or MapServer configuration code). It s very easy to make mistakes in such code. HTML code and MapServer map files are not case sensitive, but JavaScript is: the variable mysomething is different than the variable MySomeThing! Also take care of the special character ( ) in the code examples we provide: this character means the line should be typed without interruption, the move to the next line in our example is only because it would not fit otherwise. So do not type a return or enter in this place! Typing the code of longer listings is usually not necessary: You can copy the code from the filefragments folder in the exercise data. In this folder you will find all code fragments from the exercises in text files with the same names as the listing. Do not copy from the PDF exercise description! There are several software tools that can help you: Use a text editor that is more intelligent than your basic text editor, e.g. on MacOSX and Linux use TextWrangler, on Windows Notepad++.This will provide you with line numbers, automatic highlighting of recognised HTML and JavaScript keywords, etcetera. Use a modern web browser: a recent FireFox, Chrome or Opera, or Internet Explore version 9 or higher. These are HTML5 compatible and have built-in web developer tools. This provides error messages, code views and a JavaScript console, network traffic monitoring, etc... For students that intend to do more work on developing interactive web applications (using HTML5, CSS, Javascript, etcetera) there is an Integrated Development Environment available in the ITC software Manager called Webstorm. ITC University of Twente, Faculty of Geo Information Science and Earth Observation 1 27

28 1 Open Data: using OpenStreetMap First we ll show you a prime source of free maps and data on the web: OpenStreetMap. Note: The OpenStreetMap Project, based at org, is the worldwide mapping effort that includes more than half a million volunteers around the globe. OpenStreetMap is an initiative to create and provide free geographic data, such as street maps, to anyone. There are many ways in which you can access the OpenStreetMap: as a simple webmapping service (not unlike Google and Bing Maps, but based on truly free non-proprietary data on a non-commercial website), as a webservice in various gis viewers and as a database service, providing the actual vector data in raw form. This is maybe the most important difference between OSM and the others: You can download the actual vector and attribute data of OSM, and use it for your own projects free of charge, and you can also edit the data directly and in this way contribute to the building of the OSM itself! Task 1 : Visit using a web browser. Try to find the ITC building (it s just North West of the main station in downtown Enschede, a town in the East of The Netherlands)... There are many ways to find places and their coordinates. One of the easiest is to use the Search box find a location, and then look in the URL bar to see what are the X, Y and zoom numbers: The OpenStreetMap site itself uses the Leaflet Javascript API for its User Interface, just as we will do later ourselves. 2 Dept. of Geo Information Processing 28

29 Using the OpenStreetMap.org site as described above is fine for casual map browsing, but if you want to disseminate spatial data yourself, you might want to include the OpenStreetMap in your own websites. And of course that is possible. We will use a popular browser based mapping library for creating such a site: Leaflet. 2 The Leaflet API Leaflet makes it easy to put a dynamic map in any web page. It can display map tiles and overlays, loaded from many sources. Note: Leaflet is a JavaScript library for displaying map data in most modern web browsers, with no server-side dependencies. Leaflet implements a JavaScript API (Application Programming Interface) for building rich web-based geographic applications, a bit like the Google Maps pages, with one important difference: Leaflet is Free Software, developed for and by an Open Source software community based at Leaflet is a modern open-source JavaScript library for mobile-friendly interactive maps. It is developed by Vladimir Agafonkin and a team of dedicated contributors. Weighing just about 33 KB of JS, it has all the features most developers ever need for online maps. Leaflet is designed with simplicity, performance and usability in mind. It works efficiently across all major desktop and mobile platforms out of the box, taking advantage of HTML5 and CSS3 on modern browsers while still being accessible on older ones. Its basic functionality can be extended with plugins. In these exercises, we will only show the basic building blocks, and how to employ them. Those wanting to go further, should check out the tutorial pages and the examples at the website. Using the Leaflet API is done by creating webpages (using HTML) that include Javascript script; this code makes calls to the API methods to create the necessary map object and connect that to an HTML placeholder. Mostly we use an HTML <div> element as a placeholder. You can install the API by including a link to the Javascript files on the official Leaflet website, or to a version you downloaded and stored in your own site (as we do in these exercises). The former has the advantage that you always have the latest version, but the latter has the advantage you can test without a internet connection and avoid downloading the API every time you load your pages. Once you included the Leaflet library script in your webpage, you can call the methods and properties of the library using simple JavaScript functions. The Leaflet API has two concepts which are important to understand in order to build your first map: Map, and Layer. A Leaflet ITC University of Twente, Faculty of Geo Information Science and Earth Observation 3 29

30 Map stores information about the default projection, extents, units, and so on of the map. Inside the map, data is displayed via Layers. A Layer is a data source information about how Leaflet should request data and display it. We then uses the methods and properties of the API to change the content and behaviour of the map. In practice, all this means typing (and/or copying) HTML and JavaScript code. 3 Making a Leaflet viewer for OpenStreetMap In listing 1 you see the most basic example of using Leaflet with the OpenStreetMap service. Listing 1: osm.html <!DOCTYPE html> <html><head> <meta charset="utf-8"> <title>leaflet + OSM</title> <link rel="stylesheet" href="./leaflet/leaflet.css" /> <script src="./leaflet/leaflet.js"></script> <script type="text/javascript"> function init() { mymap = L.map("mapDiv"); mymap.setview([ , ], 16); html5 document header start htm & start header webpage encoding webpage title load the Leaflet styles load the Leaflet script our script starts here function triggered on load use.map method of Leaflet (L) to create map connected to html div set center and zoom osmurl = " OSM tileserver URL osmlayer = L.tileLayer(osmURL, { create layer using URL attribution: " OpenStreetMap contributors" add copyright text }); mymap.addlayer(osmlayer); add layer to map } </script> </head> <body onload="init()"> run init script after page loaded <div id="mapdiv" map placeholder style="width:400px; height:400px;"></div> placeholder CSS style </body></html> Task 2 : Create an HTML page with the content of listing 1. This is best done starting with the file osm.html.txt from the filefragments folder (see tips on page 1). Save it as osm.html. Do not copy from this PDF file! Make sure you save this file as a new 4 Dept. of Geo Information Processing 30

31 file with the extension.html, not.html.txt!). View the result by opening the file in a web browser. The result should look like figure 1, showing the OpenStreetMap for the Eiffel Tower area in Paris (France). Figure 1: result of loading listing Enhancing the lay-out Remember the connection between the Leaflet mapping engine and the webpage is the HTML element you associate with your map object (by means of providing its unique id). You can therefore change the layout by setting the properties of the separate placeholders in HTML. Many HTML elements can be used as placeholders, one of the most simple to use is the <div> element we used earlier. So let s change the layout: Task 3 : Change the line: <div id="mapdiv" style="width: 400px; height: 400px; "></div> to: {<div id="mapdiv" style="position: absolute; left: 5px; top : 5px; width: 400px; height: 600px; overflow: hidden; border : 1px solid blue; "></div> The style properties are expressed using CSS (Cascading Style ITC University of Twente, Faculty of Geo Information Science and Earth Observation 5 31

32 Sheets), a styling language standardised by the W3C, the World Wide Web Consortium. The meaning of the properties used is: position: left & top: width & height: overflow: border: this lets you define positioning absolute or relative to other divs. the (absolute) location with respect to the upper left corner of the window the size of the element (in pixels) if the content is larger than fits the div, it will not be shown if this is set to hidden. Other settings are visible (will overflow), scroll (will make scrollbars) and auto (let browser decide). the border look (width, type and colour). You can also set the fill in a similar way. 3.2 Changing the starting position and zoom You can set up the OpenStreetMap to start at any place on the globe, by setting an alternative starting point and zoom in the setview function: mymap.setview([latitude or x,longitude or y], zoom); Note that the order is longitude latitude (not the more traditional latitude longitude)! This is done to match the order x y when using projected (cartesian) systems. The zoom level is an integer, its range is depending on the map provider you are using. In the case of OpenStreetMap (and most other similar services) the range is from 0 (zoomed out to whole world) to 18 (street level). Task 4 : Try setting up the map in such a way, that it starts zoomed in to the TU Vienna building at the Erzherzog-Johann- Platz 1, 1040 Wien... 6 Dept. of Geo Information Processing 32

33 EXERCISE: QGIS Open Source desktop GIS Using QGIS to explore OGC services, OpenStreetMap and create data Barend Köbben Version 2.0 November 13, 2014 Contents 1 Introduction of QGIS 1 2 Using OGC compliant services in QGIS Connecting QGIS to Web Map Services Connecting QGIS to Web Feature Service data The OpenLayers plugin: Using webmap layers in QGIS 5 4 Creating new vector data 6 ITC University of Twente, Faculty of Geo Information Science and Earth Observation. This document may be freely reproduced for educational use. It may not be edited or translated without the consent of the copyright holder. 33

34 ii Dept. of Geo Information Processing 34

35 Key points In this exercise you will use QGIS, an open source desktop GIS, for some common tasks: to connect to Open Web Services (WMS and WFS) as defined by the Open Geospatioal Consortium (OGC); to use PostGIS data: How to show these data in the form of maps as well as how to import data into those databases; to create some vector data by digitizing on top of an OpenStreetMap background; to export vector data to the common KML (Keyhole Markup Language) format. Note that this is meant just as a quick primer to the use of QGIS, not a full-fledged manual. For more information we point you to the official documentation at org/en/docs/index.html 1 Introduction of QGIS QGIS ( is an Open Source, stand-alone GIS client, programmed in C++ using the multi-platform Qt framework. You can use it to work with vector and raster files, databases or any open standard WMS or WFS compliant server. A strong point of QGIS is its extensibility: you can add plug ins that are written in either C++ or Python, and you can connect it to GRASS, a powerful GIS analysis tool. Task 1 : Users at ITC can use the ITC software manager (if installed on your computer). You can also use QGIS in the OS- GEOLive system. Other users go to the URL and download the latest stable version (sometimes called release, whereas master is the development version). Choose the one suitable for your operating system. Windows users should use the standalone installer, and take care to install the correct 64- or 32- bits version, depending on their hardware. QGIS can load maps and data from a huge array of possible sources: online maps served as an OGC-compliant Web Map Service (WMS); online spatial data served as an OGC-compliant Web Feature Service (WFS) and Web Coverage Services (WCS); ITC University of Twente, Faculty of Geo Information Science and Earth Observation 1 35

36 2 Using OGC compliant services in QGIS 2.1 Connecting QGIS to Web Map Services various other map services, such as OpenStreetMap, Google Maps, Bing Maps, etcetera, through the use of the Open- Layers plugin; ; most vector formats supported by the OGR library, including ESRI shapefiles, MapInfo, KML, GeoJSON, GPX and GML; raster formats supported by the GDAL library, such as digital elevation models, aerial photography or satellite imagery in very many formats; spatially-enabled PostgreSQL tables using PostGIS and SpatiaLite, by means of a live connection to such databases; locations and mapsets from GRASS (an open source GIS); The list is in principle endless, because the functionality of QGIS can be extended by plugins. Plugins add functionality to QGIS, and they are usually made by others than the main QGIS developers. Because QGIS is an Open Source software, anyone can add plugins, which can be programmed using either C++ or Python. QGIS supports the Open Standards of the OGC, as described earlier in the Short Introduction to Geo-webservices. In the short examples below you will load map layers from Web Map Services (WMS) as well as vector data from Web Feature Services (WFS) Task 2 : Choose the Layer > Add WMS/WMTS Layer... menu or click the WMS layer icon. You can choose your Server Connection from the list. This list can be initially filled with some well-known services by clicking Add default servers. Try to find the service provided by LizardTech server. Once you have selected it and clicked the Connect button, the software sends a so called GetCapabilities request to the WMS. The resulting XML description of the capabilities of the service will be parsed by QGIS and it will display the layers that were advertised as being available. Try to find and load a layer that displays a satellite image of the whole earth. This will usually be the so called BMNG, or Blue Marble Next GenerationÕÕ, a dataset created from MODIS satellite images by NASA JPL. 2 Dept. of Geo Information Processing 36

37 You can also add your own WMS connections to this list, e.g., to the example Thailand WMS services provided on the ITC servers: Task 3 : Choose again the Layer > Add WMS/WMTS Layer... menu (or click the WMS layer icon). Now add your own server connection by clicking New. In the Name field you can give any name for the connection you want. In the URL field you put the so called root URL, where the software can find the GetCapabilities interface of the service. Those students that have followed the exercises on setting up their own Mapserver WMS should now enter the particulars of their own WMS server: Use the proper <URLPATH> and <NETPATH> to point to the proper.map file (as decribed in the exercises). In case you did not create your own WMS (yet), use this URL: /geoserver/mapserver/config.map& Now that connection is added, you can use it like the previous one to retrieve capabilities and load WMS layers: Combine the layers railroad and airports from the Thailand WMS with the next generation blue marble. Try to find out how to zoom to the extent of the Thailand layer, so that the map image is similar to the adjoining picture. Using these techniques, you can use any WMS available on the internet or your local intranet. The only thing you need to know is the the root URL, where the GetCapabilities interface of the service can be found. E.g., you will learn (or have learned) how to use MapServer to publish a OGC-compliant WMS. To access this service in QGIS, again you would use the base URL to the service. ITC University of Twente, Faculty of Geo Information Science and Earth Observation 3 37

38 2.2 Connecting QGIS to Web Feature Service data A big disadvantage of loading WMS images, instead of actual data, is that you can not symbolise the data the way you want. In the previous Task you may have noticed that the visualisation of the railroads was not very suitable for combination with the NASA raster image. But because it comes as a map image from a WMS, there s nothing you could do about that. By loading vector data, you can change that. That means you could use also that other type of OGC geo-webservice, the Web Feature Service (WFS) because that delivers vector data in the form of a stream of GML. Task 4 : You can add a WFS connection similar to the way you added WMS connections in the earlier Task. The root URL for the WFS is again dependent on your situation: Those students that have followed the exercises on setting up their own Mapserver WFS should now enter the particulars of their own WFS server: Use the proper <URLPATH> and <NETPATH> to point to the proper.map file (as decribed in the exercises). In case you do not have your own Mapserver WFS, use this URL: D:/Inetpub/geoserver/mapserver/wfs_config.map& Add the airports layer from that service... The data are loaded and visualised as lines using a default line width, colour and line type. But you can change these: Right click on the layer name in the Layers panel (normally on the left of the application window), and choose the Properties item from the menu. In the Layer Properties dialog, choose the Symbology tab. Task 5 : Experiment with the settings until you have an airports visualisation that fits well with the blue marble image background Dept. of Geo Information Processing 38

39 3 The OpenLayers plugin: Using webmap layers in QGIS QGIS is not limited to loading data from OGC services and files, it also allows you to use existing publicly available map services, such as Google, Bing, OpenStreetMap etcetera... We will try out the use of OpenStreetMap, using a plug in that offers that functionality named (somewhat confusingly) the OpenLayers plugin. Task 6 : First check if the OpenLayers plugin is installed: Choose Plugins > Manage and Install Plugins..., and type openlayers in the Filter box. If the plugin is available, it will be listed and enabled (checked). You can go on to the next Task. If not, select the plugin by checking the box and (if necessary) click the Upgrade Plugin button. Click Close. An item called OpenLayers Plugin should now be available in the Web menu. This plugin offers access to many publicly available map services, such as OpenStreetMap, Google Maps, Bing Maps and others. But despite its confusing name it does not offer direct access to the OpenLayers Javascript API...! Now let us use the plugin to have OpenStreetMap in the background: Task 7 : Choose Add OpenStreetMap layer from this submenu, and the OpenStreetMap will be opened, zoomed out on the whole world. Navigate to the location you were asked to set as the focus for the mapping webpage you were creating in the earlier exercises... ITC University of Twente, Faculty of Geo Information Science and Earth Observation 5 39

40 4 Creating new vector data We will create a very simple vector line dataset, that depicts a walking route. Task 8 : Choose the menu Layer > New > New Shapefile Layer... The New Layer dialog opens (see below). Make the following settings: 1. For Type choose Line; 2. Make sure the data is stored in latitude-longitude (unprojected), by checking that the field next to the Specify CRS button is set to EPSG: WGS84. If not, click the button and correct it; 3. In the New attribute section, create one attribute of type Text, call it for example routename. Click the Add to attribute list button to actually add it; 4. Click OK to create the new file. Save it as myroutes.shp. Now you can start adding lines for your route: Click first the Toggle Editing button in the QGIS menubar, then the Add Feature button. Create a nice walking route. Depending on the context you do this exercise in it could be e.g. a route from the Enschede main station to the ITC building, from the TU Vienna workshop premises to the Karlsplatz conference building, etcetera.. You can add points to the line by clicking in the map, undo them by using the CTRL-Z key or Edit > Undo menu. If you have finished a route, right-click, fill in the route name and press OK. Use the Toggle Editing button again to stop editing. Remember to save regularly! You can change the visualisation of the line (and many other properties) by double-clicking the symbol name in the layers list, or choosing the Layer > Properties menu. You should create at least two alternative walking routes in the same file. 6 Dept. of Geo Information Processing 40

41 SERVING DATA AS A WEB MAP SERVICE Using Geoserver to set up an OGC WMS Barend Köbben Version 1.1 November 13, 2014 Contents 1 Starting and Administrating GeoServer 1 2 Creating Web Map Services in GeoServer Testing the WMS Styling the WMS using Styled Layer Descriptors (SLD) 4 ITC University of Twente, Faculty of Geo Information Science and Earth Observation. This document may be freely reproduced for educational use. It may not be edited or translated without the consent of the copyright holder. 41

42 ii Dept. of Geo Information Processing 42

43 Key points In this exercise you will learn how to set up the software GeoServer as a Web Map Service using the Open Geospatial Consortium s OWS standards. You will publish data you created earlier as a WMS. Note that this exercise assumes you are using the OSGeoLive system, and that you have achieved the tasks in the exercise Using QGIS to create data. 1 Starting and Administrating GeoServer GeoServer ( is an open source software server written in Java. Designed for interoperability, it publishes data from any major spatial data source using open standards. Being a communitydriven project, GeoServer is developed, tested, and supported by a diverse group of individuals and organizations from around the world. GeoServer is the reference implementation of the Open Geospatial Consortium (OGC) Web Feature Service (WFS) and Web Coverage Service (WCS) standards, as well as a high performance certified compliant Web Map Service (WMS). GeoServer is a service: That means that it acts as a background application, listening for requests on the web. You configure it using a series of web pages. It is installed on the OSGEO Live DVD, but the service has to be started up first in order to work: Task 1 : In the menu, choose Geospatial > Web Services > GeoServer > Start GeoServer. The server starts up and (after you OK this) a browser window is opened to show the connection to the administration pages of the service, at the URL This is by default using the public interface, where you cannot change any settings. In order to set up the services you will have to login. Fill in the username ( admin ) and password ( geoserver ) and click the login button. The administrator pages are loaded... ITC University of Twente, Faculty of Geo Information Science and Earth Observation 1 43

44 2 Creating Web Map Services in GeoServer GeoServer uses a fairly elaborate setup: There are several Workspaces, that each can hold one or more DataStores. These connect the service to various datastores, either simple ones like vector and raster files, or more complicated ones like spatial databases or other OGC services. Each datastore can in turn contain one or more Layers. Task 2 : We will use an existing Workspace, the default one called it.geosolutions. In the Data menu, on the left side of the screen, click the Stores icon or name. Now click the Add New Store icon. From the list of datastores, choose the one called Shapefile ESRI(tm) Shapefiles (*.shp). Now fill in the dialog: For the Shapefile location, you can use the Browse... button to navigate to the shape file you created earlier in QGIS [alternatively, you can use a pre made one delivered to you by the teacher or via the workshop website]. Click the Save button.! Depending on the way your OSGeoLive system has been set up and ran, an error message might occur, stating that the system Could not acquire data access. This is a problem with security settings of the directory where you saved the data earlier. The simplest way to solve this is to copy your data files (so all the pieces of the shape file:.shp,.dbf,.prj, etcetera) from your directory to Geoservers Data directory. The path to that directory is usually: /usr/local/lib/geoserver-2.5/data_dir/data/ (where the version number of geoserver might be different) You have now created a new Data Store that makes the routes shapefile available within Geoserver. The next step is to publish the data as a service, in this case as a Web Map Service (WMS): 2 Dept. of Geo Information Processing 44

45 Task 3 : Click in the data menu (left side of screen) on the Layers item. In the next screen, choose the Add new Resource item. From the list choose it.geosolutions:myroutes, and in the list of layers that is shown next click on the Publish link next to the myroutes layers (this normally should be the only one to choose from). Now you can edit the layer properties. Most fields should have been filled in correctly by default. Check if the projection is set correctly (to EPSG:4326). If necessary, Click the Find button to set the Declared SRS field to EPSG:4326. Complete both the Native Bounding Box and Lat/Lon Bounding Box fields, by clicking the Compute from data links under them. Then click the Save button. 2.1 Testing the WMS You can test the if layer publishing was successful by using GeoServers built-in client: Task 4 : Choosing Layer Preview in the Data menu (left side of screen). There are many tests here, the most easy being to click on the OpenLayers link. ITC University of Twente, Faculty of Geo Information Science and Earth Observation 3 45

46 Now test if your data is available for anyone on the internet that connects to one of the webservices that the GeoServer offers, by using another client to open the WMS: Task 5 : Test if the WMS of the routes you just created indeed works for all, by opening it in QGIS, shown on top of Open- StreetMap. By using what you learned in earlier exercises, you should be able to figure out how to do this... 3 Styling the WMS using Styled Layer Descriptors (SLD) In your WMS, the routes you have created are depicted with a thin blue line. This just happens to be the default style set for line data. Of course you would rather like to change this symbolisation! In GeoServer, styling is accomplished using a markup language called Styled Layer Descriptor, or SLD for short. SLD is an XML-based markup language and is very powerful, although somewhat complex. SLD is one of the OGC standards. In GeoServer styling is most often specified using XML SLD style documents. Style documents are associated with GeoServer layers to specify how they should be rendered. A style document specifies a single named layer and a user style for it. Within the top-level style are one or more feature type styles, which act as virtual layers to provide control over rendering order (allowing styling effects such as cased lines for roads). Each feature type style contains one or more rules, which control how styling is applied based on feature attributes and zoom level. Rules select applicable features by using filters, which are logical conditions containing predicates, expressions and filter functions. To specify the details of styling for individual features, rules contain any number of symbolizers. Symbolizers specify styling for points, lines and polygons, as well as rasters and text labels. Vector data that GeoServer can serve consists of three classes of shapes: Points, lines, and polygons. Lines (one dimensional shapes) are the simplest, as they have only the edge to style (also known as stroke ). Polygons, two dimensional shapes, have an edge and an inside (also known as a fill ), both of which can be styled differently. Points, even though they lack dimension, have both an edge and a 4 Dept. of Geo Information Processing 46

47 fill (not to mention a size) that can be styled. For fills, color can be specified; for strokes, color and thickness can be specified. Task 6 : In the GeoServer administration page, choose the Data > Styles menu, and click the +-button to Add a New Style. Give the style a Name, e.g. MyRoutes, and choose the Workspace you used before (it.geosolutions). Copy the text from Listing 1 into the SLD file edit field, and click Submit. Now go back to the Layers menu and edit the layer. Click the Publishing tab and change the Default Style menu to the style you just created. x If you now load the WMS layer (in the Layers Preview or in QGIS), you will see the symbolisation has changed from the default to a 3 pixel wide, 50% transparent red line. Also, the line is labelled with its name attribute. The SLD language is rather complex, and in this small exercise we only use some very basic elements. For more possibilities it is good to know: The GeoServer website has a very useful section called the SLD Cookbook ( user/styling/sld-cookbook/); The official SLD standard specification can be found at the OGC website ( sld) The easiest way to create styles is using QGIS: In that software, any layer style you have created using the QGIS Style Properties dialog can be saved as an SLD file! Task 7 : Try to create a style that has different symbols for the different routes. The trick is to use various Rules with Filters. The easiest way to create these is to use the QGIS editing as described above. ITC University of Twente, Faculty of Geo Information Science and Earth Observation 5 47