Commercial software from the ESRI company to create display and produce maps

Transcription

1 OCEANIDES USER MANUAL & DATABASE DESCRIPTION Contract No. Project Deliverable Document Title Document File Name Reference EVK2-CT Database description and user manual User manual & database description Db description and user manual_v1.doc D3(b) Issue Author Date 1.0 Philippe Carreau (JRC)

2 contents contents... 1 Actonyms and definitions... 1 I User manual... 3 Connection... 3 Using the interface... 3 Right frame:... 5 II Database description... 7 General architecture design... 7 Access... 8 Content... 9 Loading...10 III Data management & update...12 How to load new data in the database...12 Step Step Step 3:...14 How to add new maps in the system...15 Annexes...16 Annex I Oceanides standard nomenclature...17 Annex II Oceanides input format example

3

4 Acronyms and definitions GIS Geographic Information System GML Geographic Markup Language JDBC Java DataBase Connectivity JRC Joint Research Centre ODBC Open DataBase Connectivity RDBMS Relational Database Management Systems XML extensible Markup Language ArcGIS Commercial software from the ESRI company to create display and produce maps ArcIMS Commercial software from the ESRI company to display maps on Internet ArcSDE Commercial software from the ESRI company to give GIS capabilities to a database Attribute Information link to an object. For example a point representing an oil spill might be associated with a date and a size. AXL file XML file that describe all the layer to be displayed on internet in ArcIMS Client In a client server relation, the client receive the information. EG when a user connect to an internet server, he is the client Data model It is the way the database is structured. What are the tables and field characteristics. ESRI World biggest software company producing tools for geographical information systems Feature(object) Each feature or object is a entity that represent something. Eg: a line represents a river. There are 3 feature types; line, point and polygon Firewall Software to filter information coming (generally) from Internet in order to prevent hacker s attacks and protect computer content Frame A web page is divided in different areas called frame generally separated by border. For Oceanides viewer there are 3 frames and the middle one is the map itself GML Geographic Markup Language describe geographic information (line, polygon, points) in XML. It was defined by the Open GIS Consortium (OGC). IIS Commercial server JDBC Java DataBase Connectivity is a java set of function that allows a programmer to abstract a program from a database Layer Correspond more or less to a map but contain only a single type of object eg: only river lines. Generally they are transparent so users can superimpose them OGC Open GIS consortium as its name indicates is a consortium of commercial and non-commercial GIS actors aiming to define common norms. Object See feature ODBC Open DataBase Connectivity (ODBC) is a set of function that allows a programmer to abstract a program from a database Oracle Very powerfull and world wide used commercial database RDBMS Type of database based on relation between included information

5 Page 2 Server In a client-server relation, the server sends information to the client (see client) Standard nomenclature In Oceanides project, document describing precisely the oils spill characteristics by establishing classification categories for each of them (size, area, location ) Shapefile Shapefile is a GIS format that is a collection of geographical objects. XML Simple language designed to improve the functionality of the Web by providing more flexible and adaptable information identification. 2

6 Page 3 I User manual Connection You can connect to the database using a simple internet browser as Ms Explorer, Netscape or Opera at: Through Oceanides web site: Direct connection to map viewer: Using the interface When you open the map viewer a default maps appears with in background a world satellite image, political boundaries in yellow and the oil spills included in Oceanides database as white crosses. This page (see fig1) is divided in 3 frames. In the left frame there are tools organized in 4 function groups, Move/pan, Zoom in/out, Query/Select and a miscellaneous one. If you click on display info button at the bottom a pop up windows appears with maps quality information. The middle frame is the view and in the right you can select one or a group of maps you want to display. A map is made up of layers, any number of layers can be superimposed over each other. Each map layer can be a set of points, a set of lines or a set of polygons. The layer has a set of attributes. For instance a point representing an oil spill might be associated with a date and a size. One layer at a time of all the layers is the "active" layer which allows one to extract information about the attributes of that layer. To select one in order to query it you must tick the round option. 3

7 Page 4 Fig1: Interface functions Left frame: (1) Pan functions: Use arrows to slide view in the indicated direction. You can also use the hand sign, point somewhere on the map, maintain the left button and slide. (2) Zoom + - and scale change Zoom in and out the map. Return to the original display area (European area) Display the selected layer area (it can be the world) (3) Database management functions (4) Open a window to display database layer. Then you can make queries and visualize the oil spill information. (See chapter 3 -Query database on line-) You have the possibility to select several objects either in a square zone either in a line or a polygon area. Deselect objects Open a window to create a buffer around an object Find and display underline in yellow all objects including the researched information (5) Display the legend in the right frame instead of the layer list 4

8 Page 5 (6) Display information (attribute) concerning selected object. You have to select layer first by ticking the round in the layer list. (7) Change units in scale bar and for distance measurement. Default unit is kilometer but you can choose feet, meters, decimeter (8) Measure the distance between points. A new window appears at the top right of the map. You can measure several following segments by clicking single time successively. Double click to stop process. (9) Display a new window to print selected map area (10) Display information about layer quality. Right frame: (11) Tick the square to display a layer in the map. The update is not done automatically and you have to select the button Refresh Map (13) at the bottom of layer list (12) Tick the round to select a layer. You must select a layer if you want to query information ( ) select features (3) or visualize the information table ( ) (13) Refresh map (14) These are instructions to use the right frame functions 5

9 Page 6 3) Query database on line This a powerful tool to query and extract information from the included maps in the web interface. Every layer can be interrogated but the most relevant to this project is all_europe_oilspills that contains all the oil spill information collected by the project. Thus you can extract the oils spills by date, location and the results are highlighted in yellow on the map. Using this tool may appear a little complicated for those who have never used a GIS before. Some pre-selected information is available in the layer list. You ll find all the information ever classified per year and country and you can combine them at your convenience. Otherwise the exercise below indicates how to find the information. 1) Select layer all_europe_oilspills by ticking the round option in the layer list (right frame). Check if the square option is also ticked as show below. 2) Select the option Query ( ) in the third group from the top in left frame and the following window appears: The button get sample give you a list of all the attribute existing for the field you want to query. With buttons AND, OR, NOT you can make complex query. The Operator (= > <) determines if you want to extract value strictly superior, strictly inferior or equal to the value entered in the field value. You MUST use the button Add to query string before executing the query. As example let s consider we want to extract all the oil spill information after February 5 th 2000 and with oil spill area superior to 0.4 km2. 3) In the field Field select OCEANIDES.ALLTABLE.OILSPILL_DATE using the drop down menu 4) Select Operator > using the drop down menu because you want the date after 05/02/0 (Remark: for the exact date it would have been = or for the date before < ) 5) Click on Get Samples button, the window reopens with a new field Sample Value where you can select 05/02/00" using the drop down menu. (Remark: You can also enter a value by hand but don t forget the quote for the dates.) 6

10 Page 7 6) Select on the button Add to query string, the query you want to execute (OCEANIDES.ALLTABLE.OILSPILL_DATE > "05/02/00") appears in the field in front of the button. (Remark: Here if you select the button execute you will display all the OS monitored after 05/02/00" but we also want to extract the OS with an area superior to 0.4 ) 7) Select the button AND (in the up right). If you look at the query string it is now OCEANIDES.ALLTABLE.OILSPILL_DATE > "05/02/00" AND. The string is probably longer than the field length so click one time in the field and use to right arrow on you keyboard to go at the end. 8) Select field OCEANIDES.ALLTABLE.OILSPILL_AREA, operator > and Sample value = as 3,4 and 5 steps 9) Select on the button Add to query string a second time. The field query string is now (The string is probably longer than the field length so click one time in the field and use to right arrow on you keyboard to go at the end) : OCEANIDES.ALLTABLE.OILSPILL_DATE > "05/02/00" AND OCEANIDES.ALLTABLE.OILSPILL_AREA > ) Click on Execute You can see a table containing the oil spill monitored and they also in yellow in the map see below. II Database description General architecture design Oceanides architecture (see schema below) has 2 main components, a data storage server and a Web server separated by a filtering and protection system called firewall. This system improves the data security. The data storage server is inside The Joint Research Center firewall. It stores the Oceanides data (oil spills, satellite coverage and flight monitoring routes) in an Oracle 8i database linked to a GIS management system ArcSDE (ESRI). This system communicates using a secure connection to an IIS (Microsoft) Internet server located outside the firewall. A commercial software ArcIMS (ESRI) is used to 7

11 Page 8 provide GIS capabilities and for displaying maps through the Web. The external server has also the capability to extract information directly from the database using SQL queries through an ODBC/JDBC connection. In addition some specific functions were implemented (in red) to load the heterogeneous original data in the database using JDBC and the ArcSDE java library. Fig:1 Oceanides architecture schema Access Oceanides database in included in the Digital Mapping Access (DMA) located in the Joint Research Center Ispra site (Italy). The system is composed of 4 components, the web site located on the server, an AXL file that describe layers included in the database, the database system composed of 2 subcomponents -ARCSDE to manage geographical capability and Oracle- and finally, a project specific directory to store layers not included in the database. 8

12 Page 9 Web site: Location: server ' ' located in the JRC DMZ (outside the firewall) ArcIMS service name: oceanides Path: /oceanides/oceanides Password: Contact ian.shepherd@jrc.it or philippe.carreau@jrc.it AXL file: Location: server ' ' located in the JRC network Path: /axloceanides/oceanidesa.axl Password: Contact ian.shepherd@jrc.it or philippe.carreau@jrc.it Oracle database + ArcSDE: Location: server= instance="port:5151" There is a special user to access the database ="oceanides" Password: Contact DMA responsible for the password Additional layers: Location: /oceanides./gisco for the gisco additional layer./. for the basic maps./statistics for the statistics layers Password: Contact ian.shepherd@jrc.it or philippe.carreau@jrc.it Content The Oceanides database is a geographical database containing information defined in the standard nomenclature proposed by work package WP2200 (develop standard nomenclature), existing geographical information layers in relation with the oil spill problematic (as European pipeline network or petroleum port) and oil spill statistics data. The standard nomenclature (see Annex I) includes information concerning spills together with their complete description and data concerning flight and satellite acquisition conditions. During the data collection process it became apparent that data concerning flight paths were mostly missing or inappropriate. Thus only two geographical layers are complete - 1. all_europe_oilspills and 2. satellite_overpass. Presently around 2300 oils spills have been integrated in the main dataset. We expect to reach 3000 inputs by the end of the project. Oilspill harmonized information following the standard nomenclature all_europe_oilspills satellite_overpass flight_track_data (empty) 9

13 Page 10 Additional data layers were collected from existing projects or database, such as the digital Chart of the World (DCW), GISCO and EUROSION. The layer maritime routes is not available in electronical format and probably have to be digitalized from a maritime chart. Collected additional information Sea depth (bathymetry) Petroleum port Ferry links Energy Transport Pan Europe Oil And Gas Points Energy Transport Oil Gas Pipeline Political boundary Maritime boundaries Maritime routes (expected) WEurope topo and bath EEurope topo and bath The statistics maps are delivered by the Work Package 3000 (Aggregating Oil spill information). Responsible partners can access the Oceanides database through a special service on Internet and perform analysis to produce new statistics layers that are then integrated. Statistic information Oils spill statistics and trend analysis layers (to be defined) Loading The real challenge with Oceanides is to harmonize data coming from several providers, from different countries and with different norms. This chapter won t discuss about the semantic or ontological aspects but only about format issues. The problem was how to create an efficient system to harmonize very heterogeneous format. It was decided to define a common standard input format in XML (See complete example Annex II) designed from the standard nomenclature. In addition because it is easier to create a text file than a XML file, a tool for converying text to XML has been implemented. The text file is tab column separated file where the column names correspond to the standard nomenclature (see example below). Two tools written in Java were developed to load information in the database, the first Exc2XML transform a text file in XML, the second oceanide_load loads the database from an XML file. 10

14 Page 11 Fig2: Conversion text file to XML file The advantage of such a system is more flexibility and reliability (fig 3). High flexibility and portability because there are several possibilities to create a map, thus it is possible to convert directly the text file in geographical file (Shapefile) using a GIS software or to load information in the database via XML. High reliability because there are several checks, one during text file to XML conversion and a second one during the loading phase in the database. This last step is required to create one single table from several dataset because during the project it was important for harmonization to load all the oils spill data and attribute in a single layer. 11

15 Page 12 Fig3: data loading process III Data management & update How to load new data in the database What do you need? 1) access to the data server from your computer 2) ArcSDE client installed on your computer 3) the executable exc2xml.jar to convert text file to xml and its parameter file exc2xmparam.xml. 4) the executable oceanides_load.jar with its parameter file param.xml to configurate the database connection and paramfile.xml to list the xml files to load. Loading data in the database is not an easy process because the system contains geographical capabilities that make it more complex than a normal geographical database. Thus do not use the JDBC or ODBC to modify the current tables but the ARCSDE API. 2 executables exc2xml.jar and oceanides_load.jar were developed with parameters files that can be configured (located in the same directory). Oceanides_load.jar reads a file and transfers its content into a database through a GIS tool and was developed using the Java API (set of libraries) furnished by ArcSDE (see ArcSDE documentation). So before using it is essential to 12

16 Page 13 1) have access and all the rights to the geographical database hosting server and 2) the ArcSDE client is correctly installed on your computer. The present configuration is for the system established in , you can change it in param.xml. What should you do? Step 1 Step 2 Step 3 prepare a tab file containing all the information you want to enter. convert the text file in XML by running the java application exc2xml.jar by configuring the parameter file exc2xmparam.xml load the XML file in the database by running the java application oceanides_load.jar by configuring the parameter files param.xml and paramfile.xml Step1 Prepare a text file containing all the information you want to enter. As described in chapter III (fig 2) the file must be a normal text file organized in columns separated by tabulations. Each column header must come from the standard nomenclature with the exact spelling (ex Oilspill_data, Oilspill_Lat). Another easier way is to create an Excel sheet (fig 3) and convert it in a text file separated by tabulation. For this in the Excel software menu select file/save as, then in the new opened window find the scroll list save as type, select Text (Tab delimited) txt and save the new text file. Fig 3: Excel file example Step 2 Convert the text file to XML by executing the java application exc2xml.jar with its parameter file exc2xmparam.xml Open exc2xmparam.xml and you will find an xml string as: <exc2xml> <filepath>c:/my_file.txt</filepath> <filepath>c:/my_directory/my_second_file.txt</filepath> 13

17 Page 14 < /exc2xml> Between the tag <filepath> enter the path of your text file you want to convert in xml. The new xml file will be created in the same directory with the same name but the extension.xml. If you have several files to convert you can add as many lines you want. Step 3: Load the XML file in the database by executing the java application oceanides_load.jar with its parameter files param.xml and paramfile.xml Open param.xml and you will find an xml string as: <param> <sde_param> <server> </server> <instance>5151</instance> <database /> <user>oceanides</user> <password>contact dma responsable</password> </sde_param> </param> Between tags sde_param are the server parameters. Presently the system is configurate to work on the JRC site. These parameters are described in the ARCSDE java API user manual. Contact your network administrator before modifying it. Open paramfile.xml and you will find an xml string like this: <paramfile> <file> <xmlfilepath>c:/mydir/oilslicks.xml</xmlfilepath> <db_table_name>oilspill_table</db_table_name> <dataset_index_begin>0</dataset_index_begin> <Source_Owner>denmark</Source_Owner> <Source_Distributer>helcom</Source_Distributer> <Source_Storage_Location></Source_Storage_Location> <Source_Storage_Date>15/09/00</Source_Storage_Date> <Remarks> This is the first file to load in the db</remarks> </file> <file> <xmlfilepath>c:/mydir/oilslicks2.xml</xmlfilepath> <db_table_name>oilspill_table</db_table_name> <dataset_index_begin>100</dataset_index_begin> <Source_Owner>belgium</Source_Owner> <Source_Distributer>bonn</Source_Distributer> <Source_Storage_Location></Source_Storage_Location> <Source_Storage_Date>15/06/01</Source_Storage_Date> <Remarks>This is the second file to load</remarks> </file> 14

18 Page 15. </paramfile> <file> tags contain all required information to load a new xml file. They are optional except xmlfilepath, db_table_name and dataset_index_begin. xmlfilepath is the path of the xml file you want to load including the file name. db_table_name is the name of the database table where information will be loaded. If it doesn t exist it will be created otherwise data will be added at the end of the existing table. dataset_index_begin; An index increases for each of the new oil spills added in the database. Thus you can decide by which number the index begins for each new added dataset. How to add new maps in the system 1. If your new map is not included in the ArcSDE database store the new map (shapefile or ArcIMS compatible format) in the directory /oceanides else jump in step Update the AXL file oceanidesa (location: /axloceanides/oceanidesa.axl) 3. Modify the configuration file for the layer list in /oceanides/oceanides/dbGroupToc/dbJTdata.js file All passwords and access instruction are given in chapter II sub-chapter 2) Access. IV Future development & conclusion Next step will be to complete the database and future development will aim to improve the database interoperability. Data are stored in the world most used commercial data format (Shapefile) but we want also to be compatible with the Open GIS Consortium (OGC) norm GML. To conclude during the first year the Oceanides team has developed not only a database but also a strategy to fill it. Nowadays several organisms coming from different countries collect oils spill data with very heterogeneous formats. The present database using the data model based on the standard nomenclature can be an excellent example and a step forward for future harmonized oil spill data collection. 15

19 Page 16 Annexes Annex I Annex II Oceanides standard nomenclature Oceanides input format example 16

20 Page 17 Annex I Oceanides standard nomenclature 17

21 Page 18 Annex II Oceanides input format example Remark: Each XML tag comes from the standard nomenclature described in annex I <?xml version="1.0" encoding="utf-8"?> <Oceanides> <oilspill_attr> <Oilspill_Date>03/01/02</Oilspill_Date> <Oilspill_Time>14:27</Oilspill_Time> <Oilspill_Lat> </Oilspill_Lat> <Oilspill_Long> </Oilspill_Long> <Oilspill_Length>30</Oilspill_Length> <Oilspill_Width>0.1</Oilspill_Width> <Oilspill_Area>3</Oilspill_Area> <Oilspill_Quantity>0.18</Oilspill_Quantity> <Meteo_Wind_Speed>3</Meteo_Wind_Speed> <Meteo_Wind_Direction>156</Meteo_Wind_Direction> <Remarks>SLAR and Visuel</Remarks> <Categorie>1</Categorie> </oilspill_attr> <oilspill_attr> <Oilspill_Date>05/01/02</Oilspill_Date> <Oilspill_Time>09:05</Oilspill_Time> <Oilspill_Lat> </Oilspill_Lat> <Oilspill_Long> </Oilspill_Long> <Oilspill_Length>0.1</Oilspill_Length> <Oilspill_Width>0.1</Oilspill_Width> <Oilspill_Area>0.01</Oilspill_Area> <Oilspill_Quantity>0.0001</Oilspill_Quantity> <Meteo_Wind_Speed>9</Meteo_Wind_Speed> <Meteo_Wind_Direction>270</Meteo_Wind_Direction> <Remarks>SLAR and Visuel</Remarks> <Categorie>1</Categorie> </oilspill_attr>.. </Oceanides> 18