DATA MANAGEMENT MODEL

Transcription

1 DATA MANAGEMENT MODEL InfoCoSM Arto Vuorela Vesa Roivas Jean-Pierre Houix

2 INTRODUCTION This report is an intermediary report summarizing the main objectives to be reached by the end of July (prior to the Steering Committee meeting) and the achievements reached so far. The overall objectives for the InfoCosm project have been stipulated by D. Frank-Kamensky during our last visit in St. Petersburg (13-16 June 2006). The core of the system is made of three distinct entities: a)data block containing data, metadata and data dictionary; th th b)a set of thematic maps made of layers and attribute data selected from the data block and stored on a server; c)thematic maps are accessible through information services using ArcIMS, and allowing querying, selecting and exporting the data. Users are distributed into two groups: registered and unregistered. GENERAL USER REQUIREMENTS FOR METADATA SERVICE 1 User groups The target groups for data user s has been generally defined as registered users, and unregistered users. The level of permission granted to registered user is defined on a case-by-case during the registration process: the latter will have access to a certein number of information services. Non registered users have access to descriptive information about the data but not to the data itself: they may browse for information rather than downloading it. Registered users can be split into three groups depending on their competencies: a)administrative officers (e.g. ECAT officers); b)territorial authorities c)external users that are granted access following registration. General user groups are as follows: User group User Permissions for metadata Permissions to data Administrative officers Administrative officers in city level. Browse thematic class metadata and get brief information about their source data Get description of raw source data (data structure and field descriptions) Create & update metadata (for certain users) View all data in database and make queries to data Edit data content (for certain users). Process data and create new themes (for certain users).

3 Territorial authority Users in district level. District ecologist that needs access to all data in database and need to preprocess it for analysis in local use. Browse thematic class metadata and get brief information about their source data Get description of raw source data (data structure and field descriptions) Send request to modify metadata. View thematic maps with ArcMap Make queries to map objects. View all data in database and make queries to data Download raw data from the server. Possibly : update data in UDB External registered users Municipal units, advanced user like scientists and students. Users in other governmental bodies Browse thematic class metadata and get brief information about their source data. Get description of raw source data (data structure and field descriptions) View thematic maps with web map browser or in ArcMap etc. Make queries to map objects. Make case-by-case contract for getting specific type of data from data base. Unregistered users General public, citizens Browse thematic class metadata View thematic maps with web map browser or in ArcMap etc. Make queries to map features. 2 Registered users and information services Registered user permissions will be further restricted at both functional and data levels. The functional level describes what kind of operation user can run with the data e.g. view, download, query, or edit. Certain features or attributes will be neither accessible nor visible for some users even following registration (e.g. some user group might have access to company database to make queries about their location but not access to company s emission information). Functional levels will be organised into information services grouping pre-defined operation a user may perform within a dataset. This suppose however that users' needs have been clearly identified so that information services will match their effective needs; testing information services is one objective of the InfoCosm project. 3 Unregistered users Access via internet to browse certain set of thematic map presentations. User can search for certain theme using: d)key word (e.g. air quality ). e)select area from map and target search for that area. f)select theme dataset from list of available theme map datasets and their subset datasets

4 As a results from search or selection, the list of thematic feature classes and their short description is showed to the user. User can select one theme and obtain its detailed information using a link to metadata about data source, data process, and also possibly a snapshot of the data (illustrative information). User can open map view to browse live thematic layers. There are some pre-defined basic map themes in background that user can turn off and on individually. Public user can also make queries and browse the data using on-line viewer, can print the result, get statistics, but can not download raw data. 4 Data restrictions Because the database includes only legally valid information, the permissions to manipulate the data in database should be hardly granted to certain users only. The data updating process should follow well defined steps and that process should be documented for each type of data. The traceability of data should be guaranteed at each step of the processing. Illustrating and fixing data manipulation is the objective of the data dictionary that emphasis every steps in data acquisition, processing and up-dating. Data dictionary will be developed for a certain number of information services with the InfoCosm project. 5 Data mining Search for certain data types saved into unified database, get general information about their content and make access to theme if necessary. Metadata is arranged according to logical hierarchical data structure to information blocks, layers and data objects. User can navigate in metadata service (web pages) through hierarchical structure or search by using key words or geographical extent. 6 Requirements for infrastructure A permanent broad-band internet connection is required for each workers within ECAT organization in city and district level to reach most updated information from database. If two-way data traffic (upload / download) between offices was considered, this is mandatory. Otherwise the system design turns to be more complicated and error-prone e.g. local database replication is needed and more software licences (for DBMS and ArcSDE). In practise there exist already suitable internet connection almost in all offices. This point should be however clarified also from the regulation point of view. 7 Software requirements The central database in ECAT will be based on SQL Server DBMS and ArcSDE applications. ArcGIS 9 software are already in use at City office. However there are no ArcGIS licences in Kronstadt. One should consider the cost for purchasing licences for ArcGIS and take into account the whole prime and annual costs for maintaining those software in all locations. Alternative software may be use (Arc Explorer or open-source software). However, data manipulation and therefore information services extends, will greatly depend on the available software, hardware, and broadband capacities.

5 The InfoCosm project is to proposed a software and hardware architecture as well as to provide broadband capacities minimum requirements. METADATA MANAGEMENT 1 Metadata from data provider The stakeholder provides data to ECAT with particular specifications. Specifications are based on ECAT s application schema that defines feature classes and the related data and its attributes and relationships in ECAT s data base. Specifications are derived from application schema. Data producers can not violate the conventions in use without modifying the whole application schema. Therefore, data producer will have to stick to the data schema provided or the data they provide will have to be re-processed before being introduced into the database. Data provider should deliver XML metadata file for each delivered feature class or dataset (group of files) as agreed between ECAT and the data provider. ECAT provides template for metadata document for certain type of feature class and possibly also the tool or interface by which data provider will produce the XML metadata document. The metadata should include general core elements (contact information, linkages, sources.) and attribute descriptions; this information could be pre-filled in metadata template provided by ECAT to reduce work load. Field descriptions are based on ECAT s application schema as described in the Data dictionary. When data provider follows data specification, the newly produced metadata documents will automatically update existing ones; in opposite case, data and metadata will have to be reprocessed. The InfoCosm project is to provide providers' metadata structure, templates, and management schemes. 2 Metadata and data dictionary Each entity included in the database should be documented by both its metadata and the data dictionary. The data dictionary describes only the data logical structure (e.g. ER-diagram) such as entities, relations, fields description, but not actual data contents e.g. data quality, producer, dates etc. The actual content of the data set is described into metadata. Because the data is collected from different sources at different times, the metadata management is more complex due to data heterogeneity. A three-level metadata structure based on ISO could be proposed as follows: 1.Metadata on metadata (who is managing the metadata) 2.Dataset identification 3.Reference system 4.Distribution information (data set owner, originator, processor, and access rights) 5.Other dataset information (description of the data set).

6 Note that metadata can be attached to datasets and/or to each of the data (to layer, tables, and other documents) within a particular dataset. The metadata describing the whole dataset so likely to present general information and will not describe the individual features or attributes. A second level of metadata may be therefore used for fine-tuning operations.

7 PROPOSED IMPLEMENTATION PLAN Steps 1 to 6 would be implemented for the next reporting period (August 2006) ; steps 7 to 14 until December Define information services and the related user requirements for database application. 2.Design data model / application schema for each information service. 3.Document data model. Use of graphical UML modelling language to create conceptual data schema and ER-model to describe physical data model. Create data dictionary to describe data, it s attributes and relations. Use consistent name conventions. 4.Design pilot application schemas for different datasets that ECAT could use to require from data providers. 5.Design metadata schema for a.data provider b.database objects c.thematic layers (the first priority) 6.Design logical functionality of database application (data migration processes, metadata handling processes, data administration). 7.Implement physical data model into ArcSDE / SQL Server running in ECAT. 8.Implement data based application. 9.Design and implement (stand-alone) metadata editor for data providers. 10.Design and implement metadata editor to edit metadata service for internal data mining and for thematic layers (the latter has the first priority). 11.Set up information services for both registered and unregistered users. Define user authentication rules for metadata and information services. 12.Customize ArcIMS based metadata explorer for: a.unregistered users b.registered users 13.Purchase ArcView 9.1 software for Kronstadt or install alternative software. 14.Train people to use metadata services in Kronstadt and to use ArcMap software.

8 Figure 2. Conceptual schema of general data model in ECAT database

9 Figure 3. Proposed metadata structure Field name Level 1 Level 2 Level 3 ISO Number Description Defines the metadata on the data set Metadata on metadata Point of contact Responsible organisation and individual for the metadata Organisation name Responsible organisation name Individual name Responsible individual name Position name Role Address: Delivery point Address: City Address: State, province Address: Postal code Address: Country Address: The responsible individual role or position in the organisation Function performed by the responsible organisation Address line for the location City of the location State, province of the location Postal code of the location Country of the location The electronic mail address of the responsible organisation or individual Last modified 9 Date of the last modification of the metadata Name of standard 10 Name of metadata standard Dataset identifycation Version of standard 11 Version of the metadata standard Basic information's required to identify the dataset Title Title of the dataset Alternative title Alternative titles of the dataset Brief Abstract 15. Brief abstract explaining in short the content of the dataset Abstract An abstract explaining the content of the dataset

10 Keywords Keywords helping to classify the dataset Topic category An predefined ISO category Dataset version Version of the dataset Reference system Reference date Date of last modification to the dataset Definition of the reference system used for the dataset Name Name of reference system Datum Identity of the datum Name Name of datum Ellipsoid Identity of the ellipsoid Name Name of ellipsoid Semi-major axis Radius of the equatorial axis of the ellipsoid Axis units Units of the semi-maior axis Project ion Flattening ratio Ratio of the difference between the equatorial and polar radii of the ellipsoid to the equatorial radius when the numerator is set to 1 Identity of the projection Name Name of projection Standard parallel Longitude of central meridian Latitude of projection origin False easting Line of constant latitude at which the surface of Earth and the plane or developable surface intersect Line of longitude at the centre of a map projection generally used as the basis for constructing the projection Latitude chosen as the origin of rectangular coordinates for a map projection Value added to all "x" values in the rectangular coordinates for a map projection. This value frequently is assigned to eliminate negative numbers. Expressed in the unit of measure identified in planar coordinate units

11 Distribution information Owner Originator False northing False easting northing units Scale factor at equator Longitude of projection centre Latitude of projection centre Value added to all "y" values in the rectangular coordinates for a map projection. This value frequently is assigned to eliminate negative numbers. Expressed in the unit of measure identified in planar coordinate units Units of false northing and false easting Ratio between physical distance and corresponding map distance, along the equator Longitude of the point of projection for azimuthally projections Latitude of the point of projection for azimuthally projections Information about the distributors of the dataset Information about the owner organisation Organisation name Name of the owner organisation Individual name Name contact person in the owner organisation Position name Position of the contact person in the owner organ isation Role Always "Owner" role Address: Delivery point Address: City Address: State, Province 383 Address: Postal code 384 Address: Country 385 Address: Organisation name Individual name Address line for the location City of the location State, province of the location Postal code of the location Country of the location The electronic mail address of the owner organisation or individual Information about intellectual creator (person and/or organisation with intellectual rights) of the dataset Name of the creating organisation Name contact person in the creating organisation Position name Position of the contact person in the creating organisation Role Always "Originator" role

12 Proces sor Address: Delivery point Address: City Address: State, Province 383 Address: Postal code 384 Address: Country 385 Address: Address line for the location City of the location State, province of the location Postal code of the location Country of the location The electronic mail address of the originator/creator organisation or individual The technical producer or processor of the data Organisation name Name of the processor organisation Individual name Name contact person in the processor organisation Position name Position of the contact person in the processor organisation Role Always "Processor" role Other dataset information's Access rights Address: Delivery point Address: City Address: State, Province 383 Address: Postal code 384 Address: Country 385 Address: Type of constraint Address line for the location City of the location State, province of the location Postal code of the location Country of the location The electronic mail address of the processor organisation or individual Defines access rights for the dataset The type of access right applied to assure the protection of privacy or intellectual property, and any special restriction or limitations on obtaining the resource Restriction Description of the restriction of the access right Other aspects explaining the dataset Language Language used within the dataset Format Defines exchange format

13 Name Name of the used exchange format for the dataset Version Version of the used exchange format for the dataset General explanation of the data producer's knowledge about how the geometry was Methodology description constructed/derived and how the attribute information being part of the dataset was generated. Changes 18. Changes Description of the changes since last version of the dataset Process steps Information about the event in the creation process of the dataset Description Description of the process step including related parameters or tolerance Resource name Name of the resource used in process step 0 Resource date Date of the resource used in process step Scale Gives a rough value of accuracy of the dataset; e.g means dataset has an accuracy suitable for use at scale 1:2.5 million at best Geographic accuracy Geographic accuracy of location, ground distance as an value in meters Geographic box West bound longitude East bound longitude South bound latitude North bound latitude Geographic position bounding box of the dataset Western-most coordinate of the limit of the dataset extent, expressed in longitude in decimal degrees (positive east) Eastern-most coordinate of the limit of the dataset extent, expressed in longitude in decimal degrees (positive east) Southern-most coordinate of the limit of the dataset extent, expressed in latitude in decimal degrees (positive north) Northern-most coordinate of the limit of the dataset extent, expressed in latitude in decimal degrees (positive north)