Dl 9 Development and implementation of an IWRMS

Transcription

1 Project no: GOCE Project acronym: BRAHMATWINN Instrument: Specific Targeted Research Project Thematic Priority: Global Change and Ecosystems Project title: Twinning European and South Asian River Basins to enhance capacity and implement adaptive management approaches Deliverable Report Dl 9 Development and implementation of an IWRMS Due date of deliverable December 2009 Actual submission date December 2009 Start date of project: Duration: 43 Month Organisation name of lead contractor for this deliverable: Project homepage: Dissemination Level: FSU, CARR PU

2 Content Executive Summary... 3 Objectives... 5 The development of a component interface for the IWRMS toolset... 5 The IWRMS implementation at stakeholder sites Input of the what-if? scenarios Documentation of the IWRMS toolset Directory of figures Fig. 1: data management architecture... 6 Fig. 2: IWRMS modul structure... 7 Fig.3: Interaction between IWRMS core componnents... 8 Fig. 4 Asynchronous simulation backend... 9 Fig. 5 Gui time series generation for modeling data... 9 Fig. 6: What if scenarios results Fig. 7: Flash demo tour about IWRMS... 12

3 Executive Summary The WP_9 started in the 30th month of the Brahmatwinn project to develop a framework, which integrates different types of basin information and develops integrated water resources management (IWRM) options for climate change mitigation. The deliverables of WP9 comprise the results of the WPs 1 to 10 in an electronically database based format and delivers a graphical user interface for stakeholders with respect to their local infrastructure. That is the reason why all the work as development and workshops were done in the last project period when the basic knowledge of the River Basin Information System (RBIS), the DANUBIA modelling System and the NetSyMod approach with its modelling decision support system (mdss) component was gained by the stakeholders. All of the results of these other work packages have been put together by adapting interfaces for communication and data exchange between them as required by the specified what-if? IWRM scenarios. The developed integrated multi tier software system containing all measured and simulated data, was completely installed at the project coordinator site in Jena and with a local subset of simulated data at different project partner sites. The development and implementation of the Integrated Water Resources Management System (IWRMS) was done by linking the data of the River Basin Information System (RBIS) as climate modelling data for the Danube and Brahmaputra catchment. A special interface was needed for the integration of the enhanced DANUBIA model input and the NetSyMod with its modelling decision support system (mdss) component, to develop these three components to an innovative IWRM component toolset. The web based IWRM system contains and combines different types of data and methods to derive new catchment based data and decisions. To merge these different types of measured, predicted and simulated data, new interfaces based on RBIS data description definitions were developed and implemented. IWRMS is based on a three tier software framework which uses html/javascript at the client tier, the PHP programming language to realize the application tier and a postgresql/postgis database to manage and storage all data, except the DANUBIA modelling raw data which are file based registered, in the database tier. All different tiers can reside on one or different computers and considers the local infrastructure. The component interface for the IWRMS was the core component to integrate the DANUBIA modelling output and the NetSyMod data and methods; the design and workflow proposals were gathered by the FSU, codematix, LMU and FEEM and presented at the 2 nd annual project workshop in Munich in October At the following field trip different stakeholders from Austria and Germany were visited, the concept was represented to get a detailed feedback of European needs. Together with the ICIMOD and CARR partners, codematix and FSU improved the first proposal and introduced the planned architecture and graphical user interface during the stakeholder workshop (November 2008) in Kathmandu. At this workshop, organized by ICIMOD, FSU, LMU, FEEM and codematix, a first prototype for the IWRMS was presented and tested by the Asian stakeholders from China, Nepal, Buthan and India. The implementation process at stakeholder sites was driven by the usability aspect of the amount of climate and hydrological raw modelling data.

4 Especially the requirements of an easy to use toolset respective to the low bandwidth Internet structure were figured out. At the workshops in Munich and in Kathmandu (Oct/Nov 2008) the management of the huge amount of simulated data using the raster based DANUBIA modelling system was discussed as well. A solution for a tributary based management was delineated and later implemented by codematix. The IWRM system architecture and the interface for the data and methods considers the different infrastructure of European and Asian stakeholders, the system is based on a complete replicated database and web based application structure which uses advantages of inter- and intranet applications and do not need further local software installations or configurations. The leading host system managed by the FSU, contains the five terabyte of simulation data, subsets of them, e.g. for tributaries, can be selected and downloaded on the fly by all stakeholders. The realized approach of asynchronous data preparing processes collects all catchment relevant data from the DANUBIA output, which resides on the central IWRMS server in Jena and sends the data in a time series based format to the local IWRMS database. The result is described by metadata as well. From the point of view of the stakeholders, complexity of DANUBIA results and the amount of data is hidden and reduced to a small local IWRM system, which loads extended data on demand. The similar approach is used continuously for data and methods (e.g. the mdss program mulino and its results (DI_8)). The data exchange is described by XML based description files, the data interface can be extended to other modelling systems with a raster based output. The main problem for this task was the integration of the catchment based data for the IWRM system, especially at the stakeholder sites, which follows from the amount of simulated DANUBIA data. The automated installation process and the use of the system were instructed at different user workshops. A video tour/tutorial and a user manual about the IWRMS offer beginners the easy use of the system. First practices to achieve DI_9.2( IWRMS implementation at stakeholder sites together with the data) were done at the workshop in Kathmandu in November The workshop was also used to evaluate the developed IWRMS replication concept, which is partly (transfer extended data on demand) described above. The installation of the system and the catchment core data was realized by the concept of a virtualized computer environment. The concept behind this approach is to replicate new data of the main IWRM system to the stakeholders periodically. The installation procedure of the IWRMS needs at the first time local administration rights, later updates can be done by a common copy command. The DI_9.3 (Input of the what-if? scenarios) was done by the WP leaders of WP_2 to WP_8 with the help of FSU and ICIMOD. This loaded real world information are assigned to (spatial) metadata, associated time series and indicators, which describe the what-if? scenarios and are the base for evaluating of adaptive IWRM options for climate change mitigation. The procedure was represented at the Kathmandu Symposium in Nov too. To realize the DI_9.4 (Documentation of the IWRMS toolset and its components) comprehensive movie based use cases of the whole IWRM system were developed and deployed. Each function of the IWRMS is described and explained. All steps are simulated by a flash movie / tutorial and can be actuated by the user. The movie was published in September 2009, all methods were tested and improved by ICIMOD and the Asian and European partners, the developed web map services (WMS) and the integration to other information systems as the use in geographical information systems (GIS) were tested by the University of Jena.

5 Objectives The main objective was the development of an comprehensive toolset to include data and methods, based on the deliverables of the River Basin Information System, the derived real world data, the results of the DANUBIA modelling system and the NetSyMod - mulino Decision Support System mdss). The deliverables of all preceding work packages are partly included and joined in the Integrated Water Resources Management System (IWRMS) and can be used by all stakeholders using a web-based graphical user interface. The result of the work package is an easy to use IWRMS application which extends the RBIS approach and deploys all basin specific data with consideration of the local computer network structure. To achieve this, the following work was done. The development of a component interface for the IWRMS toolset The IWRMS was developed by means of GUI handling the data and information exchange between the DANUBIA hydrological model, the RBIS, and the mdss. By providing a methodology for data exchange, between applications the interface of the IWRMS comprises (i) the descriptions of how applications want to access input and output data in the database, and (ii) what kind of API methods are used by applications to access the data. The design of the IWRMS is based on the structure of the existing RBI system. From the point of the users/stakeholders it was not necessary to get further new skills to use the system, the data and application handling is similar to the existing system. All modules and interfaces of the IWRM system are modular extensions and can be added to on demand. The following requirements were figured out by the partners: (i) (ii) metadata and space/time description for all new datasets including of different data types and their interactions among each other (how applications access data) grouped by GIS data (raster and vector) as geology, land use, river network, vegetation, etc., according the input modeling data Measured time series (climate, radiation, precipitation) of the partners for the whole catchment Raster data regionalized data of Global Circulation Models (GCM)DI_2 for different scenarios (historical, 2020,2050,2080), as input for WP7 an WP8 (grid cell of 50 km) Output and configuration of the hydrological model DANUBIA (15 modeled parameter (precipitation, evaporation, etc.) and downscaled meteorological data of WP2 (grid cell 1 km) Results of mulino (analysis matrix, evaluation matrix, weights and options for indicators, sensitivity analysis, decisions) Integration of the Special Report on Emissions Scenarios (SRES) scenarios Integration of indicators Any type of documents (diagrams, pictures, spreadsheets, word, pdf, etc.) Assignment of catchment, indicators and scenarios Web based access to delivered original (e.g. xls) and derived (e.g. graphs) data

6 Fig. 1: data management architecture The derived data fluxes and necessary web services in the Brahmatwinn project are summarized in Fig. 1: data management architecture. The following architecture and graphical user interface (gui) considerations were made during the meetings. It is desirable to use a known screen design. In opinion of the Asian partners the use of the existing RBIS needs already a lot of skills especially in terms of the map, web map service (wms) definition, and user management. Therefore the integration of further data, functions and programs as the DANUBIA modeling results, the indicators or the mdss mulino system should provide a similar web based approach or an adaptable interface. As a result the final IWRM system was designed with a set of adaptable and RBIS compatible extension modules. The programming language is PHP; all data are stored in a set of databases. There is no distinction between alphanumerical, location based and binary data items. The underlying database uses spatial based extensions (postgis) according to the open geospatial consortium (ogc) standards.

7 Fig. 2: IWRMS modul structure The developed modular extensions can be used by the user on demand. The idea behind this functionality is to reduce the complexity of the application in dependency of the logged in user. Each module of the IWRM system delivers a set of different functions, which can be combined to a new workflow facility. The rights to use this functions or workflows can be granted or revoked by a higher privileged user. In dependency on the role, the menu structure and screens are changed. This approach reduces the complexity of the system in depend on the working level tremendously. The system of fine grained user based functionality is assigned to the managed data as well. Different rights (hidden, read, download, write, owner) can be granted to on value, indicator, map, plot, time series, layer, etc.. To avoid a set of only readable data islands, the partners agreed, that at least the readable right should be assigned for all data in the Brahmatwinn project. The implemented interaction structure between external applications is shown in Fig.3. The data fluxes can be distinguished between the mdss mulino and the DANUBIA modeling system: mdss mulino needs a tight coupling, because data will be exchanged between both systems, especially indicators from IWRMS are used in mulino. DANUBIA needs a lose coupling, the original modeling results (raster data with cell values for each parameter and for each timestamp) are linked and registered only. Used time series in the IWRMS are generated (derived from the original data) on demand or for exposed locations in the tributaries only (e.g. climate, precipitation or gauging stations).

8 Fig.3: Interaction between IWRMS core componnents The IWRM system integrates the results of all preceding deliverables. The different data interfaces and definitions were built up for all included WP data. One objective for the developed half automatically interface was to derive as much data as possible without user efforts. Derived data could be metadata from file headers, aggregated sum or average values, responsible user / organisation data or converted reference system data. This approach was presented in Munich October 2008, a first local test installation was installed at ICIMOD in Kathmandu in November At the stakeholder workshop the planed interaction between mdss mulino, the DANUBIA modelling system and IWRMS was shown at a first prototype. During the middle of 2009, project partners FSU, LMU, ICIMOD, FEEM, UniVie and codematix defined the end design for the IWRMS according the user needs, which were figured out together with the stakeholders. The final function-matrix and system/interface architecture was defined as well. The implementation of the IWRMS modules, based on the defined formats and data fluxes, was done by SME codematix. The implemented workflow considers the existing data amount and the requirements of a web based application from the view of a stakeholder. To read a simulated time series of a parameter, for each time step a separate file has to process. All this work is done by asynchronous processes. After registering new data, the first process derives associated metadata. The user starts then the generation of simulated time series according to the selected simulation parameters and the spatial based locations (Fig. 4). All needed data are transposed form the raster based formats to time series with parameter sets which are central stored in the IWRMS database or can be downloaded and local proceeded. The end of this process is not assigned to any stakeholder/user action, so the further work is independent form all data processes. After generating the time series of assigned simulation data, all times series functions of the IWRMS can be used similar to measured data. The management of the basin specific (different available parameters) data was presented and trained in November 2008 in Kathmandu. Hints and improvements for the time series management were implemented in 2009, especially further automated filling gaps methods for the measurement data.

9 Fig. 4 Asynchronous simulation backend Another designed and implemented function is the integration of virtual stations and their asynchronous time series generation of DANUBIA modelling data. This function uses parts of the workflow described above, but is completely integrated in the map application component of RBIS (RBISmap, Fig. 5). With this function, stakeholders can generate data for any point or area in the simulated catchment. This approach delivers an on demand processing method which combines the data from the remote central server (e.g. at the FSU, mentioned above) with the local catchment server. The resulting time series of a remote server call can be transferred to the local server. Metadata are added to the new virtual station too, so also all RBIS searching functions can be used. To avoid an overloading of the central server, the number of possible simultaneous processes are limited, further requests are queued. At present it takes approximately four minutes to generate a complete time series (1970 to 2080, one hour time resolution) with a complete parameter set (20 parameters) for one virtual station. At the user management tier, this function is also assigned to a workflow, which can be switched on or off, depending on the user rights. Fig. 5 Gui time series generation for modeling data The integration of mdss mulino integration was made using the same modular approach (RBISmdss). IWRMS manages different mdss core program versions, the basin specific configuration files and the associated results of different Delphi rounds. During the stakeholder workshop in November 2008 in Kathmandu, the use of mdss in a local environment was taught by the FEEM group. IWRMS can be used to represent the results, together with all metadata relevant information or to run the decision process again locally. Therefore the stakeholder can varying general regulations and adapts the mdss

10 configuration to its local needs. Different runs can later be added together with new metadata to IWRMS in dependency of the user rights. The IWRMS implementation at stakeholder sites Suitable sites for the implementation of the IWRMS were selected in the UDRB and UBRB based on the stakeholder processes. In terms of the partly different European and Asian infrastructure of the Internet bandwidth and the amount of simulated data, it was required to preconfigure local installations with data from the central server in Jena (FSU). The methodical approach was used to built up a pack and go process for existing server data. At the destination servers, no further compilation processes of source code are needed; the images deliver complete preconfigured web based server architecture for the IWRMS at the stakeholder site. For all implementation sites the measured time series data and all socio-economical data of the whole Danube/Brahmaputra basin were included together with a subset of simulated data as described before to a new site specific server image. The resulting server images have had sizes between four and five gigabyte. To deploy the server images and keep the installation process simple, the open source software virtual box technology was used. The installation steps of the image are self explained and comprise for a later update a simple copy of the image file. These steps were presented first time at the stakeholder workshop in Kathmandu; the actual image can be downloaded from the ftp server. After installation at a local PC or in a LAN environment, IWRMS can be used without any explicit server starting or further network configuration. Periodical updates can be made by downloading a new packed and gone stakeholder image from the central server or the updating of less data amount directly. Installations were done at the sites of the partners FSU, LMU, IIT, ICIMOD, UniBu, ITP and CARR. After installation the researchers working with the IWRMS were introduced to run the system as part of the capacity building program in DI_11. The installation process, the user handling and the actual content of the IWRMS image were intensively final tested by ICIMOD in December Input of the what-if? scenarios The integration of indicators and scenarios based on the different climate and hydrological modelling data was realized by designing and implementation of the modules RBISscen and RBISind. RBISscen delivers an extended database structure and PHP application to integrate the different SRES scenarios with linkages to the metadata, catchments, socio-economic data and the hydrological modelling results. RBISscen uses the same programming technology mentioned before. RBISind integrates indicators of different types as values, normalized values, documents (diagrams, pictures, spreadsheets, word/pdf documents, time series, etc.) and assign catchment, indicators and scenarios. The IWRMS toolset was loaded with the real world information supplied by Dl_2 to Dl_6 and the environmental parameters of the what-if? scenarios delivered by Dl_2, Dl_8 and Dl_10. These steps were partly made in DI_9.2, because e.g. the stored DANUBIA modelling results have linkages in their configuration to source or input data from DI_2 to DI_6 (climate data, socio economic data, etc.). All

11 what if scenarios were linked to IWRMS metadata as responsible persons, organisations, spatial location based data (stations an associated time series) and further more (e.g. any kind of specific binary documents). This provides the knowledge basis for the application of the IWRMS toolset for developing and evaluating adaptive IWRM options for climate change mitigation. The procedure was tested and further improved in cooperation with the Asian partners ICIMOD, UniBu and CARR; and FSU, LMU, IIT, stakeholders and end users. To meet the stakeholder needs, the results and dependencies of different indicators are saved in the original formats (e.g. xls), in the database format and derived representative forms (e.g. graphs or maps, Fig. 6). Fig. 6: What if scenarios results The deliverables were tested by the partners to provide a user friendly, accepted tool system for the partners and stakeholders, containing information from project data to adaptive IWRM options for climate change mitigation. The following general system requirements and functionality were figured out by the stakeholders and tested in RBIS and extended IWRMS in December 2009: User friendly graphical user interfaces (GUI) for populating the system based on the full implementation of the ISO standard and carrying out data and information queries. Representation of the river basin real world by differentiating between o o Generic components that are represented in all river basins Specific basin related components that reflect the regional conditions Import and export functionality and management of GIS digital data layers that have a geographic reference specified by a common coordinate system. Integration of the different kinds of data and information for the design of what-ifscenarios applied for the evaluation of prognostic system management alternatives in the decision making process.

12 Adaptively of the system s data model structure to allow for extensions depending on the progress of the system s understanding and knowledge achieved by researchers and decision makers. Documentation of the IWRMS toolset The system is well documented using user manuals, online help for each screen and item which can be selected or inserted. A flash based interactive demo tour /tutorial (Fig. 7) describes all functions and steps to work with the system, insert new metadata, upload GIS layers, create maps, manage time series (insertion, management of gaps, interpolation procedures), simulation data registration and interactive time series derivation, indicator management. The guided tour is harmonised with the user manual and can be executed in a internet browser environment or standalone without any player software. The flash tutorial is integrated at the website Fig. 7: Flash demo tour about IWRMS In finalizing the research work of the work package all results were properly mapped, documented, reported and presented as the deliverable Dl_9 to the research consortium of BRAHMATWINN during the workshop in Kathmandu in November 2009 at the end of the third project phase. The tutorial was presented first time in September 2009, different changes also in terms of the final testing procedure were made in December All deliverables were accepted by the consortium, the whole work package and its integration to the Brahmatwinn project, the use, the benefits and the further development were presented at the Brahmatwinn symposium (Kathmandu 2009, 8 th -10 th November).