Delft-FEWS2020 in your organization

Transcription

1 Delft-FEWS2020 in your organization Impact and scope of the 3 roadmaps implemented Break-out Presentation DFUDA 2018 Gerben Boot (Deltares) 3 rd of May 2018

2 Overview Impact of the Delft-FEWS version Questions and discussion

3 Delft-FEWS 2020 Roadmaps: Deltares Open Archive Delft-FEWS PI Webservice Delft-FEWS Backend archive server archive data external applications pi web service operator client compute nodes database admin gateway rest web service

4 Impact of the Delft-FEWS version Delft-FEWS : first version with Delft-FEWS 2020 roadmap features JMS (activemq) removed: new (DDA) communication and heartbeat mechanism New data synchronisation mechanism (MC-OC & MC-MC) New rollingbarrel mechanism (Temporary) new look of Admin Interface New ForecastingShell registration mechanism Configuration Manager working with DDA Delft-FEWS Backend

5 Impact: Installation package Package and Installation/Upgrade Client package Server side package Log collector service Configurator tool Delft-FEWS Backend

6 Database update Database (update script) Migration from or earlier: Database schema is (completely) new Most important change: unique identifier across multiple MCs: globalrowid this is added to all records during upgrade (tool!) Make backup (No rolling back possible!). Cause globalrowid Minimum version for Oracle (Oracle 12c) Delft-FEWS Backend

7 Impact: Database Initialization Tool Pre-run checks Tool runs once for all configured/connected databases All existing rows will get a unique globalrowid All configured databases will get a new sequence offset that will be used to create new globalrowids records are now labelled (creator DB) All Default* configuration tables will be migrated by setting the synchlevel of the corresponding table row to 11( = default configuration). Drop any Default* configuration tables. Drop the RecordPendingDeletion table. Post-run checks: integrity & validation /path/to/java/jre/bin/java -jar database-initialization.jar clientconfig.xml Delft-FEWS Backend

8 Impact: Master Controller No change: MC-package (containing several classes to be extracted, installed) MC binaries: smaller, code reduced Installation of: MC classes LogCollector service Delft-FEWS Backend

9 Impact: Forecasting Shell Servers Main change: once installed the FSS registers itself /signs-in at the MC Installation of FSS requires: Extracting the FSS folders (as usual) Installing the FSS service (as usual) Switching on the service (as usual) LogCollector service (new) Delft-FEWS Backend

10 Impact: Admin Interface Changed: (temporary) look & feel of main page More (status/metrics) info on: Connected databases Taskruns (forecast tasks, system tasks) Delft-FEWS Backend

11 Impact: New Synchronisation Mechanism: FSS: DDA FSS writes directly in central database OC OC writes directly in central database (e.g. manual edits) and downloads them (irrespective of DDA or synchronized OC) MC-MC No change in concept: MC-MC sync is a pull (=download) Several configurations possible Support up to 40 MCs Database replication (= create LDS): same mechanism, code Delft-FEWS Backend

12 Impact: New RollingBarrel All forecast records have expirytime columns, also config tables DeletedRows table replace RecordsPendingDeletion, RecordsPendingUpdateExpiry New RollingBarrel collects globalrowids for expired rows and stores them in the DeletedRows table (synchronized) extends tasks and taskruns to the greatest expiry time for child timeseries DeletedRows are deleted, recorded in ProcessedDeletedRows Delft-FEWS Backend

13 Impact: Configuration Manager: DDA only Delft-FEWS Backend

14 Simplified usage Import: local files direct into central database Validate: validate files against xsd (schema) Save all: create a new config-revision of all new imported files Delft-FEWS Backend

15 Simplified usage Set active: make this file (row) the active version (yellow background) Set inactive : make this file (row) inactive (white background) Delete: deletes this file (row) from the database To confirm changes: Save all: create a new config-revision of all changed (default) files Delft-FEWS Backend

16 Impact: Operator Client Not changed: User Interface Operator Client has not changed Configuration DDA Client has not changed Changed: Synchronisation client does not use ActiveMQ (JMS), but VJBC Root-configuration has changed (part of installation procedure) Simplified synchronisation configuration Delft-FEWS Backend

17 Impact: Stand-Alone Not changed: User Interface Stand Alone (SA) has not changed Configuration DDA Client has not changed Remarks: When starting a SA with an older (than ) LDS, you can: Convert LDS to new database-scheme cannot use LDS in older versions of Delft-FEWS No conversion of LDS can work with can use LDS in older versions of Delft-FEWS

18 Impact: Delft-FEWS PI Webservice Delft-FEWS PI Webservice: 1. Single installation file for all webservices 2. No additional configuration Tomcat 3. One Test page for all webservices 4. Tests scripts using the curl examples 5. PI SOAP and PI REST allignment 6. URL change for PI REST service

19 Main page for testing

20 Test page for PI REST web service

21 PI Webservice: dynamic html page with graph (example)

22 PI Webservice: dynamic html page with map (example)

23 Impact: Deltares Open Archive Easier to install Configuration can be done by any configurator without archive knowledge Elastic search at configurable location Simplified Thredds installation Updated Documentation Web Viewer Third party applications Delft-FEWS PI-webservice Tomcat THREDDS Tomcat Catalogue webservice FSS other data harvester Index files ElasticSearch index

24 Installation: made much simpler config deployment

25 Impact: Deltares Open Archive (admin interface) Easier to maintain Health checker Upgrade web framework archive server (JSF) System metrics Data management Use multiple storages Incremental harvester Verify export (netcdf and metdata files)

26 Impact: Deltares Open Archive More integrated with FEWS Forecast management dialog New tab with archived runs information Performance seamless integration (additional index to database) Seamless integration in workflows System monitor (OC)

27 Impact: read about it Delft-FEWS WIKI New installation documentation (final stages) Install from scratch Upgrade from New technical documentation (work in progress)

28 Questions & Discussion