Challenges in Extending, Testing, and Delivering FEWS

Size: px

Start display at page:

Download "Challenges in Extending, Testing, and Delivering FEWS"

Rodney Ross
5 years ago
Views:

1 Challenges in Extending, Testing, and Delivering FEWS Lee Cajina,, U.S.A 2014 FEWS-User Days (10/28/2014)

2 Presentation Outline What is the Community Hydrologic Prediction System (CHPS) and How is it Related to FEWS? Customers Providers Developing Software Against FEWS Challenges Strategies Releasing Updates to FEWS Challenges Strategies Conclusion Looking Ahead Slide 1

Background The NWS has almost completely transitioned from a mix of standalone programs under the umbrella name of the National Weather Service River Forecast System (NWSRFS) to a FEWS-centric

3 Background The NWS has almost completely transitioned from a mix of standalone programs under the umbrella name of the National Weather Service River Forecast System (NWSRFS) to a FEWS-centric environment using a mix of functionality adapted to FEWS under the umbrella name of the Community Hydrologic Prediction Service (CHPS). Primary Drivers Were: Future Maintenance and Support Reducing the Time from Research to Operations NWSRFS CHPS Slide 2

4 CHPS vs FEWS CHPS Consists of 4 packages 1. FEWS 2. OHD-CORE-CHPS, 3. HEC-RAS, 4. HEC-RESSIM FEWS provides the CHPS modeling environment and primary graphical user interface OHD-CORE-CHPS, HEC-RAS, and HEC-RESSIM Are external packages with functionality that can run within FEWS Each has their own FEWS Adapter(s) Slide 3

covering the US RFCs are federal entities tasked to provide fresh

5 CHPS Customers NWS Headquarters () Deltares USA () CHPS is the primary system used at 13 NWS River Forecast Centers (RFCs) covering the US RFCs are federal entities tasked to provide fresh water-related information for the protection of lives and properties Slide 4

6 CHPS Providers NWS HQ has employees spread across two groups responsible for the support, maintenance, enhancement and deployment of CHPS (OHD) Jon Roe (Branch Chief) Neftali Cajina (Project Leader) Mark Fresch (Project Leader) Russ Erb (Content Manager) Hank Herr (Senior Developer) Cham Pham (Senior Developer) Raymond Chui (Support Focal Point) Xuning Tan (Junior Developer) Gautam Sood (Build and Configuration Leader) Hydrologic Services Division (HSD) Mark Glaudemans (Branch Chief) Randy Rieman (Technical Support) Bradley Mccuune (Technical Support) Only Federal Employees shown, NWS HQ has a team of supporting contractors also Slide 5

7 CHPS - Providers Roles A CHPS Software Development Organization Develop and maintain software designed to run within FEWS Manage CHPS Deployment, Includes updates to FEWS and the three other packages adapted to it Monitor the progress of the other development organizations and coordinate beta testing Slide 6

8 Developing Software Against FEWS Excluding some reservoir and channel routing (covered by HEC-RESSIM or HEC-RAS), OHD-CORE-CHPS consists of River Forecasting functionality used at RFCs not included in FEWS, but instead adapted to FEWS Type of Functionality Physical Process Hydrologic Models Flash Flood Guidance Algorithms Graphical User Interface for Data and Product Preparation Automated Data Retrieving Customized Data Analysis Technology Used to Adapt to FEWS FEWS General Adapter FEWS General Adapter + FEWS Transformations FEWS PI service and GUI plugins FEWS PI service FEWS API + FEWS Transformations A complete list is available here: Slide 7

9 Extending FEWS OHD-CORE-CHPS makes use of the many approaches FEWS provides to extend it with native functionality Java Adapter calls C/Fortran executable Java Adapter calls Java Program JAVAGUI plugin uses pi-service FEWS pi-service no adapter JAVA CUSTOM FEWS API executable call - no adapter Slide 8

10 Extending FEWS Slide 9

11 Extending FEWS - Challenges Challenges for OHD related to extending FEWS include: Change in functionality Even when it s the right thing (i.e. a bug was fixed) Changes in shared code Missing modules (e.g. deprecation) Change in module signature Undocumented or Incompletely Documented Features Schema Helps Changes in agreed upon data interface Missing elements This is not usually a problem Netcdf changes Pi-XML changes Unknown changes These are not always bad, but we don t learn about enhancements until later PI-version changes (e.g. forecastdate ) Performance Our users were spoiled by Fortran/C programs using custom formatted database files Maintenance We often hear about the number of configuration files to maintain There s Recent Improvements Slide 10

12 Extending FEWS -Strategies To address challenges, OHD uses: Regression Testing Quantitative Qualitative Performance Test-Driven Development Bi-Weekly Meetings with Deltares, USA Code Re-use and Code Sharing A majority of OHD functionality reuses a single FEWS Adapter For example, code to read pi-xml files is common and shared Common CHPS software development practices Slide 11

13 Strategies - Regression Testing Static Data Sets (run during nightly builds and during integration testing) For Quantitative Comparisons Actual - <event date=" " flag="0" time="18:00:00" value=".440"/> Benchmark - <event date=" " flag="0" time="18:00:00" value=".440"/> Performance Testing Workflow Completed in 15 minutes 30 seconds GUI Robot The Robot program is written in Java. It uses several technologies, including 3rd party Java program Sikuli( uses other GNU open source computer vision C programs (tesseract-ocr, opencv and other programs) Slide 12

14 Releasing Updates to FEWS - Challenges Challenges for OHD Related to Upgrading FEWS during a CHPS Release, include: Shaking out bugs Software and bugs go hand-in-hand, but change in functionality is especially noticeable when we skip a FEWS build Required Configuration Changes Our users often complain about maintaining configuration files. In general we ve heard software changes are better than required configuration changes Dependency To FEWS Our schedule is dictated by FEWS releases, but in general this has not been a problem Coordinating with the number of partners involved in a CHPS release is not always easy Deltares is extremely flexible here Slide 13

15 Releasing Updates to FEWS - Strategies To address these challenges, during a CHPS release OHD uses Repeatable release process using fixed milestones Methodical beta testing aimed at minimizing the number of moving parts FEWS only for several weeks Non-FEWS packages later for a few weeks OHD assumes the responsibility to ensure all 13 RFC setups are considered during testing Download latest configuration for SA testing In-Field beta testing using a subset of RFCs An HSEB focal point coordinates communication during beta across all participants Regular teleconferences during Beta Testing, using tools useful for remote collaboration (Google Docs, list serve) to: Tracking issues, including progress on fixes and verification of fix Test Spreadsheets, with enumerated tests covering most of CHPS functionality Mutually agreed on approach ( Free fix) to give Deltares an incentive to ensure features that used to work still work new Slide 14

16 Conclusion By all accounts CHPS has been a success! Because of our switch to FEWS, the research to operations path is better understood and based on more modern technology As an example, a major upgrade to CHPS is planned for Early 2015, Hydrologic Ensemble Forecasting Service (HEFS) With greater flexibility comes other challenges External Dependencies Performance Newness OHD has implemented some techniques to address these challenges Some ideas on a combined solution to these challenges are: Developing a North American CHPS/FEWS community for the purpose of sharing ideas, workarounds, tips Deltares posting of alpha software, for the purpose of gathering earlier feedback Investigate ways to improve/quicken the time for posting updates to FEWS documentation Others? Slide 15

17 Questions Lee Cajina, Slide 16

22 EXTRA SLIDES Slide 21

23 CHPS Providers Hydrologic Services Division Front line CHPS support for 13 River Forecast Centers Hardware and Software Work with CHPS Development Organizations (Deltares USA, HSEB, RMA) for requesting additional support Manage the Web-based Issue Tracking System - FogBugz Slide 22

Open Software Development for CHPS: Towards a software API

Open Software Development for CHPS: Towards a software Chris Brunner, CHPS Project Manager & Andy Rost, Director NOHRSC Office of Hydrologic Development (OHD) National Weather Service (NWS) FEWS User Days