Getting started with Hydra Modeller

Transcription

1 Getting started with Hydra Modeller Download Installation Hydra Modeller Basics Logging in Installing a template Building a network manually Importing a pre-defined network Setting GIS Layers Managing Data Scenarios Adding a new scenario by CSV Viewing Data Editing Data Installing an App Example 1: The Water Allocation Demo model Example 2: California sequential modelling Download Hydra Modeller is available at Installation Double-click on hydra.exe to open the InstallShield Wizard for Hydra

2 Click the I accept the terms of the license agreement and hit Next Choose Complete and hit Next Click Install. This will bring up a progress bar, which will show hydra installing on your computer.

3 And you re done! Click on Launch Hydra and hit Finish Hydra Modeller Basics Hydra Modeller is the user interface for the Hydra Platform data management system. Hydra Platform is an open source web server which can be hosted on a user s local machine or on a remote server. Hydra Modeller can connect to multiple different Hydra Platforms and for each platform the user must have a username and password. By default, the Hydra Modeller setup package installs Hydra Platform a user s local machine. When Hydra Modeller starts up, it will start the Hydra Platform server automatically and when the user closes the software, the server will be shut down routinely also.

4 Logging in When opening Hydra Modeller, you will be presented with a login screen. By default the server is set to the user s local machine (not accessible by anyone else) but by simply changing the address and port, a user can connect to one of many Hydra Platform systems running in the cloud. If you don t know which server or port to connect to, simply leave them untouched and type root to the username slot and leave password empty. Then click OK. Once logged in, you will see this screen: Notice that the icons across the top are not active. To activate them and start building networks, you can now start a new project.

5 To create a new project, click on the Project tab on the top-left. This will bring you to the New project page. Enter the name of your project and an optional description, as below. Then click Save. This will bring you back to the main page. This time you will notice that the icons across the top are now active.

6 Installing a template The next step before building a network is to install a template. A template defines the types of nodes and links you will be using, and is normally associated with a model (that could be a simulation or optimisation calculation routine). Each node and link definition contains information about associated attributes including dimension, unit, restrictions that guide validation routines and other information that defines an attribute. The template also defines visual features of the nodes and links such as the images or shapes used to represent them in Hydra Modeller. This example uses the Water Allocation Demo template, which accompanies the Water Allocation Demo gams model (for more information on the demo models, see the demo model starter pack document). To install a template click the Install App/Template button...

7 ...and navigate to the template zip file and click Open... If successful, you will see the message Successful Import. Click OK.

8 Now that a template is installed, you can start building a network. Click on Add Network and fill in the details as below. Choose Water Allocation Demo from the Associated Applications drop down menu: Now right click on the network name and click on set active.

9 Now notice that a new network appears in the left-hand menu and a new tab appeared in the top, called Water Allocation Demo. In that tab are the nodes and links which you can add to the network: Building a network manually To build a network from scratch, click on the node type you want to add. The cursor will then change to the node s symbol. Next click where on the map you want to add the node and when prompted, give a name and optional description to the node. (Note: Node name must be unique within a network.), like shown:

10 After adding a few nodes, notice that they appear in the Nodes box on the right-hand menu: Next, add some links. To do this, choose river section. Then click on the starting node and double-click on the end node to save the link. This link will now appear in the right-hand menu, under the Links tab. Your resulting network should look something like this:

11 Importing a pre-defined network Building a network from scratch can be time consuming, so Hydra Modeller provides a way to import a network from an external source. In this case, a set of CSV files, made especially for the water allocation tutorial. First, get the water allocation demo data from the hydra app store: To do this, visit and register with a new account. When logged in, search for Water Allocation and select Non-Shortage (Baseline) Data for Water Allocation Model. Download the file as a zip, but there s no need to unzip it. Now, back in Hydra Modeller, click on Import -> from CSV as shown

12 An input box will pop up. Click on Import from zip file and navigate to the file you just downloaded. This will automatically fill in the network name and the Network File entry. Next, select Water Allocation Demo from the Template file input box and hit Run. At this point a new window will open, telling you the status of the import. At the end, it should look like this. Press Exit. Now the imported network will be available on the left hand pane.

13 This network is not yet active (you can t edit it). To activate it, right-click on the network and click Set Active :

14 The imported Water Allocation Network is now active. Setting GIS Layers Often the points in a network represent real entities, so GIS layers can be used to add some context to the network. A GIS layer in Hydra is effectively a background image, with the network sitting on top. To import GIS data, click the Add layer button and navigate to the openstreet.wms layer, watercourses and Hydronodes layers that you downloaded from the Hydra App Store. This GIS data can be found by searching for Water Allocation in the app store and locating the GIS data. Download and unzip the GIS data.

15 Back in Hydra Modeller, load the GIS layers on the bottom left pane with the Add layers button: and navigate to the GIS data.

16 Add the HydroNodes.shp, Watercourses and openstreet.shp layers, and you will see something like this:

17 Managing Data Scenarios The scenario manager allows users to view and edit all the data related to the currently active scenario. A scenario is a container for data associated with the network. For example, in the water allocation model, the attribute initial_storage on all reservoirs in the network can be decreased in a new scenario to represent a dry winter. First let s look at how data in a given scenario can be accessed and edited. Editing data in the main UI The simplest and most convenient way to edit data in the current scenario is by editing the values directly from the attribute window. The attribute window is located on the bottom-right of the main UI. For scalar values, the value can be edited by typing in the input box:

18 Time series can be edited by clicking on the edit icon, on the right-hand side of the input box. For example, for inflow attribute of a reservoir: Scenario Manager For more sophisticated data editing, comparison of attributes within and between scenarios, use the scenario manager. Access to the scenario manager is via the Scenario Manager button in the Home tab:

19 This will open a new window, similar to this. Notice the list of attributes on the left. This is a list of all the node, link, group and network attributes for this network: To retrieve the data, select the attribute (or attributes) by checking the check box and then hitting Extract data in the new Scenario Manager tab. In this example, demand is chosen and after clicking on Extract data the summary tab lists all the network resources that use the attribute demand.

20 Editing Data To edit time series data, either click on the column header to view the demand values for all resources or click on View time series data in one cell to view the demand values for one resource. Once in the time series tab, to edit data, simply change the value in a cell. In this example, the demand values of node Ag1 have been increased. Changes are reflected by the column header colour turning orange and the attribute tab colour turning yellow. To save this change, select for this scenario from the save panel on the top menu. The other option will save the value for any other scenario that uses the same demand time series. A dataset would be shared between scenarios if each scenario was cloned from the same original scenario. This save option can help correct value that s wrong in multiple scenarios without having to go to each scenario in turn to fix it. Note: you can check if scenarios share a dataset by using the Dataset details window or viewing the information for more than one scenario at a time in Scenario Manager.

21 This will result in a message saying Changes have been saved and the orange box turning white again, like so: Cloning a Scenario A network can contain multiple scenarios. For the water allocation example, the default is the Baseline scenario. Scenarios can be seen as sub-elements of networks in the Networks tree view. Creating a new scenario is done in two ways: by cloning an existing scenario or by importing new data via csv. To clone a scenario, right-click on a scenario (making sure the scenario is active) and click Clone:

22 This will result in a second scenario appearing in the network menu, called Baseline (clone) : After right-clicking on Baseline (clone) and selecting Set active, click on the Scenario Manager button on the top ribbon if you want to edit scenario sets of information for the cloned scenario. Adding a new scenario by CSV The second way of adding a new scenario is to add data from an external source, such as a set of CSV files, as described earlier in this quick start guide. In the water allocation model, there are two sets of csv files -- baseline and shortage. These csv files are almost identical, except that the values of some attributes are changed to represent a shortage of water. First, search for and download the Shortage data for the water allocation model on the hydra app store. Next, click on Import -> From CSV

23 Click on import from zip file and navigate to the new dataset: Call the new scenario Water Shortage. Ensure that the network Water Allocation Network is chosen, and so is the template: This tells Hydra to import data as a second scenario in the same network, rather than building a whole new network. Press Run and wait for the import to complete. A new scenario will now be added to the network list. To activate the new scenario, right-click on it and hit Set Active. Installing an App In this example, we will install an app that exports a scenario set of data ready for optimisation modelling with the Generic Algebraic Modelling System (GAMS) because the model that completes the water allocation calculation for this network is written in GAMS. GAMS is a generic mathematical programming language commonly used in modelling.

24 In this example, we show how to install the GAMS Connector App. This app is designed to export a hydra network to a GAMS input file, then to run a GAMS model against this file and finally to extract the results of this model and return them to Hydra for a user to view. This saves a user having to deal directly with GAMS input files, which can be time-consuming and error-prone. Navigate to log in or register and then search for GAMS. You will see several results, but notice their categories: Modelling System, Model and Connector App. The modelling system is the GAMS software itself (required to run the gams connector app). A model is the GAMS model we will run and the connector app is the app which Hydra Modeller will install and run.

25 The connector app will be able to connect to any of the Models. Click on GAMS Connector App and click download. The price for the gams app is currently fictitious, so can be ignored. This will download a zip file. Note the location of the download. Now, back in Hydra Modeller, open the water allocation project and click Install App/Template. Navigate to the downloaded zip file and hit OK.

26 You will now see three new buttons in the Apps tab in the main window. GAMS Automatic, GAMS Export and GAMS Import. Clicking on GAMS Automatic, you will be presented with a set of inputs similar to those seen when importing the CSV files. To see what you need to enter for these, see the Water Allocation Demo Tutorial.

27 Example 1: The Water Allocation Demo Model Stephen Knox, University of Manchester, December 2015 The Water Allocation Model was developed by the research team of Prof. Julien Harou and is designed to distribute water from a variety of supplies (River reservoir supplied from river abstraction, a desalination plant and a groundwater borehole) to three types of demands (urban, agricultural and discharge which can be considered an environmental demand in terms of minimum flow required to be delivered to a river). The water allocation model is written in Generic Algebraic Modelling System (GAMS) 1. It s a generic simple application to explore water resource planning issues as part of a university level tutorial. In summary Water is transferred along conduits (which could be either rivers of pipes which have maximum and minimum flows as well as a flow_multiplier value (0-1) to represent the potential for losses (e.g. leakage and evaporation). If the flow_multiplier is less than one some water is lost. A priority value is assigned to each demand node (urban, agricultural and discharge) to represent the importance of water being provided to each demand. If all priorities equalled the demand for each node, then each demand node would be considered to have equal priority of delivery. When drought occurs, demand node(s) with the highest priority will face the least water shortage. There are water treatment works that affect the flow based on their min and max storage and their percent_loss (i.e. a percent_loss of 5% means that 5% of the water is lost to process needs). A hydropower node is included to capture how much water could be available for power generation and a simple calculation is 1

28 included to calculate revenue (dependent mostly on the net head at the site location and the flow of water). You can review the model formulation in the allocation.gms file. Water Allocation Model in Hydra In order to explore the model with Hydra, a user must have the latest version of GAMS installed, must have the model itself and its data in CSV or Excel format (for importing into Hydra) and must have the Hydra GAMS Connector App, which transfers the input data stored in Hydra to GAMS and returns results into Hydra. For instructions on how to find the water allocation model on the Hydra App Store, see the Getting Started with Hydra Modeller document. Inspecting the data Having loaded the data, ensure the Baseline scenario is active. Notice how the orange node is located over a city, and when selected via the arrow button on the interface, changes colour and is highlighted in the right-hand menu. Its attributes: priority and demand are available to view also, on the bottom -right. In turn, click on each supply node Click on the inflow attribute, followed by the timeseries button (top menu) and the time series button for the inflow attribute in the attribute viewer. Do the same for each demand node, selecting demand.

29 This will give you a comparison of inflow versus demand for each node. This is a simple 9 month model with monthly time steps and the initial boundary conditions include an initial_storage for all of the supply sources. Your time series plot will look similar to the image below. Finally, notice on the bottom of the Attributes section for each node, there is a Results section. Notice that these are inactive -- there is no result data available. This means the model has not been run. So let s do that now. Running the Model The purpose of this model is to allocate water based on priority such that if there is a shortage of water, those demands with lesser priority will receive a smaller percentage of their demand. Priorities are assigned to each demand node dictating which node must receive water first. In order to run this model, you will need to: Go to the Hydra App Store to download the model and the GAMS software (USE THE 32-bit VERSION). Go to You don t need to sign in to download free Apps, only to submit reviews, become a developer and purchase commercial Apps.

30 In the search bar near the top, type in Water Allocation. You will see a collection of Apps relate to the model, one of which is the model itself: Download the model. In the search bar near the top, type in GAMS. You will see a collection of Apps relating to GAMS, the first of which is the modelling system itself. If you do not have GAMS already installed, click on the Generic Algebraic Modelling System app, which will take you to the GAMS website, then download and install it. Make sure you are installing the 32 bit version of GAMS and that its version is greater than 24! You are now ready to run the model. Time Steps The GAMS app needs to be told what time-steps to run for, so it can retrieve the correct data from Hydra. This can be specified manually when running the app or, more easily, set as a property on the scenario so it s automatically filled in every time you run the model.

31 Before sending a set of scenario data to GAMS for a model run, right-click on the scenario, select properties and make sure the start time is January 1st 2020, end time to September 1st 2020 and the time step is 1 month. GAMS models compatible with this app needs the time-step to be manually specified. The 1 month format is important and must follow this format, as it indicates to the app what the time-step for the model should be. Now select the Apps tab and click on the GAMS Automatic run. The start time, end time and timestep should be pre-populated based on the scenario properties. All that s left to do is select the Network, Scenario and then locate the GAMS model you downloaded previously. The GAMS model is the allocation.gms file Once the Mandatory fields and the scenario time information has been entered, Click on Run to run the model.

32 Hydra Modeller will report on the progress of the model run. After the model is complete and results have been returned from GAMS to Hydra, Click on the Ag1 and Urb 2 demand nodes and select demand_met and demand, followed by Time series -> general on the top menu to see if there was enough water to satisfy all demands. Because this sample data set has enough water supply, all demands have been met. The demand and demand_met are equal for two separate demand nodes, demonstrating that for this scenario the demand has been met for these nodes:

33 Run a variation of the scenario. There are two approaches to creating a new scenario -- cloning an existing scenario or uploading different scenario data. This example explains the latter. First, search for and download the Shortage data for the water allocation model on the hydra app store. Next, click on Import -> From CSV Click on import from zip file and navigate to the new dataset: Call the new scenario Water Shortage. Ensure that the network Water Allocation Network is chosen, and so is the template: Press Run and wait for the import to complete. Ensure the Water Shortage scenario is active.

34 Now re-run the GAMS model, except this time choosing the Water Shortage scenario: Once complete, we can compare the results of the two scenarios. As before, select the demand and demand_met for both the agricultural nodes, pressing Time Series - > General each time.

35 The graph should look like this: Note how the demand_met for Ag2 on the 1st of May is 5.5 whereas the demand is 30. AG1 is unaffected by the shortage. Several approaches can be taken to deal with this shortage. We will implement a demand reduction, where we reduce the demand for water by AG2. In the home tab, click on Scenario Manager Click on the check-box demand on the left, followed by Extract Data at the top (see getting started if you are unsure). Now click on View Time Series beside AG2

36 Now reduce the demand values like so (Note how the top goes orange). This means that values have changed but are unsaved. You must save the changes using the save changes button at the top. Now re-run the model and inspect the demand vs demand met for AG2. Note how the demand is now met for ag2 at each time step. The demand reduction policy worked!

37 Example 2: California sequential modelling In the installation package there is a demonstration project preloaded in Hydra Modeller to illustrate how to map attributes for sequential modelling based on a very simplified version of two California networks. To try Attribute mapping, follow these steps: Open the project titled, California sequential modelling click on the schematic view and set the CalLite network active Filter the Links for the DSM2 transfer Type. There is only one representative attribute currently stored for the CalLite (outflow) and DSM2 (inflow) models. Each model has these attributes for a handful of Links (in terms of CalLite) and a handful of Nodes (in terms of DSM2) as these are the locations where outputs of the CalLite statewide model inform the functioning of the DSM2 San Francisco Bay model. Select the AD_Calaveras link in the CalLite network, right-click on the Outflow attribute and select Map attribute.

38 Map this attribute to the Inflow attribute of the Calaveras Eastside to Delta node of the DSM2 model. Now there is an arranged and recallable transfer relationship from the CalLite Resource-Attribute combination to the DSM2 Resource-Attribute combination. This relationship can be called upon to transfer data from one scenario in the source network to another scenario in the target network.

39 Right-click on one of the Networks in the Network Tree and select View Attribute Mapping (don t update yet). You will see a number of other mappings are already saved. Before choosing to Update, close the Attribute Mapping window and create a Climate Change scenario for DSM2 by cloning the DSM2 Existing scenario and renaming it.

40 Before updating, compare the difference of the Inflow attribute for any of the DSM2 nodes of the CalLite connection Type. You can do this by clicking on the attribute and choosing the Difference option of the Time series button (found in the Toolbox menu). The difference will be zero. Flow comparison before Attribute mapping update

41 Flow comparison after the Attribute mapping update At this point when the App is fully developed, a user would run the CalLite model to generate a Climate Change version of the dataset, by clicking on the CalLite App button in the Apps ribbon. Clicking on this button now will initiate a CalLite model run after selecting the correct batch file (if you have CalLite loaded it is the CalLite_w2.bat, but the set-up configuration is not yet arranged by Hydra Modeller). For now, some results from a CalLite climate change scenario are pre-loaded into Hydra. Then a user would update input required for DSM2 from the results of a CalLite run using the Attribute Mapping utility. These steps can also be automated in an App that increases the speed and efficiency of updating a target network based on source network data.