Hd Hydrologic Information System

Transcription

1 Sharing Hydrologic Data with the CUAHSI Hd Hydrologic Information System David Tarboton, David Maidment, Ilya Zaslavsky, Dan Ames, Jon Goodall, Jeffery Horsburgh, Tim Whiteaker, Michael Piasecki, David Valentine, Tom Whitenack, Kim Schreuders CUAHSI HIS Sharing hydrologic data Support EAR

2 Consortium of Universities for the Advancement of Hd Hydrologic Si Science, Inc. 110 US University members 6 affiliate members 12 International affiliate members (as of March 2009) An organization representing more than one hundred United States universities, receives support from the National Science Foundation to develop infrastructure and services for the advancement of hydrologic science and education in the U.S.

3 Goal A system that enhances access to hd hydrologic data for education and research to better understand hydrologic processes. How can we better organize hydrologic data to enhance the analysis it can support? How can we better provide access to the data sources, tools and models dlthat enable the synthesis, visualization and evaluation of the behavior of hydrologic systems? Water quantity Water quality Precipitation Meteorology

4 CUAHSI HIS The CUAHSI Hydrologic Information System (HIS) is an internet t based system to support the sharing of hydrologic data. It is comprised of hydrologic databases and servers connected through web services as well as software for data publication, discovery and access. HIS Central Data Discovery and Integration platform Like search portals Google, Yahoo, Bing HydroServer Data Publication platform Like web servers Data Services Water Data Services Spatial Data Services HydroDesktop Data Synthesis and Research platform Like HTML Like browsers

5 Query, Visualize, and Edit data using ODM Tools CUAHSI Water Data Services System Platform independent Analysis GIS Matlab Splus R IDL Java C++ VB Discovery and Access Hydro Desktop HydroExcel Base Station Computer(s) Streaming Data Loader Excel, text ODM Database WaterML Service Registry Hydrotagger GetSites GetSiteInfo GetVariableInfo GetValues Harvester HIS Central Telemetry Network ODM Data Loader USGS NWIS WaterOneFlow Web Service Water Metadata Catalog Sensors EPA STORET NCDC Others

6 Water Data Streamflow Groundwater levels Precipitation & Climate ODM Soil moisture data Water Quality Flux tower data 6

7 Why an Observations Data Model dl Provides a common persistence model for observations data Syntactic heterogeneity (File types and formats) Semantic heterogeneity Language for observation attributes (structural) Language to encode observation attribute values (contextual) Publishing and sharing research data Metadata to facilitate unambiguous interpretation Enhance analysis capability 7

8 Scope Focus on Hydrologic Observations made at a point Exclude Remote sensing or grid data. These are part of a digital watershed but not suitable for an atomic database model and individual value queries Primarily store raw observations and simple derived information to get data into its most usable form. Limit inclusion of extensively synthesized information and model outputs at this stage.

9 What are the basic attributes to be associated with each single data value and how can these best be organized? Value DateTime Variable Location Units Interval (support) Accuracy Offset OffsetType/ Reference Point Source/Organization Censoring Data Qualifying Comments Method Quality Control Level Sample Medium Value Type Data Type

10 Point Observations Information Model Utah State Data Source Univ Little Bear River Network Little Bear River at Mendon Rd Sites Dissolved Oxygen Variables 9.78 mg/l, 1 October 2007, 5PM Values {Value, Time, Metadata} A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time Metadata provide additional information about the value

11 Observations Data Model (ODM) A relational database at the single observation level l (atomic model) When Time, T Stores observation data made at points t Metadata for unambiguous interpretation Traceable heritage from raw measurements to usable s V information i Standard format for data What sharing Variables, V Cross dimension retrieval and analysis A data value v i (s,t) Where Space, S 11

12 Data Storage Relational Database Values Value Date Site Variable Sites Site Name Latitude Longitude Value Name Date Latitude Site Variable Longitude 4.5 Cane3/3/2007 Creek Streamflow Cane3/4/2007 Creek Streamflow Town3/3/2007 Lake Temperature Town3/4/2007 Lake Temperature Site Name Latitude Longitude 1 Cane Creek Town Lake Simple Intro to What Is a Relational Database

13 Why Use a RDBMS Mature and stable technology Structured Query Language (SQL) Sharing of data among multiple li l applications i Data integrity and security Access by multiple users at the same time Tools for backup and recovery Reduced application development time

14 Observations Data Model Sites Values and Time Variables Horsburgh, J. S., D. G. Tarboton, D. R. Maidment and I. Zaslavsky, (2008), "A Relational Model for Environmental and Water Resources Data," Water Resour. Res., 44: W05406, doi: /2007wr

15 The Data Cube what where when When Time, T t 3/4/2007 / A data value D s Where What V i Streamflow Cane Creek Space, S Variables, V

16 Data Values Table Time, T t v i (s,t) s V i Space, S Variables, V

17 Simplified ODM Structure Sites Table SiteID SiteCode SiteName Latitude Longitude LatLongID 1 AcmeP1 Backyard Pond AcmePR2 Mill River gage Station Spatial References Table SpatialReferenceID SRSID SRSName 0 Unknown NAD NAD83 Units Table UnitsID UnitsName UnitsAbbreviation 12 parts per million ppm 23 cubic feet per second cfs

18 Discharge, Stage, Concentration and Daily Average Example

19 Water Chemistry from a Profile in a Lake

20 Stage and Discharge Example

21 Daily Average Discharge Example Daily Average Discharge Derived from 15 Minute Discharge Data

22 Stage and Streamflow Example

23 HydroServer A Platform for Publishing Space Time Hydrologic Datasets Map Server Time Series Analyst HydroServer Website HydroServer Capabilities Web Service HydroServer Database Configuration Tool WaterOneFlow Services HydroServer Database Spatial Services WaterOneFlow WaterOneFlow WaterOneFlow ODM ODM ODM ODM Databases and WaterOneFlow Web Services ArcGIS Server Spatial Data Services

24 HydroServer Goals A platform pato for publishing space time ehydrologic og datasets that: Provides local control of data Makes data universally available Is autonomous HydroServer should be functional independent of the rest of HIS Texas Hydrologic Information System INRAConstellation of Experimental Watersheds

25 Ongoing Data Collection Point Observations Data Internet Applications Historical Data Files ODM Database GIS Data GetSites GetSiteInfo GetVariableInfo GetValues WaterML WaterOneFlow Web Service HydroServer Dt Data presentation, tti visualization, and analysis through Internet enabled applications

26 HydroServer Regions A region is an experimental epe etawatershed, ed, research site, etc. for which data are published Little Bear River Experimental Watershed Reynolds Creek Experimental Watershed T.W. Daniels Experimental Forest Organizing concept for HydroServer One or more ODM dtb databases/wateroneflow /Wt services for a region One or more spatial data services for a region

27 Publication of Point Observations Observations Data Model (ODM) ODM Tools ODM Data Loader ODM Streaming Data Loader ODM Controlled Vocabularies WaterOneFlow web services Data are transmitted in WaterML format

28 Managing Data Within ODM ODM Tools Query and export export data series and metadata Visualize plot and summarize data series Edit delete, modify, adjust, interpolate, average, etc.

29 Loading data into ODM Interactive ODM Data Loader Loads data from spreadsheets and ODM Data Loader comma separated tables in simple format Streaming Data Loader (SDL) SDL Loads data from datalogger files on a prescribed schedule Interactive configuration SQL Server Integration Services (SSIS) Microsoft application accompanying SQL Server useful for programming complex loading or data management functions SSIS

30 Publication of Spatial (GIS) Datasets Publishing spatial datasets using ArcGIS Server Using OGCstandards that can be consumed by a number of GIS clients WMS, WFS, WCS

31 Data Presentation Via a Map Interface Internet Map Server built using ArcGIS Web browser client Combine spatial data and observational data Launch data visualization tools Based on a Region

32 Data Preview, Visualization, and Analysis Time Series Analyst Web Browser Client Cet Multiple ODM Database Support Variety of plot types Descriptive statistics Linked to the map application Data preview and download

33 HydroServer Website HydroServer e home page Observational data services GIS data services Online map Time Series Analyst Data Query and Download

34 HydroServer Capabilities Database Hd HydroServer Capabilities Dtb Database Configuration Tool Simple Windows application for creating metadata records in your database Associating Regions with services and maps

35 HydroServer Capabilities Web Service Publish capabilities of each HydroServer Published observational data services Published spatial data services Information transmitted in XML format Supports automatic cataloging of available services at HIS Central Makes HydroServers self describing

36 ICEWATER A Regional HIS ICEWATER INRA Constellation of Experimental p WATERsheds Coalition of 8 universities Point Observations Stream gages Water quality sampling Weather stations Soil moisture Snow monitoring Groundwater level/quality Spatially Distributed Data Land use/cover Terrain Hydrography AK WA OR CA I D NV UT AZ MT W Y CO NM

37 How do I Create a HydroServer? 1. Get a Windows Server Machine with IIS and.net Framework 2. Install Microsoft SQL Server 3. InstallHydroServer Software from his.cuahsi.org ODM and Utilities WaterOneFlow web services HydroServerWebsite Website, TSA, Map, Capabilities 4. Install ArcGIS 5. Create Services and Document them in Capabilities database 6. Register Services With HIS Central and register variables with the Ontology

38 HydroServer Future Considerations Security and Data Access Control More generic data model Flexibility in attributes Moving platforms Additional data types Tighter integration with Hydrologic Ontology Enhanced spatial data dt sharing What features and functionality does HydroServer need What features and functionality does HydroServer need to better serve the community?

39 Conclusions HydroServer provides a self contained autonomous data publication system Local control of data Downloadable user (data publisher) configurable software stack that contains: ODM and associated tools WaterOneFlow web services Geographic data sharing using WFS, WCS, WMS from ArcGIS server Time Series Analyst ArcGIS server based web map application HydroServer Capabilities web service that publishes metadata about regions and services (observational and spatial)

40 Demonstrations ICEWATER: ICEWATER (local): Little Bear River: tt e ea e