NOTICE. This document is only one section of a larger document. All sections together collectively form the NNEW Documentation.

Transcription

1 NOTICE This document is only one section of a larger document. All sections together collectively form the NNEW Documentation. Please be advised that: This section may need to be interpreted in the context of the other sections and may refer to content outside of itself. Editors of this section should refrain from using custom formatting which might complicate assembly of all sections into one document. (Editors are also encouraged to keep the HTML version of this document updated on the Wiki.) Please consult the Table of Contents on the NNEW Wiki for an overview of, and access to, all sections. 1

2 NextGen Network Enabled Weather (NNEW) Phase II Implementation Verification Procedures Version 0.2 5/23/2008 National Center for Atmospheric Research MIT Lincoln Laboratory National Oceanic and Atmospheric Administration/Global Systems Division 2

3 DOCUMENT REVISION REGISTER Version Date Content Changes Editors Contributors 0.1 4/24/2008 Initial Template Arnaud Dumont 0.2 5/22/2008 First Draft, needs input from other labs Arnaud Dumont Oliver Newell Minna Win Please direct comments or questions to the editors of the sections listed above. For overall comments or questions, contact: Arnaud Dumont National Center for Atmospheric Research Research Applications Laboratory 3450 Mitchell Lane Boulder, CO (303)

4 TABLE OF CONTENTS 1. OVERVIEW SYSTEM REQUIREMENTS DEMONSTRATION APPLICATIONS IMPLEMENTATION VERIFICATION Gridded Data Access Verification with NNEW Phase II Integrated Java Application Verification of THREDDS with a Web Browser and ncview Cross-Section Extensions Service Request Standards Verification with NNEW Phase II Integrated Java Application Verification of NCAR WCS Server with a Web Browser Data Encoding Standards Verification with ncview and ncdump Catalog/Registry Discovery of Web Coverage Service Interface Information Discovery of Single Dataset by Weather Phenomenon Type Discovery of all datasets in the Single Authoritative Source (SAS) Domain Addition of New Dataset to the Weather Cube as the Single Authoritative Source for Lightning Data Non-Gridded Data Access Verification of GeoServer with NNEW Phase II Integrated Java Application Verification of GeoServer with a Web Browser Verification of GeoServer with the NOAA/GSD FXC Client APPENDIXES Acronyms and Abbreviations

5 5.2 References

6 1. OVERVIEW This document provides procedures for verification of the implementation of NNEW Phase II. The capabilities to be verified were developed during the federal fiscal year 2008 (FY 08). In many cases, however, these capabilities are extensions to the capabilities developed in NNEW Phase I (FY 07). Specific mention will be made when appropriate. Phase II of NNEW research focused primarily on 3 areas: prototyping cross-section extensions to existing standards and dataservers, implementing a distributed data registry and refining metadata semantics, and researching support for feature data in existing data servers and standards. The three areas are addressed in separate sections below. 6

7 2. SYSTEM REQUIREMENTS The verification procedures outlined in this document may be executed on any computer connected to the internet, provided that it satisfies a minimum set of requirements. These requirements vary according to the demonstration application. Overall requirements which will enable verification of all implementations using all demonstration applications are: Memory The test computer should have a minimum of 256M of RAM. This memory is primarily needed for responsiveness when running the NNEW Phase II Integrated Java Application. Performance for visualization of some of the larger datasets is greatly improved for clients with 512M or 1G of RAM. Internet Connection The test computer must be able to initiate an outgoing TCP/IP connection to port 80 of a remote host. The test computer must also be able to receive data across the connection it initiates. This fundamental capability can be verified using any commercial off-the-shelf web browser. The test computer may be connected via a proxy server and/or have its data filtered through a web filter, as long as the intervening software does not corrupt digital signatures on data. Some web filters have been known to invalidate the certificate signatures on valid data when they attempt to inspect its contents. If you encounter corrupted certificate errors when running the tests, please contact your system administrator for assistance. Java Web Start The test computer must have the latest stable version of Java installed. This is currently Java Java may be downloaded and installed by pointing a web browser to the following URL: Installation requires administrator privileges on most platforms. The latest version of Java includes Java Web Start. Java Web Start is a client-side Java Virtual Machine environment that facilitates execution of code downloaded from the internet. Java Web Start provides many security protections not available when running applications directly. It also allows signed applications to request access to local printers, create and load files on the local file system, initiate outgoing TCP/IP connections, etc. The NNEW Phase II Integrated Java Application, Catalog/Registry Interface Application, and FXC Application all require Java Web Start 7

8 Ncview and Ncdump In order to decode and view raw NetCDF data files, the test computer must have a decoding or viewing application installed. A complete list of software for viewing and manipulating NetCDF data can be found on the Unidata website at the following URL: Two applications have been selected for verification use in this document: ncdump and ncview. Ncdump is a utility which comes bundled with the NetCDF library. The library can be downloaded from the following URL: Ncview is a simple application for viewing NetCDF files. It can be downloaded from the following URL: 8

9 3. DEMONSTRATION APPLICATIONS Several applications were developed to demonstrate the Phase II implementation. Applications are required because there is no way for humans to directly interact with a complex system such as NNEW; security requires authentication of clients, access services adhere to rigid protocols for information exchange, and returned datasets are typically encoded in compact binary formats. In addition, the 4D Weather Functional Requirements for NextGen Air Traffic Management specifically calls for Weather integrated directly into sophisticated decision-support capabilities. That requirement clearly pushes development focus towards machine-to-machine interoperability and away from human interfaces. As such, the NNEW team has chosen not to provide documentation on the interactions needed for users to test the implementation directly. Instead, a set of applications are provided that can be used to test the implementation. The demonstration applications range from simple, single-use, tools to complex, integrated, displays. In addition to verifying the implementation, these demonstrations serve to showcase the flexibility of the operational concept: many types of clients, interacting with the system using several different protocols, and requesting data at many different granularities. The demonstration applications are: NNEW Phase II Integrated Java Application This is an extension of the Phase I demonstration application. It is the most integrated tool of the NNEW Phase II demonstration. This tool demonstrates the use of the service registries, gridded data services, and feature services from all three NNEW laboratories. Catalog/Registry User Interface Application This tool connects to a registry and allows for interactive uploading, querying, and downloading of service information (WSDL s and associated schemas), dataset metadata, classification taxonomies, and other artifacts associated with geospatial data artifacts, such as coordinate reference system dictionaries. FXC Client This application is currently under development at NOAA/GSD. More information on this application, including downloading and installation instructions, will be provided as it becomes available. Ncview The ncview application is a simple viewer for NetCDF files. It is used in this verification as a way to confirm the values of weather variables in returned data files. Ncview was not developed as part of NNEW, but is freely available, as described in the System Requirements section. 9

10 Ncdump The ncdump application is a simple script for printing out the data structures of NetCDF files. It is used in this verification as a way to confirm the metadata of weather variables in returned data files (their name, dependent dimensions, units, etc). 10

11 4. IMPLEMENTATION VERIFICATION 4.1 GRIDDED DATA ACCESS Gridded data access services are well supported by three NNEW laboratories. This capability was included in the FY 08 NNEW demonstration and has not been significantly modified for this demonstration. Since this capability was not verified with a formal verification document last year, it is included in this document Verification with NNEW Phase II Integrated Java Application 1) Launch the application by pointing a web browser to the following URL: 2) Select the Default configuration to load in the Configuration Manager window. 3) Select each gridded dataset in turn, by selecting it from the Weather heading in the menu bar. As appropriate for each dataset, change the selected time and altitude. Compare rendered data with plots from an independent source. Descriptions of each dataset follow: Dataset Name Description Host Service Protocol Forecast 3D Temperature Rapid Update Cycle (RUC) model Exp ADDS NCAR/RAL WCS Server WCS 1.1 X X Relative Humidity Rapid Update Cycle (RUC) model Exp ADDS NCAR/RAL WCS Server WCS 1.1 X X Wind Speed Rapid Update Cycle (RUC) model Exp ADDS NCAR/RAL WCS Server WCS 1.1 X X Turbulence Graphical Turbulence Guidance (GTG) Exp ADDS NCAR/RAL WCS Server WCS 1.1 X X Icing Current/Forecast Icing Potential (CIP/FIP) Exp ADDS NCAR/RAL WCS Server WCS 1.1 X X Icing Severity Current/Forecast Icing Potential (CIP/FIP) Exp ADDS NCAR/RAL WCS Server WCS 1.1 X X 11

12 Surface Temperature MADIS Observations NOAA/GSD THREDDS Data Server WCS 1.0 Surface Dewpoint MADIS Observations NOAA/GSD THREDDS Data Server WCS 1.0 Surface Windspeed MADIS Observations NOAA/GSD THREDDS Data Server WCS 1.0 CIWS VIL CIWS Vertically Integrated Liquid MIT/LL MIT/LL WCS Server WCS 1.1 CIWS Echo Tops CIWS Cloud Tops MIT/LL MIT/LL WCS Server WCS Verification of THREDDS with a Web Browser and ncview 1) Open a web browser 2) In the location bar, enter the following URL to browse the catalog: 3) Browse to the desired dataset and click the WCS link under the Access heading 4) Edit the URL in the location bar, replacing GetCapabilities with GetCoverage and press return. A download window will appear, asking what to do with the file. Save it to the local disk. 5) Launch ncview 6) Select Load from the menu bar. 7) In the file chooser, select the data file you downloaded in step CROSS-SECTION EXTENSIONS One of the primary goals of NNEW research is refining data exchange standards to support trajectory based operations (TBO). An initial step towards that goal is the ability to request and receive vertical cross sections through the full 3-dimensional data volume along a client-specified horizontal trajectory at a given time. Phase I of NNEW simulated this feature by performing the cross section in the demonstration client. Phase II of NNEW has prototyped an extension to the WCS interface to support 12

13 cross section requests and an extension to the NetCDF Climate and Forecast (CF) convention to support cross section responses. Verification of both extensions is described below Service Request Standards The WCS protocol supports requesting data within a bounding box aligned with a coordinate reference system (CRS), but it does not currently support requesting data within a distance (Δxy, Δz, and/or radius) of a trajectory. As part of NNEW Phase II development, the CRS construct has been extended to be defined in terms of points along a trajectory. This has enabled the request bounding box to be defined relative to the trajectory CRS Verification with NNEW Phase II Integrated Java Application 1) Launch the application by pointing a web browser to the following URL: 2) Select the Default configuration to load in the Configuration Manager window. 3) Select the RUC Temperature forecast dataset by selecting Weather Temperature from the menu bar. 4) Begin specifying a flight path trajectory by selecting Tools Flightpath Click Way Points from the menu bar. 5) Left-click several points on the map to set intermediate way points. A line will be drawn between each waypoint. 6) Right-click anywhere on the map and select Show Crosssection from the dropdown menu to complete the path. Result: A new window should appear with the flight path cross section (example below) Verification of NCAR WCS Server with a Web Browser 1) Open a web browser 2) In the location bar, enter the following URL: 3) In the form, enter the coordinates of the way points and press the Submit button. 13

14 Result: The returned XML file should automatically render in you web browser. If your browser asks you what to do with the file, save it to your local file system and open it with a text editor. 4) In the downloaded XML file, find the URL within the <AbstractReferenceBase> elements. Copy this URL into your browser s location bar to download the data file. Result: The browser should ask what to do with the returned file. Save it to the local file system Data Encoding Standards The data which are returned from a WCS service may be requested as encoded in one of several standard formats. The data encoding which the NNEW team has been prototyping is NetCDF 3 with the CF convention Verification with ncview and ncdump 1) Follow the request steps listed in the Service Request Standards section ) Launch ncview 3) Select Load from the menu bar. 4) In the file chooser, select the data file you downloaded in section CATALOG/REGISTRY The NNEW registry is used to store high-level information about datasets and their associated services. Service interface information is used at build-time to generate client applications that conform to a given interface. At run time, a common usage model for the registry is to discover datasets using concepts such as air_temperature, and then discover the service(s) capable of serving those datasets. The ability to discover datasets classified as members of the NextGen Single Authoritative Source (SAS) for weather is also an important use case. The following steps cover these core build-time and run-time usage scenarios Discovery of Web Coverage Service Interface Information A Web Coverage Service is any service capable of retrieving data as ISO Coverages, which encompass gridded data but also other forms of coverages, such as vertical wind profiles or measurements of weather phenomenon along a trajectory. The OGC Web Coverage Service and JMBL are two examples of services with these capabilities. 14

15 1. Launch the registry user interface by pointing a web browser to the following URL: Select the query FindServiceInterfaces. Enter *WebCoverageService* for the service type parameter. Use a plain wildcard ( * ) for any other query parameters. Submit the search request by hitting the Search button. Service interface matches should be visible in the Search Results panel at the bottom of the screen. There should be 2 matches one for the OGC Web Coverage Service interface and one for the JMBL service interface. Double-click on the OGC Web Coverage Service in the Search Results panel to view the service interface details. Click on the WSDL reference to download the associated WSDL file Verify that the returned file is an OGC Web Coverage Service WSDL file by visual inspection using a text editor. Repeat steps 4 and 5 for the JMBL service interface description Discovery of Single Dataset by Weather Phenomenon Type For the sake of the demonstration, the registry has been pre-loaded with metadata for the majority of supported datasets. The exception is the lightning dataset, which has been omitted to demonstrate the addition of a new dataset to the weather cube Launch the registry user interface by pointing a Web browser to the following URL: Select the query Find_Data Set in the registry UI. Specify the pattern for the weather phenomenon name (e.g. *air_temperature* ) in the appropriate field. For other fields in the query, enter the wildcard character ( * ) Observe the query results and verify that metadata for a single dataset is returned Discovery of all datasets in the Single Authoritative Source (SAS) Domain Launch the registry user interface by pointing a web browser to the following URL: Select the query Find_By_Classification_Path. Specify the classification path used by for the Single Authoritative Source (SAS). The classification path for the SAS is: /classifications/wxcube/open-unrestricted/sas. Verify that registry objects for the <N> expected datasets are returned in response to the query, and also verify that metadata for a lightning dataset is not among them. This dataset will be dynamically added in the following section. 15

16 4.3.4 Addition of New Dataset to the Weather Cube as the Single Authoritative Source for Lightning Data Launch the registry user interface by pointing a web browser to the following URL: Using the File/Import menu in registry UI, browse to the demonstration directory containing lightning dataset metadata. This metadata file conforms to the ISO metadata standard, and in particular, the ISO XML Schema associated with that standard. Import the file to the registry. The registry recognizes files of this type, and during the upload extracts relevant metadata, making it searchable Using the registry UI, tag the lightning dataset with the SAS classification. Redo the steps in section 4.3.2, verifying that the lightning dataset is now part of the weather cube, and part of the SAS in particular. 4.4 NON-GRIDDED DATA ACCESS Non gridded data includes, but is not limited to, airport, reporting station, and navaid information, surface and aloft observations, aircraft trajectories, and airspace volume boundaries. Standards for encoding and dissemination of these data are not as mature as those for gridded data. Research into feature servers and protocols has uncovered fewer viable alternatives. For the NNEW Phase II demonstration, the team has prototyped services using GeoServer ( GeoServer is an open-source, fully functional WFS-T and WMS server that follows the OGC specifications. It is built on the GeoTools Java toolkit Verification of GeoServer with NNEW Phase II Integrated Java Application 1) Launch the application by pointing a web browser to the following URL: 2) Select the Default configuration to load in the Configuration Manager window. 3) Select each feature dataset in turn, by selecting it from the Overlays heading in the menu bar. Compare rendered data with plots from an independent source. Descriptions of each dataset follow: Dataset Name Description Host Service Protocol Data MADIS Obs Observations from METARs, buoys, etc NOAA/GSD NOAA/GSD GeoServer WFS 1.1 GML

17 METARs METARs decoded from NOAAPort Exp ADDS NCAR/RAL GeoServer WFS 1.1 GML 3.2 PIREPs PIREPs decoded from NOAAPort Exp ADDS NCAR/RAL GeoServer WFS 1.1 GML 3.2 Lightning Lightning observations from NADIN system MIT/LL MIT/LL WFS Server WFS 1.1 GML Verification of GeoServer with a Web Browser 1) Point a web browser to the following URL to determine what data sets are available: etype 2) Enter one of the URL s listed in the output from step 1 to obtain more details about a particular dataset. 3) Query for data using the following URL: pename=gsd:metar Note: The data can be subset through a filter query. Append the above URL with the appropriate syntax as outlined in the OGC Filter Encoding Specification and WFS specification Verification of GeoServer with the NOAA/GSD FXC Client A demo of GSD's FX-C client will demonstrate access to LL CIWS data that will be provided via the AWIPS server (located within GSD). This will demonstrate baseline performance characteristics of an operational capability required by the weather service. The CIWS data will be populated via a LDM feed from LL in real-time (as soon as it arrives). This type of access, will define a baseline for data access. We will also configure the FX-C client to access Lincoln's WCS server and download the data when requested by the user. We will compare performance of the OGC method to the AWIPS like environment. Both full resolution domains and subsets will be considered for access and performance. We will also look at a variety of accesses to point data that exercises our WFS - both in terms of flexible queries, and performance of data delivery. 17

18 5. APPENDIXES 5.1 ACRONYMS AND ABBREVIATIONS GML Geography Markup Language The Open Geospatial Consortium s XML standard for geographic feature data NextGen Next Generation Air Transportation System The JPDO program coordinating the Agencies contributions to data exchange NNEW NextGen Network Enabled Weather The FAA weather contribution to the NextGen effort WCS Web Coverage Service The Open Geospatial Consortium s standard for coverage data services WFS Web Feature Service The Open Geospatial Consortium s standard for feature data services 5.2 REFERENCES JPDO Weather Functional Requirements Team. Four Dimensional Weather Functional Requirements for NextGen Air Traffic Management Joint Planning and Development Office. Jan 18, 2008 < 18