Downloading and Compiling MET

Transcription

1 Downloading and Compiling MET Julie Prestopnik

2 Release History MET was first released in 2007 METv4.1: Released May 22, 2013 New Tools: MODIS-Regrid Series-Analysis MET-Tropical Cyclone Enhancements: Spread-Skill Variance (SSVAR) for Ensemble-Stat -by case option added to STAT-Analysis Support for little_r and SURFRAD point obs format added to ASCII2NC Pre-installed on tutorial machines registered users from 119 countries 49/27/15/11%: University/Gov t/nonprofit/private 29/14/7/3%: USA/China/India/Italy On-line and hands-on tutorials

3 Downloading MET Download MET release and compile locally. Register and download: Language: Primarily in C++ with calls to a Fortran library Supported Platforms and Compilers: 1. Linux with GNU compilers 2. Linux with Portland Group (PGI) compilers 3. Linux with Intel compilers

4

5 Dependencies REQUIRED: GNU Make Utility ( Intel C++/Fortran Compilers (GNU, PGI, Unidata s NetCDF Library NCEP s BUFRLIB Library GNU Scientific Library (GSL) OPTIONAL: GRIB2 C-Library with JASPER, PNG, and Z libraries HDF4 and HDF-EOS2 libraries for MODIS-Regrid RECOMMENDED: Unified Post-Processor ( Processor COPYGB (included with Unified Post wgrib and wgrib2 R statistics and graphics package

6 Directory Structure File or Directory Contents README User_defs_xxx.mk (gnu, pgi, intel) lib/ include/ src/ doc/ bin/ scripts/ data/ out/ Installation instructions and release notes Top-level Makefile to be configured prior to building MET Compiled libraries Header files MET Source code MET User s Guide Built MET executables Test scripts to be run after building MET Sample data used by the test scripts Output generated by the test scripts

7 Building MET Steps for building MET: 1. Build required libraries. Same family of compilers for MET 2. Download and unpack latest MET patches. 3. Select the appropriate user_defs.mk file. GNU, PGI, or Intel 4. Edit the user_defs.mk file. C++ and Fortran compilers Paths for NetCDF, BUFRLIB, and GSL libraries The compilation of various tools can be turned on/off 5. Run Make to build all of the MET tools. 6. Run the test script and check for runtime errors. Runs each of the MET tools at least once. Uses sample data distributed with the tarball.

8 MET Flowchart

9 Configuration Files MET tools controlled using command line options and ASCII configuration files Well commented and documented in MET User s Guide Easy to modify Distributed with the tarball Configuration files control things such as: Fields/levels to be verified Thresholds to be applied Interpolation methods to be used Verification methods to be applied Regions over which to accumulate statistics README file in data/config area describes various settings

10 Graphics Limited graphics incorporated into MET Options for plotting MET statistical output R, NCL, IDL, GNUPlot, and many others Sample plotting scripts on MET website Future METViewer database/display system

11 R Statistics and Graphics The R Project for Statistical Computing ( Powerful statistical analysis and plotting tools Large and growing user community Freely available and well supported for Linux/ Windows/Mac Sample R plotting and analysis scripts posted on the MET website Use R to plot data in the practical sessions

12 Getting Started with MET A Typical Use Case Julie Prestopnik

13 Use Case: Point Verification

14 Use Case: 2-m Temperature Verify 2-m temperature versus PREPBUFR point observations. Initialized with 3-hourly output to 48h Run WRFOUT through UPP to create GRIB Unidata s IDV for display

15 Use Case: Find Observations MET downloads page Select Observation Datasets NCAR CISL Site: DS337.0 Global PREPBUFR point observations

16 Use Case: Get Observations Register and log in NCAR CISL Site: DS337.0 Select through Pull daily PREPBUFR files Can view by Faceted Browse or Complete File List (shown here)

17 Use Case: Process Observations GDAS PREPBUFR 4/day in 6-hr chunks nr = not restricted Run each through pb2nc All observation types Full 6-hr time window Quality marker of 9 Only over CONUS

18 Use Case: Plot 2-m TMP Obs 2-m TMP obs in ADPSFC message type Run plot_point_obs utility to display obs prepbufr.gdas t00z.nc

19 Use Case: Verify 2-m TMP Fcst Run point_stat to compare gridded forecast to point observations. Once/fcst time Obs +/- 5 min Bilinear interp Score over FULL model domain Continuous statistics line (incl RMSE)

20 Use Case: Continuous Statistics One STAT output file for each lead time Continuous line type (CNT) including CI s Column 48hr fcst Column 48hr fcst TOTAL 771 MAE FBAR MSE FSTDEV BCMSE OBAR RMSE OSTDEV E PR_CORR E ME E ESTDEV E MBIAS E

21 Use Case: Time Series of RMSE 48hr RMSE = 2.35

22 Use Case: Conclusions Diurnal cycle for surface temperature RMSE best in the morning (perfect RMSE = 0) RMSE worst in the evening Performance degrades as lead time increases Too little data to make strong conclusions This is a single initialization Verify/aggregate over a month, season, or year

23 File Formats and Pre-Processing File Formats Pre-processing Tools Useful Links Tressa L. Fowler

24 FILE FORMATS

25 Supported File Formats Forecasts GRIB GRIdded Binary file format (*version 1 or 2) NetCDF Output from WRF PINTERP and MET specific network Common Data Format Observations PREPBUFR binary dataset prepared by NCEP from varied data sources. ASCII MET specific format (11-cols x n-rows), little r, SURFRAD gridded GRIB i.e. NEXRAD Level II or IV NetCDF MET specific MADIS netcdf RAOB and METAR observations WWMCA World Wide Merged Cloud Analysis TRMM Tropical Rainfall Measuring Mission MODIS Moderate-Resolution Imaging Spectroradiometer

26 Data Inventory Tools wgrib dumps GRIB1 headers and data. wgrib2 dumps GRIB2 headers and data. ncdump - dumps NetCDF headers and data. ncview plots gridded NetCDF data. GrADS command line interface to produce plots. NCL command line interface to produce plots. IDV gui-driven visualization of many gridded and point datasets.

27 PRE-PROCESSING TOOLS

28 Pre-Processing / Reformatting

29 Data Reformating Tools PB2NC, ASCII2NC, MADIS2NC Arrange observational data into the NetCDF point format expected by Point-Stat. MODIS_Regrid Regrid MODIS observations into NetCDF gridded format expected by Grid-Stat or Series Analysis Tool. PCP_Combine (optional) Sum precipitation values across two or more time periods. Subtract precipitation values to create values for finer subperiods. Produces gridded NetCDF file that can be used as input grid for any Statistics tool. Gen_Poly_Mask (optional) Used when more complex masking is needed. Produces a NetCDF file of pre-defined mask. May be used for masking in any Statistics tools.

30 1. PB2NC Tool Stands for PREPBUFR to NetCDF Functionality: Filters and reformats PREPBUFR point observations into intermediate NetCDF format. Configuration file specifies: Observation types, variables, locations, elevations, quality marks, and times to retain or derive for use in Point-Stat. Data formats: Reads PREPBUFR using NCEP s BUFRLIB. Writes point NetCDF as input to Point-Stat.

31 PREPBUFR BUFR is the World Meteorological Organization (WMO) standard binary code for the representation and exchange of observational data. The PREPBUFR format is produced by NCEP for analyses and data assimilation. The system that produces this format: Assembles observations dumped from a number of sources Encodes information about the observational error for each data type background (first guess) interpolation for each data location Performs both rudimentary multi-platform quality control and more complex platform-specific quality control North American and Global datasets MET currently only supports PREPBUFR.

32 PB2NC: Usage Usage: pb2nc prepbufr_file netcdf_file config_file [-pbfile prepbufr_file] [-valid_beg time] [-valid_end time] [-nmsg n] [-dump path] [-log file] [-v level] prepbufr_file netcdf_file config_file -pbfile -valid_beg -valid_end -nmsg -dump -log Input obs file in PrepBufr format Output name for NetCDF file PB2NC configuration file Additional input PrepBufr files Beginning of valid time window [YYYYMMDD_[HH[MMSS]] End of valid time window [YYYYMMDD_[HH[MMSS]] Number of PrepBufr messages to process Dump entire contents o PrepBufr file to file in path Output file for log messages -v Level of logging

33 PB2NC: Run METv4.1/bin/pb2nc \ ndas.t00z.prepbufr.tm nr \ out/tutorial_pb.nc PB2NCConfig_tutorial -v 2 ==> append : to filename to view the data source BUFR 230ADPUPA UPPER-AIR (RAOB, PIBAL, RECCO, DROPS) REPORTS 231AIRCAR MDCRS ACARS AIRCRAFT REPORTS 232AIRCFT AIREP/PIREP, AMDAR(ASDAR/ACARS), E-ADAS(AMDAR BUFR) ACF233SATWND SATELLITE-DERIVED WIND REPORTS 234PROFLR WIND PROFILER REPORTS 235VADWND VAD (NEXRAD) WIND REPORTS 236SATEMP TOVS SATELLITE DATA (SOUNDINGS, RETRIEVALS, RADIANCES) 237ADPSFC SURFACE LAND (SYNOPTIC, METAR) REPORTS 238SFCSHP SURFACE MARINE (SHIP, BUOY, C-MAN PLATFORM) REPORTS 239SFCBOG MEAN SEA-LEVEL PRESSURE BOGUS REPORTS 240SPSSMI SSM/I RETRIEVAL PRODUCTS (REPROCESSED WIND SPEED, TPW) 241SYNDAT SYNTHETIC TROPICAL CYCLONE BOGUS REPORTS 242ERS1DA ERS SCATTEROMETER DATA (REPROCESSED WIND SPEED) 243GOESND GOES SATELLITE DATA (SOUNDINGS, RETRIEVALS, RADIANCES) 244QKSWND QUIKSCAT SCATTEROMETER DATA (REPROCESSED WIND SPEED) 245MSONET MESONET SURFACE REPORTS (COOPERATIVE NETWORKS) 246GPSIPW GLOBAL POSITIONING SATELLITE- INTEGRATED PRECIP. WATER 247RASSDA RADIO ACOUSTIC SOUNDING SYSTEM (RASS) TEMP PROFILE RPTSM063000BYTCNT What obs are in a PREPBUFR file? >less \ ndas.t00z.prepbufr.tm nr

34 2. ASCII2NC Tool Stands for ASCII to NetCDF Functionality: Reformat ASCII point observations into intermediate NetCDF format. Three input ASCII formats supported (11 column, little r, or SURFRAD). No configuration file. Data formats: Reads MET Point ASCII, little r (data assimilation), SURFRAD. Writes point NetCDF as input to Point-Stat. Future: support additional standard ASCII formats.

35 ASCII2NC: Usage Usage: ascii2nc ascii_file netcdf_file [-format ascii_format] [-log file] [-v level] ascii_file netcdf_file -format -log Input obs file in ASCII format Output name for NetCDF file met_point", "little_r, or surfrad log messages to the specified file -v Level of logging

36 Msg STID ValidTime Lat Lon Message type WMO Station ID Valid time for observation Latitude [North] Longitude [East] Elev Elevation [m] (Note: currently not used by MET code so can be filled with ) GC Lvl Hgt QC flag Ob MET Point ASCII Format Msg STID ValidTime Lat Lon Elev GC Lvl Hgt QC Ob ADPUPA _ NA 1618 ADPUPA _ NA ADPUPA _ NA ADPUPA _ NA 92 ADPUPA _ ADPUPA _ ADPUPA _ GRIB code for variable (i.e. AccPrecip = 61; MSLP = 2; Temp = 11, etc ) Pressure [mb] or Accumulation Interval [hr] Height above Mean Sea Level [m MSL] (Note: currently not used by MET code so can be filled with ) Quality control flag value Observed value

37 MET specific ASCII Format Msg STID ValidTime Lat Lon Elev GC Lvl Hgt QC Ob Ob assigns value to variable ADPUPA _ NA 1618 ADPUPA _ NA ADPUPA _ NA ADPUPA _ NA 92 ADPUPA _ ADPUPA _ ADPUPA _ Msg STID ValidTime Lat Lon Message type WMO Station ID Valid time for observation Latitude [North] Longitude [East] *HGT *TMP *DPT *RH *MixRat *HGT *TMP Elev Elevation [m] (Note: currently not used by MET code so can be filled with ) GC Lvl Hgt QC flag Ob GRIB code for variable (i.e. AccPrecip = 61; MSLP = 2; Temp = 11, etc ) Pressure [mb] or Accumulation Interval [hr] Height above Mean Sea Level [m MSL] (Note: currently not used by MET code so can be filled with ) Quality control flag value Observed value * Use a value of "-9999" to indicate missing data

38 ASCII2NC: Run METv4.1/bin/ascii2nc sample_obs.txt sample_ascii.nc -v 2 netcdf sample_ascii { dimensions: mxstr = 15 ; hdr_arr_len = 3 ; obs_arr_len = 5 ; nhdr = 5 ; nobs = UNLIMITED ; // (2140 currently) variables: char hdr_typ(nhdr, mxstr) ; hdr_typ:long_name = "message type" ; char hdr_sid(nhdr, mxstr) ; hdr_sid:long_name = "station identification" ; char hdr_vld(nhdr, mxstr) ; hdr_vld:long_name = "valid time" ; hdr_vld:units = "YYYYMMDD_HHMMSS UTC" ; float hdr_arr(nhdr, hdr_arr_len) ; Result of ncdump h Result of ncdump v obs_arr hdr_arr:long_name = "array of observation station header values" ; hdr_arr:_fill_value = f ; hdr_arr:columns = "lat lon elv" ; ; float obs_arr(nobs, obs_arr_len) ; obs_arr:long_name = "array of observation values" ; obs_arr:_fill_value = f ; obs_arr:columns = "hdr_id gc lvl hgt ob" ; obs_arr:hdr_id_long_name = "index of matching header data" ; ; obs_arr = 0, 7, 837, 1618, 1618, 1, 11, 837, 1618, , 2, 17, 837, 1618, , 3, 52, 837, 1618, 92, 4, 53, 837, 1618, , 5, 7, 826, 1724, 1724, 6, 11, 826, 1724, , 7, 17, 826, 1724, , 8, 52, 826, 1724, 84, 9, 53, 826, 1724, , 10, 7, 815.3, 1829, 1829, 11, 11, 815.3, 1829, , 12, 17, 815.3, 1829, , 13, 52, 815.3, 1829, 45, 14, 53, 815.3, 1829, , 15, 7, 815, 1832, 1832, 16, 11, 815, 1832, , 17, 17, 815, 1832, , 18, 52, 815, 1832, 44, 19, 53, 815, 1832, , 20, 7, 784.7, 2134, 2134, 21, 11, 784.7, 2134, , 22, 17, 784.7, 2134, , 23, 52, 784.7, 2134, 47,

39 3. MADIS2NC Tool Stands for MADIS to NetCDF Functionality: Reformat MADIS point observations into intermediate NetCDF format. No configuration file. Data formats: Reads MADIS RAOB, METAR, profiler, or maritime data. Writes point NetCDF as input to Point-Stat.

40 MADIS2NC: Usage Usage: madis2nc madis_file out_file -type str [-qc_dd list] [-lvl_dim list] [-rec_beg n] [-rec_end n] [-log file] [-v level] madis_file out_file -type str -qc_dd list -lvl_dim list -rec_beg n -rec_end n -log file Input obs file in MADIS netcdf format Output name for NetCDF file type of MADIS observations (metar, raob, profiler or maritime) QC flag values to be accepted (Z,C,S,V,X,Q,K,G,B) vertical level dimensions to be processed first MADIS record to process last MADIS record to process log messages to the specified file -v Level of logging Copyright 2014, University Corporation for Atmospheric Research, all rights reserved

41 MADIS2NC: Run METv4.1/bin/madis2nc profiler_ _1800.nc test.nc -type profiler -v 2 DEBUG 1: Reading MADIS File: profiler_ _1800.nc DEBUG 1: Writing MET File: test.nc DEBUG 2: Processing PROFILER recs = 22 DEBUG 2: Rejected based on QC = 0 DEBUG 2: Rejected based on fill = 1674 DEBUG 2: Retained or derived = 1494 Result of ncdump v obs_arr obs_arr = 0, 33, -9999, 1000, , 0, 34, -9999, 1000, , 0, 33, -9999, 1250, , 0, 34, -9999, 1250, , 0, 33, -9999, 2250, , 0, 34, -9999, 2250, , 0, 33, -9999, 2500, , 0, 34, -9999, 2500, , 0, 33, -9999, 3750, , 0, 34, -9999, 3750, , 1, 33, -9999, 500, , 1, 34, -9999, 500, , 1, 33, -9999, 750, , 1, 34, -9999, 750, , 1, 33, -9999, 1000, , 1, 34, -9999, 1000, , 1, 33, -9999, 1250, , 1, 34, -9999, 1250, , 1, 33, -9999, 1500, , 1, 34, -9999, 1500, , 1, 33, -9999, 1750, , 1, 34, -9999, 1750, ,

42 3. MODIS Regrid Tool Functionality: Reformat MODIS satellite observations into intermediate NetCDF format. No configuration file. Data formats: Reads MODIS level 2 data. Writes gridded NetCDF as input to Grid-Stat or Series Analysis Tool.

43 MODIS Regrid: Usage Usage: modis_regrid -data_file path -field name -out path -scale value -offset value -fill value [ -units text ] modis_file data_file field out Data files used to get grid informaiton Field to process in MODIS file, e.g. temperature Name of output NetCDF file -scale value Scale factor to use -offset value Offset factor -fill value -units text modis_file Bad data value Specifies units string in the global attributes section of the output file MODIS input file Copyright 2014, University Corporation for Atmospheric Research, all rights reserved

44 MODIS_REGRID: Run METv4.1/bin/modis_regrid -field Cloud_Fraction \ -data_file grid_file -out t2.nc \ -units percent -scale offset 0 -fill 127 /home/joeuser/modis_regrid_test_data/modisfile

45 3. PCP-Combine Tool Stands for Precip-Combine Functionality: Mathematically combines precipitation fields across multiple files. Add precipitation over 2 files 2 NMM output files to go from 3-hr to 6-hr accumulation. Sum precipitation over more than 2 files 12 WSR-88D Level II data to go from 5 min accumulation to 1-hr accumulation. Subtract precipitation in 2 files 2 ARW output files to go from 12 hr accumulations to 6 hour accumulation Specify field name on the command line. No configuration file. Data formats: Reads GRIB1, GRIB2, or pinterp NetCDF format. Writes gridded NetCDF as input to stats tools.

46 PCP-Combine: Usage Usage: pcp_combine [-sum sum_args] or [-add add_args] or [-subtract sub_args] [-log file] [-v level] [-config config_str] [-varname variable_name] -sum -add -subtract -config -varname Accumulates data over multiple files. Sum_args: (init_time, in_accum, valid_time, out_accum, out_file, -pcpdir path, -pcprx reg_exp) Accumulates data over two files. Add_args: (in_file1, Accum1, in_file2, Accum2, out_file). Subtracts data over two files. Sub_args: (in_file1, Accum1, in_file2, Accum2, out_file). configuration string to use when searching for records in input files name of combined variable in output NetCDF file

47 PCP-Combine: Run Two example command lines 1) Summing two 6-hourly accumulation forecast files into a single 12-hour accumulation forecast. METv4.1/bin/pcp_combine \ -add _ grb 6 \ _ grb 6 \ sample_fcst.nc -pcpdir data/ ) Summing 12 1-hourly accumulation observation files into a single 12-hour accumulated observation. METv4.1/bin/pcp_combine \ -sum _ \ _ \ sample_obs.nc -pcpdir data/st2ml

48 PCP-Combine: Example # _ hr acc _ hr acc _ hr acc Graphics produced using ncview

49 4. Gen Poly Mask Tool Stands for Generate Polyline Mask Accumulated Precip CONUS Functionality: Uses a lat/lon polyline to generate a 0/1 mask field to be applied to your data. Applies this mask once prior to running Point-Stat or Grid-Stat No configuration file more points Data formats: Reads ASCII formatted polyline file. Reads GRIB file. Reads NetCDF files from PCP-Combine. Writes gridded NetCDF file of 0/1 mask. Accumulated Precip

50 Point-Stat Tool John Halley Gotway

51 Point-Stat Tool Input Reformat Plot Statistics Analysis MET-TC Gridded Model Forecasts and Observation Analyses MODIS Data WWMCA Data ASCII Point Obs PrepBufr Point Obs MADIS Point Obs Gen Poly Mask PCP Combine MODIS Regrid WWMCA Regrid ASCII2NC PB2NC MADIS2NC NetCDF Mask File Gridded NetCDF NetCDF Point Obs Plot Data Plane WWMCA Plot Plot Point Obs PS MODE Wavelet Stat Grid Stat Series Analysis Ensemble Stat Point Stat ASCII NetCDF PS STAT ASCII NetCDF PS STAT ASCII NetCDF NetCDF STAT ASCII NetCDF STAT ASCII MODE Analysis ASCII Stat Analysis NetCDF DLand TC PAIRS TCST Pairs TC STAT ASCII Land Data File TC DLAND ATCF Track Data

52 Point-Stat: Overview Compare gridded forecasts to point observations. Accumulate matched pairs over a defined area at a single point in time. Verify one or more variables/ levels. Analysis tool provided to aggregate through time. Verification methods: Continuous statistics for raw fields. Single and Multi-Category counts and statistics for thresholded fields. Parametric and non-parametric confidence intervals for statistics. Compute partial sums for raw fields and/or the raw matched pair values. Methods for probabilistic forecasts.

53 Point-Stat: Input/Output Input Files Gridded forecast file GRIB1 output of Unified Post-Processor (or other) GRIB2 data from NCEP (or other) NetCDF output of PCP-Combine, p_interp, or other Point observation file NetCDF output of PB2NC, ASCII2NC, or MADIS2NC ASCII configuration file Output Files ASCII statistics file with all output lines (end with.stat ) Optional ASCII files sorted by line type with a header row (ends with _TYPE.txt )

54 Point-Stat: Usage Usage: point_stat fcst_file obs_file config_file [-climo climo_file] [-point_obs netcdf_file] [-obs_valid_beg time] [-obs_valid_end time] [-outdir path] [-log file] [-v level] fcst_file obs_file config_file -climo -point_obs Gridded forecast file NC point observation file ASCII configuration file Climo file for computing anomaly partial sums Additional NC point observation files -obs_valid_beg Beginning of valid time window for matching -obs_valid_end End of valid time window for matching -outdir -log Output directory to be used Optional log file -v Level of logging

55 Point-Stat: Configuration Many configurable parameters only set a few: 2-meter temperature. Threshold temperatures near freezing. Match to obs at the surface. Accumulate stats over all the points in the domain. Match observation to the nearest forecast value. Generate all possible output types, except probabilistic. fcst = { message_type = [ "ADPSFC"]; field = [ { name = "TMP"; level = [ "Z2" ]; cat_thresh = [ >273.0, >283.0, >293.0 ]; } ]; }; obs = fcst; mask = { grid = [ "FULL" ]; poly = []; sid = ""; }; interp = { vld_thresh = 1.0; type = [ { method = UW_MEAN; width = 1; } ]; }; output_flag = { fho = BOTH; ctc = BOTH; cts = BOTH; mctc = BOTH; mcts = BOTH; cnt = BOTH; sl1l2 = BOTH; sal1l2 = BOTH; vl1l2 = BOTH; val1l2 = BOTH; pct = NONE; pstd = NONE; pjc = NONE; prc = NONE; mpr = BOTH; };

56 Point-Stat: Input 2-meter TMP (IDV) 4003 TMP ADPSFC Obs (plot_point_obs)

57 Point-Stat: Run METv4.1/bin/point_stat \ sample_fcst.grb sample_pb.nc \ PointStatConfig_TMPZ2 -outdir out -v 2 DEBUG 1: Default Config File: METv4.1/data/config/PointStatConfig_default DEBUG 1: User Config File: PointStatConfig_TMPZ2 DEBUG 1: Forecast File: sample_fcst.grb DEBUG 1: Climatology File: none DEBUG 1: Observation File: sample_pb.nc DEBUG 2: DEBUG 2: Reading data for TMP/Z2. DEBUG 2: For TMP/Z2 found 1 forecast levels and 0 climatology levels. DEBUG 2: DEBUG 2: Searching observations from 9396 messages. DEBUG 2: DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over region FULL, for interpolation method UW_MEAN(1), using 4003 pairs. DEBUG 2: Computing Categorical Statistics. DEBUG 2: Computing Multi-Category Statistics. DEBUG 2: Computing Continuous Statistics. DEBUG 2: Computing Scalar Partial Sums. DEBUG 2: DEBUG 1: Output file: out/point_stat_360000l_ _120000v.stat DEBUG 1: Output file: out/point_stat_360000l_ _120000v_fho.txt DEBUG 1: Output file: out/point_stat_360000l_ _120000v_ctc.txt DEBUG 1: Output file: out/point_stat_360000l_ _120000v_cts.txt DEBUG 1: Output file: out/point_stat_360000l_ _120000v_mctc.txt DEBUG 1: Output file: out/point_stat_360000l_ _120000v_mcts.txt DEBUG 1: Output file: out/point_stat_360000l_ _120000v_cnt.txt DEBUG 1: Output file: out/point_stat_360000l_ _120000v_sl1l2.txt DEBUG 1: Output file: out/point_stat_360000l_ _120000v_sal1l2.txt DEBUG 1: Output file: out/point_stat_360000l_ _120000v_vl1l2.txt DEBUG 1: Output file: out/point_stat_360000l_ _120000v_val1l2.txt DEBUG 1: Output file: out/point_stat_360000l_ _120000v_mpr.txt

58 Point-Stat: ASCII Output Types Statistics line types: 15 possible Categorical Single Threshold Contingency table counts and stats (FHO, CTC, CTS) Categorical Multiple Thresholds NxN Contingency table counts and stats (MCTC, MCTS) Continuous - raw fields Continuous statistics (CNT) Partial Sums (SL1L2, SAL1L2, VL1L2, VAL1L2) Probabilistic Nx2 Contingency table counts and stats (PCT, PSTD) Continuous statistics and ROC curve (PJC, PRC) Matched pairs Raw matched pairs a lot of data! (MPR) 21 header columns common to all line types Remaining columns specific to each line type

59 Point-Stat: Sample Output 1. STAT file output for sample run: 1 line each for CNT, SL1L2, MCTC, MCTS 3 lines each for FHO, CTC, CTS 4,003 lines for MPR! 2. Additional TXT files for each line type Output file: out/point_stat_360000l_ _120000v.stat Output file: out/point_stat_360000l_ _120000v_fho.txt Output file: out/point_stat_360000l_ _120000v_ctc.txt Output file: out/point_stat_360000l_ _120000v_cts.txt Output file: out/point_stat_360000l_ _120000v_mctc.txt Output file: out/point_stat_360000l_ _120000v_mcts.txt Output file: out/point_stat_360000l_ _120000v_cnt.txt Output file: out/point_stat_360000l_ _120000v_sl1l2.txt Output file: out/point_stat_360000l_ _120000v_sal1l2.txt Output file: out/point_stat_360000l_ _120000v_vl1l2.txt Output file: out/point_stat_360000l_ _120000v_val1l2.txt Output file: out/point_stat_360000l_ _120000v_mpr.txt

60 Point-Stat: CTC Output Line VERSION V4.1 MODEL WRF FCST_LEAD FCST_VALID_BEG _ FCST_VALID_END _ OBS_LEAD OBS_VALID_BEG _ OBS_VALID_END _ FCST_VAR TMP FCST_LEV Z2 OBS_VAR TMP OBS_LEV Z2 OBTYPE ADPSFC VX_MASK FULL INTERP_MTHD UW_MEAN INTERP_PNTS 1 FCST_THRESH > OBS_THRESH > COV_THRESH NA ALPHA NA LINE_TYPE CTC TOTAL 4003 FY_OY (hits) 3111 FY_ON (f.a.) 78 FN_OY (miss) 215 FN_ON (c.n.) 599

61 Point-Stat: Matched Pairs Matched Pair (MPR) line type contains 1 line for each matched pair. Data overload! TOTAL INDEX OBS_SID OBS_LAT OBS_LON OBS_LVL OBS_ELV FCST OBS CLIMO NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA

62 Stat-Analysis Tool Filtering Summarizing Aggregating of Grid-Stat, Point-Stat, Ensemble-Stat & Wavelet-Stat output Tara Jensen

63 What can Stat Analysis do? Questions to MET Help - Can I get Q: Overall statistics for all gridded observations compared to the forecasts for hours 0 through 24 together? A: Yes - using Stat Analysis Tool on Grid-Stat output Q: My contingency table statistics aggregated over multiple runs? A: Yes using Stat Analysis Tool on any output Q: Long-term statistics at individual sites (e.g., mean absolute error or RMS error for daily forecasts for a month)? A: Yes - using Stat Analysis Tool on Point-Stat output Q: Statistics aggregated for a large number (N) of individual stations in one simultaneous run? A: Yes but it would be cumbersome. You would have to configure Stat Analysis Tool to run (N) number of jobs OR use our soon to be released METViewer tool.

64 Stat Analysis Tool Statistics Analysis User Defined Display MODE ASCII NetCDF PS MODE Analysis Wavelet Stat Grid Stat Ensemble Stat STAT ASCII NetCDF PS STAT ASCII NetCDF STAT ASCII NetCDF Stat Analysis ASCII ASCII User Graphics Package User Graphics Package Point Stat STAT For Stat Analysis Tool: MET provides the analysis in ASCII output. You provide the graphing / plotting capability.

65 Stat Analysis Jobs Filtering filter - filters out lines from one or more stat files based on user-specified filtering options. Summarizing summary - produces summary information from a single column of data including: mean, standard deviation, min, max, and the 10th, 25th, 50th, 75th, and 90th percentiles. Customized tool for AFWA go_index - computes the GO Index, a performance statistic used primarily by the United States Air Force.

66 Stat Analysis Jobs Aggregation aggregate - aggregates stat data across multiple time steps or masking regions. Output line type is the same as input line type (i.e. SSVAR SSVAR) aggregate_stat aggregates across multiple times/ regions then calculates statistics. Output line is different from input line types. Valid line type combinations include: -line_type FHO, CTC yields -out_line_type CTS -line_type MCTC yields -out_line_type MCTS -line_type SL1L2, SAL1L2 yields -out_line_type CNT -line_type VL1L2, VAL1L2 yields -out_line_type WDIR -line_type PCT yields -out_line_type PSTD, PJC, PRC -line_type NBRCTC yields -out_line_type NBRCTS -line_type MPR yields -out_line_type FHO, CTC, CTS MCTC, MCTS, CNT, SL1L2 SAL1L2, PCT, PSTD, PJC, PRC

67 Stat Analysis Tool: Usage Usage: stat_analysis -lookin path [-out filename] [-tmp_dir path] [-v level] -config config_file or job at command line options with associated arguments [filter] [summary] [aggregate] [aggregate_stat] [go_index] -lookin -out -tmp_dir Path to *.stat files this can be a directory or a single file name (Use one or more times) Output name for ASCII file Folder for temporary files -v Level of logging -config filter summary aggregate aggregate_stat go_index StatAnalysisConfig file See previous 2 slides See previous 2 slides See previous 2 slides See previous 2 slides See previous 2 slides

68 Stat-Analysis: Configuration Many configurable parameters only set a few: 10-m U-component of wind. Aggregate stats over DTC165 and DTC166 regions Accumulate only CTCs calculated using Distance- Weighted Mean interpolation Dump lines included in accumulation Dump aggregation to file - OR - You can put it all in the jobs area fcst_var = ["UGRD"]; obs_var = []; fcst_lev = []; obs_lev = []; obtype = []; vx_mask = ["DTC165", "DTC166"]; interp_mthd = ["DW_MEAN"]; jobs = [ "-job filter -line_type CTC -dump_row outdir/ job_filter_ctc_ugrd.stat", "-job aggregate -line_type CTC -dump_row outdir/ job_aggregate_ctc_ugrd.stat" ]; - OR - jobs = [ "-job filter -line_type CTC dump_row out/job_filter_ctc_ugrd.stat", "-job aggregate -line_type CTC -fcst_var UGRD -vx_mask DTC165 -vx_mask DTC166 -interp_mthd DW_MEAN -dump_row out/ job_aggregate_ctc_ugrd.stat" ];

69 Stat Analysis Tool: Run job aggregate "-job aggregate -line_type CTC -fcst_var UGRD -vx_mask DTC165 -vx_mask DTC166 -interp_mthd DW_MEAN -dump_row out/job_aggregate.stat" Stat Analysis Filter Output in job_aggregate.stat V4.1 WRF _ _ _ _ UGRD Z10 UGRD Z10 ADPSFC DTC165 DW_MEAN 9 >=5.000 >=5.000 CTC NA NA V4.1 WRF _ _ _ _ UGRD Z10 UGRD Z10 ADPSFC DTC166 DW_MEAN >= NA >=5.000 NA CTC (NOTE: header modified to show only pertinent info) F C S T F C S T OBS Y N Y N OBS Y N Y N

70 Stat Analysis Tool: Run job aggregate "-job aggregate -line_type CTC -fcst_var UGRD -vx_mask DTC165 -vx_mask DTC166 -interp_mthd DW_MEAN -dump_row out/job_aggregate.stat" Stat Analysis Output in the file specified by out flag (i.e. stat_analysis.out) JOB_LIST: -job aggregate -fcst_var UGRD -vx_mask DTC165 - vx_mask DTC166 -interp_mthd DW_MEAN -line_type CTC -dump_row out/aggregate2.stat COL_NAME: TOTAL FY_OY FY_ON FN_OY FN_ON CTC: F C ST OBS Y N Y N

71 Stat Analysis Tool: Run job aggregate_stat "-job aggregate_stat -line_type CTC out_line_type CTS -fcst_var UGRD - vx_mask DTC165 -vx_mask DTC166 -interp_mthd DW_MEAN -dump_row out/ job_aggregate_stat.stat" Aggregate_stat Output (stat_analysis.out continued) COL_NAME: TOTAL BASER BASER_NCL BASER_NCU BASER_BCL BASER_BCU FMEAN FMEAN_NCL FMEAN_NCU FMEAN_BCL FMEAN_BCU ACC ACC_NCL ACC_NCU ACC_BCL ACC_BCU FBIAS FBIAS_BCL FBIAS_BCU PODY PODY_NCL PODY_NCU PODY_BCL PODY_BCU PODN PODN_NCL PODN_NCU PODN_BCL PODN_BCU POFD POFD_NCL POFD_NCU POFD_BCL POFD_BCU FAR FAR_NCL FAR_NCU FAR_BCL FAR_BCU CSI CSI_NCL CSI_NCU CSI_BCL CSI_BCU GSS GSS_BCL GSS_BCU HK HK_NCL HK_NCU HK_BCL HK_BCU HSS HSS_BCL HSS_BCU ODDS ODDS_NCL ODDS_NCU ODDS_BCL ODDS_BCU CTS: NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA F C ST Y OBS N Y N Base Rate: 0.04 Freq Bias: 1.27 PODY: 0.35 FAR: 0.72 CSI: 0.18 GSS: 0.15

72 Stat Analysis Tool: Run job summary -job summary -fcst_var UGRD -interp_mthd DW_MEAN -line_type CTS -column GSS -dump_row out/job_summary.stat # Description 1 Column Name Summary 2 Total Mean* Includes normal and bootstrap upper and lower confidence limits 8-10 Standard deviation** Includes bootstrap upper and lower confidence limits Minimum value th percentile th percentile Median (50 th percentile) th percentile th percentile Maximum value Summary Output (stat_analysis.out cont.) COL_NAME: TOTAL MEAN MEAN_NCL MEAN_NCU MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10 P25 P50 P75 P90 MAX SUMMARY:

73 Use your favorite plotting software Stat Analysis ASCII User Graphics Package MET provides the analysis in ascii ouput. You provide the graphing / plotting capability.

74 User Contributed Plotting Scripts Please feel free to send your contributions to