Downloading and Compiling MET

Transcription

1 Downloading and Compiling MET John Halley Gotway

2 Release History METv0.9: Beta release July, 2007 METv1.0: First official release January, 2008 METv1.1: Incremental upgrades July, 2008 METv2.0: Probabilistic forecasts April, 2009 METv3.0: Ensembles September, 2010 METv3.0.1: Bugfixes March 17, 2011 METv3.1: Internal streamlining Jan 13, 2012 Pre-installed on tutorial machines registered users from 102 countries 50/25/15/10%: University/Gov t/nonprofit/private 30/15/7/3%: USA/China/India/Italy On-line and hands-on tutorials

3 Downloading MET Download MET release and compile locally. Register and download: Language: Primarily in C++ with calls to a Fortran library Supported Platforms and Compilers: 1. Linux with GNU compilers 2. Linux with Portland Group (PGI) compilers 3. Linux with Intel compilers 4. IBM machines with IBM compilers

4

5 Dependencies REQUIRED: GNU Make Utility ( IBM C++/Fortran Compilers (GNU, PGI, Intel, or Library BUFRLIB Library GNU Scientific Library (GSL) F2C or G2C Library (only for some compilers) RECOMMENDED: Unified Post-Processor ( WRF-Post COPYGB (included with R statistics and graphics package

6 Directory Structure File or Directory Contents README Makefile_gnu (pgi, intel, ibm) lib/ include/ src/ doc/ bin/ scripts/ data/ out/ Installation instructions and release notes. Top-level Makefile to be configured prior to building MET. Compiled libraries. Header files MET Source code. MET User s Guide. Built MET executables. Test scripts to be run after building MET. Sample data used by the test scripts. Output generated by the test scripts.

7 Building MET Steps for building MET: 1. Build required libraries. Same family of compilers for MET 2. Download and unpack latest MET patches. 3. Select the appropriate Makefile. GNU, PGI, Intel, or IBM 4. Edit the Makefile. C++ and Fortran compilers Paths for, BUFRLIB, and GSL libraries 5. Run Make to build all of the MET tools. 6. Run the test script and check for runtime errors. Runs each of the MET tools at least once. Uses sample data distributed with the tarball.

8 METv3.1 Flowchart Input Reformat Statistics Analysis Gridded GRIB Input: Observation Analyses Model Forecasts Point Obs Gen Poly Mask PCP Combine 2NC Mask Gridded Point Obs MODE Wavelet Stat Grid Stat Ensemble Stat PS STAT PS STAT STAT MODE Analysis Stat Analysis PrepBufr Point Obs PB2NC Point Obs Point Stat STAT = optional

9 Configuration Files MET tools controlled using command line options and configuration files Well commented and documented in MET User s Guide Easy to modify Distributed with the tarball Configuration files control things such as: Fields/levels to be verified Thresholds to be applied Interpolation methods to be used Verification methods to be applied Regions over which to accumulate statistics

10 Graphics Limited graphics incorporated into MET Options for plotting MET statistical output R, NCL, IDL, GNUPlot, and many others Sample plotting scripts on MET website Future METViewer database/display system

11 R Statistics and Graphics The R Project for Statistical Computing ( Powerful statistical analysis and plotting tools Large and growing user community Freely available and well supported for Linux/Windows/Mac Sample R plotting and analysis scripts posted on the MET website Use R to plot data in the practical sessions

12 Getting Started with MET A Typical Use Case John Halley Gotway

13 Use Case: Point Verification Input Reformat Statistics Analysis Gridded GRIB Input: Observation Analyses Model Forecasts Point Obs Gen Poly Mask PCP Combine 2NC Mask Gridded Point Obs MODE Wavelet Stat Grid Stat Ensemble Stat PS STAT PS STAT STAT MODE Analysis Stat Analysis PrepBufr Point Obs PB2NC Point Obs Point Stat STAT = optional

14 Use Case: 2-m Temperature Verify 2-m temperature versus PREPBUFR point observations. Initialized with 3-hourly output to 48h Run WRFOUT through WPP to create GRIB Unidata s IDV for display

15 Use Case: Find Observations MET downloads page. NCAR CISL Site: DS337.0 Global PREPBUFR point observations

16 Use Case: Get Observations Register and log in NCAR CISL Site: DS337.0 Select through Pull daily PREPBUFR files

17 Use Case: Process Observations GDAS PREPBUFR 4/day in 6-hr chunks nr = not restricted Run each through pb2nc All observation types Full 6-hr time window Quality marker of 9 Only over CONUS

18 Use Case: Plot 2-m TMP Obs 2-m TMP obs in ADPSFC message type Run plot_point_obs utility to display obs

19 Use Case: Verify 2-m TMP Fcst Run point_stat to compare gridded forecast to point observations. Once/fcst time Obs +/- 5 min Bilinear interp Score over FULL model domain Continuous statistics line (incl RMSE)

20 Use Case: Continuous Statistics One STAT output file for each lead time Continuous line type (CNT) including CI s Column 48hr fcst Column 48hr fcst TOTAL 1457 MAE FBAR MSE FSTDEV BCMSE OBAR RMSE OSTDEV E PR_CORR E ME E ESTDEV E MBIAS E

21 Use Case: Time Series of RMSE? Investigate 48hr RMSE = 3.39

22 Use Case: Conclusions Need to investigate 33 hr lead time Clear diurnal cycle for surface temperature RMSE best in the morning (perfect RMSE = 0) RMSE worst in the evening Performance degrades as lead time increases Too little data to make strong conclusions This is a single initialization Verify/aggregate over a month, season, or year

23 File Formats and Pre-Processing File Formats Pre-processing Tools Useful Links Tressa L. Fowler

24 FILE FORMATS

25 Supported File Formats Forecasts GRIB GRIdded Binary file format (*version 1) specificmet andpinterpwrffromoutput network Common Data Format Observations PREPBUFR binary dataset prepared by NCEP from varied data sources. METspecific format (10-cols x n-rows) gridded GRIB i.e. NEXRAD Level II or IV METspecific MADIS netcdf RAOB and METAR observations WWMCA World Wide Merged Cloud Analysis

26 GRIB vs GRIB2 GRIB (or GRIB1): Until 2008 NCEP and other WMO organizations have historical data in GRIB1 Unified postprocessor (UPP) produces GRIB1 format using copygb Performs horizontal interpolation and destaggering (in the case of WRF-NMM) onto a defined grid. Useful for both cores in creating an output grid not fixed by the model integration domain. GRIB2: NCEP and other WMO organizations now use GRIB 2 as the standard for gridded binary data. At least two GRIB2toGRIB1 converters available NCEP cnvgrib UCAR/CISL Grib Converter

27 New for V3.0 PINTERP output This utility interpolates WRF-ARW netcdf output files to user specified pressure levels. Utility can also destagger grid WRF Online Tutorial has more information on PINTERP utility WMMCA satellite product Binary Produced by Air Force

28 Data Inventory Tools wgrib dumps GRIB1 headers and data. wgrib2 dumps GRIB2 headers and data. ncdump - dumps headers and data. ncview plots gridded data. GrADS command line interface to produce plots. NCL command line interface to produce plots. IDV gui-driven visualization of many gridded and point datasets.

29 PRE-PROCESSING TOOLS

30 Pre-Processing / Reformatting Input Reformat Statistics Analysis Gridded GRIB Input: Observation Analyses Model Forecasts Point Obs Gen Poly Mask PCP Combine 2NC Mask Gridded Point Obs MODE Wavelet Stat Grid Stat Ensemble Stat PS STAT PS STAT STAT MODE Analysis Stat Analysis PrepBufr Point Obs PB2NC Point Obs Point Stat STAT = optional

31 Data Reformating Tools PB2NC, 2NC, MADIS2NC Arrange observational data into the point format expected by Point-Stat. PCP_Combine (optional) Sum precipitation values across two or more time periods. Subtract precipitation values to create values for finer subperiods. Produces gridded file that can be used as input grid for any Statistics tool. Gen_Poly_Mask (optional) Used when more complex masking is needed. Produces a file of pre-defined mask. May be used for masking in any Statistics tools.

32 1. PB2NC Tool Stands for PREPBUFR to Functionality: Filters and reformats PREPBUFR point observations into intermediate format. Configuration file specifies: Observation types, variables, locations, elevations, quality marks, and times to retain or derive for use in Point-Stat. Data formats:. BUFRLIBNCEP susingprepbufrreads Writes point as input to Point-Stat.

33 PREPBUFR BUFR is the World Meteorological Organization (WMO) standard binary code for the representation and exchange of observational data. The PREPBUFR format is produced by NCEP for analyses and data assimilation. The system that produces this format: Assembles observations dumped from a number of sources Encodes information about the observational error for each data type background (first guess) interpolation for each data location Performs both rudimentary multi-platform quality control and more complex platform-specific quality control North American and Global datasets MET currently only supports PREPBUFR.

34 PB2NC: Usage Usage: pb2nc prepbufr_file netcdf_file config_file [-pbfile prepbufr_file] [-valid_beg time] [-valid_end time] [-nmsg n] [-dump path] [-v level] prepbufr_file netcdf_file config_file -pbfile -valid_beg -valid_end -nmsg -dump Input obs file in PrepBufr format Output name for file PB2NC configuration file Additional input PrepBufr files Beginning of valid time window [YYYYMMDD_[HH[MMSS]] End of valid time window [YYYYMMDD_[HH[MMSS]] Number of PrepBufr messages to process Dump entire contents o PrepBufr file to file in path -v Level of logging

35 PB2NC: Run METv3.1/bin/pb2nc \ ndas.t00z.prepbufr.tm nr \ out/tutorial_pb.nc PB2NCConfig_tutorial -v 2 ==> append : to filename to view the data source BUFR 230ADPUPA UPPER-AIR (RAOB, PIBAL, RECCO, DROPS) REPORTS 231AIRCAR MDCRS ACARS AIRCRAFT REPORTS 232AIRCFT AIREP/PIREP, AMDAR(ASDAR/ACARS), E-ADAS(AMDAR BUFR) ACF233SATWND SATELLITE-DERIVED WIND REPORTS 234PROFLR WIND PROFILER REPORTS 235VADWND VAD (NEXRAD) WIND REPORTS 236SATEMP TOVS SATELLITE DATA (SOUNDINGS, RETRIEVALS, RADIANCES) 237ADPSFC SURFACE LAND (SYNOPTIC, METAR) REPORTS 238SFCSHP SURFACE MARINE (SHIP, BUOY, C-MAN PLATFORM) REPORTS 239SFCBOG MEAN SEA-LEVEL PRESSURE BOGUS REPORTS 240SPSSMI SSM/I RETRIEVAL PRODUCTS (REPROCESSED WIND SPEED, TPW) 241SYNDAT SYNTHETIC TROPICAL CYCLONE BOGUS REPORTS 242ERS1DA ERS SCATTEROMETER DATA (REPROCESSED WIND SPEED) 243GOESND GOES SATELLITE DATA (SOUNDINGS, RETRIEVALS, RADIANCES) 244QKSWND QUIKSCAT SCATTEROMETER DATA (REPROCESSED WIND SPEED) 245MSONET MESONET SURFACE REPORTS (COOPERATIVE NETWORKS) 246GPSIPW GLOBAL POSITIONING SATELLITE-INTEGRATED PRECIP. WATER 247RASSDA RADIO ACOUSTIC SOUNDING SYSTEM (RASS) TEMP PROFILE RPTSM063000BYTCNT What obs are in a PREPBUFR file? >less \ ndas.t00z.prepbufr.tm nr

36 2. 2NC Tool Stands for to Functionality: Reformat point observations into intermediate format. One input format supported (10 columns). No configuration file. Data formats: Reads MET Point. Writes point as input to Point-Stat. Future: support additional standard formats.

37 2NC: Usage Usage: ascii2nc ascii_file netcdf_file [-format ascii_format] [-v level] ascii_file netcdf_file -format Input obs file in format Output name for file Override default MET Point format (Future Option) -v Level of logging

38 Msg STID ValidTime Lat Lon Message type WMO Station ID Valid time for observation Latitude [North] Longitude [East] Elev Elevation [m] (Note: currently not used by MET code so can be filled with ) GC Lvl Hgt Ob MET Point Format Msg STID ValidTime Lat Lon Elev GC Lvl Hgt Ob ADPUPA _ ADPUPA _ ADPUPA _ ADPUPA _ ADPUPA _ ADPUPA _ ADPUPA _ GRIB code for variable (i.e. AccPrecip = 61; MSLP = 2; Temp = 11, etc ) Pressure [mb] or Accumulation Interval [hr] Height above Mean Sea Level [m MSL] (Note: currently not used by MET code so can be filled with ) Observed value

39 MET specific Format Msg STID ValidTime Lat Lon Elev GC Lvl Hgt Ob Ob assigns value to variable ADPUPA _ *HGT ADPUPA _ *TMP ADPUPA _ *DPT ADPUPA _ *RH ADPUPA _ *MixRat ADPUPA _ *HGT ADPUPA _ *TMP Msg STID ValidTime Lat Lon Message type WMO Station ID Valid time for observation Latitude [North] Longitude [East] Elev Elevation [m] (Note: currently not used by MET code so can be filled with ) GC Lvl Hgt Ob GRIB code for variable (i.e. AccPrecip = 61; MSLP = 2; Temp = 11, etc ) Pressure [mb] or Accumulation Interval [hr] Height above Mean Sea Level [m MSL] (Note: currently not used by MET code so can be filled with ) Observed value * Use a value of "-9999" to indicate missing data

40 2NC: Run METv3.1/bin/ascii2nc sample_obs.txt sample_ascii.nc -v 2 netcdf sample_ascii { dimensions: mxstr = 15 ; hdr_arr_len = 3 ; obs_arr_len = 5 ; nhdr = 5 ; nobs = UNLIMITED ; // (2140 currently) variables: char hdr_typ(nhdr, mxstr) ; Result of ncdump h Result of ncdump v obs_arr hdr_typ:long_name = "message type" ; char hdr_sid(nhdr, mxstr) ; hdr_sid:long_name = "station identification" ; char hdr_vld(nhdr, mxstr) ; hdr_vld:long_name = "valid time" ; hdr_vld:units = "YYYYMMDD_HHMMSS UTC" ; float hdr_arr(nhdr, hdr_arr_len) ; hdr_arr:long_name = "array of observation station header values" ; hdr_arr:_fill_value = f ; hdr_arr:columns = "lat lon elv" ; ; float obs_arr(nobs, obs_arr_len) ; obs_arr:long_name = "array of observation values" ; obs_arr:_fill_value = f ; obs_arr:columns = "hdr_id gc lvl hgt ob" ; obs_arr:hdr_id_long_name = "index of matching header data" ; ; obs_arr = 0, 7, 837, 1618, 1618, 1, 11, 837, 1618, , 2, 17, 837, 1618, , 3, 52, 837, 1618, 92, 4, 53, 837, 1618, , 5, 7, 826, 1724, 1724, 6, 11, 826, 1724, , 7, 17, 826, 1724, , 8, 52, 826, 1724, 84, 9, 53, 826, 1724, , 10, 7, 815.3, 1829, 1829, 11, 11, 815.3, 1829, , 12, 17, 815.3, 1829, , 13, 52, 815.3, 1829, 45, 14, 53, 815.3, 1829, , 15, 7, 815, 1832, 1832, 16, 11, 815, 1832, , 17, 17, 815, 1832, , 18, 52, 815, 1832, 44, 19, 53, 815, 1832, , 20, 7, 784.7, 2134, 2134, 21, 11, 784.7, 2134, , 22, 17, 784.7, 2134, , 23, 52, 784.7, 2134, 47,

41 3. PCP-Combine Tool Stands for Precip-Combine Functionality: Mathematically combines precipitation fields across multiple files. Add precipitation over 2 files 2 NMM output files to go from 3-hr to 6-hr accumulation. Sum precipitation over more than 2 files 12 WSR-88D Level II data to go from 5 min accumulation to 1-hr accumulation. Subtract precipitation in 2 files 2 ARW output files to go from 12 hr accumulations to 6 hour accumulation Specify field name on the command line. No configuration file. Data formats: Reads GRIB format. Writes gridded as input to stats tools.

42 PCP-Combine: Usage Usage: pcp_combine [-sum sum_args] or [-add add_args] or [-subtract sub_args] [-gc code] [-v level] -sum -add -subtract -gc Accumulates data over multiple files. Sum_args: (init_time, in_accum, valid_time, out_accum, out_file, -pcpdir path, -pcprx reg_exp) Accumulates data over two files. Add_args: (in_file1, Accum1, in_file2, Accum2, out_file). Subtracts data over two files. Sub_args: (in_file1, Accum1, in_file2, Accum2, out_file). GRIB code for variable (i.e. ACPC = 61; NCPCP =62; ACPCP =63, etc ). -v Level of logging

43 PCP-Combine: Run Two example command lines 1) Summing two 6-hourly accumulation forecast files into a single 12-hour accumulation forecast. METv3.1/bin/pcp_combine \ -add _ grb 6 \ _ grb 6 \ sample_fcst.nc -pcpdir data/ ) Summing 12 1-hourly accumulation observation files into a single 12-hour accumulated observation. METv3.1/bin/pcp_combine \ -sum _ \ _ \ sample_obs.nc -pcpdir data/st2ml

44 PCP-Combine: Example # _ hr acc _ hr acc _ hr acc Graphics produced using ncview

45 4. Gen Poly Mask Tool Stands for Generate Polyline Mask Functionality: Uses a lat/lon polyline to generate a 0/1 mask field to be applied to your data. Applies this mask once prior to running Point-Stat or Grid-Stat No configuration file. Data formats: Reads formatted polyline file. Reads GRIB file. Reads files from PCP-Combine. Writes gridded file of 0/1 mask. Accumulated Precip Accumulated Precip CONUS more points

46 MET Point-Stat Tool John Halley Gotway

47 Point-Stat Tool Input Reformat Statistics Analysis Gridded GRIB Input: Observation Analyses Model Forecasts Point Obs Gen Poly Mask PCP Combine 2NC Mask Gridded Point Obs MODE Wavelet Stat Grid Stat Ensemble Stat PS STAT PS STAT STAT MODE Analysis Stat Analysis PrepBufr Point Obs PB2NC Point Obs Point Stat STAT = optional

48 Point-Stat: Overview Compare gridded forecasts to point observations. Accumulate matched pairs over a defined area at a single point in time. Verify one or more variables/levels. Analysis tool provided to aggregate through time. Verification methods: Continuous statistics for raw fields. Single and Multi-Category counts and statistics for thresholded fields. Parametric and non-parametric confidence intervals for statistics. Compute partial sums for raw fields and/or the raw matched pair values. Methods for probabilistic forecasts.

49 Point-Stat: Input/Output Input Files Gridded forecast file GRIB output of Unified Post-Processor (or other) output of PCP-Combine or p_interp Point observation file output of PB2NC, 2NC, or MADIS2NC configuration file Output Files statistics file with all output lines (end with.stat ) Optional files sorted by line type with a header row (ends with _TYPE.txt )

50 Point-Stat: Usage Usage: point_stat fcst_file obs_file config_file [-climo climo_file] [-point_obs netcdf_file] [-fcst_valid time] [-fcst_lead time] [-obs_valid_beg time] [-obs_valid_end time] [-outdir path] [-v level] fcst_file obs_file config_file -climo -point_obs -fcst_valid -fcst_lead GRIB or NC forecast file NC point observation file (PB2NC or 2NC) configuration file Climo file for computing anomaly partial sums Additional NC point observation files Forecast valid time Forecast lead time -obs_valid_beg Beginning of valid time window for matching -obs_valid_end End of valid time window for matching -outdir Output directory to be used -v Level of logging

51 Point-Stat: Configuration 28 configurable parameters only set a few: Temperature at the surface (2-meter). fcst_field[] = [ TMP/Z2 ]; Temperature below freezing. fcst_thresh[] = [ gt273 gt283 gt293 ]; Match to observations at the surface. message_type[] = [ ADPSFC ]; Look at all the points in my domain. mask_grid[] = [ FULL ]; Match observation to the nearest forecast value. interp_wdth[] = [1]; Generate all possible statistic types, except probabilistic. output_flag[] = [ 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 0, 0, 0, 0, 2 ];

52 Point-Stat: Input 2-meter TMP 4255 TMP ADPSFC Obs

53 Point-Stat: Run METv3.0/bin/point_stat \ sample_fcst.grb sample_pb.nc \ PointStatConfig_TMPZ2 -outdir out -v 2 Forecast File: sample_fcst.grb Climatology File: none Configuration File: PointStatConfig_TMPZ2 Observation File: sample_pb.nc Reading records for TMP/Z2. For TMP/Z2 found 1 forecast levels and 0 climatology levels Searching observations from 9716 PrepBufr messages Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over region FULL, for interpolation method UW_MEAN(1), using 4250 pairs. Computing Categorical Statistics. Computing Multi-Category Statistics. Computing Continuous Statistics. Computing Scalar Partial Sums Output file: out/point_stat_360000l_ _120000v.stat Output file: out/point_stat_360000l_ _120000v_fho.txt Output file: out/point_stat_360000l_ _120000v_ctc.txt Output file: out/point_stat_360000l_ _120000v_cts.txt Output file: out/point_stat_360000l_ _120000v_mctc.txt Output file: out/point_stat_360000l_ _120000v_mcts.txt Output file: out/point_stat_360000l_ _120000v_cnt.txt Output file: out/point_stat_360000l_ _120000v_sl1l2.txt Output file: out/point_stat_360000l_ _120000v_sal1l2.txt Output file: out/point_stat_360000l_ _120000v_vl1l2.txt Output file: out/point_stat_360000l_ _120000v_val1l2.txt Output file: out/point_stat_360000l_ _120000v_mpr.txt

54 Point-Stat: Output Types Statistics line types: 15 possible Categorical Single Threshold Contingency table counts and stats (FHO, CTC, CTS) Categorical Multiple Thresholds NxN Contingency table counts and stats (MCTC, MCTS) Continuous - raw fields Continuous statistics (CNT) Partial Sums (SL1L2, SAL1L2, VL1L2, VAL1L2) Probabilistic Nx2 Contingency table counts and stats (PCT, PSTD) Continuous statistics and ROC curve (PJC, PRC) Matched pairs Raw matched pairs a lot of data! (MPR) 21 header columns common to all line types Remaining columns specific to each line type

55 Point-Stat: Sample Output 1. STAT file output for sample run: 1 line each for CNT, SL1L2, MCTC, MCTS 3 lines each for FHO, CTC, CTS 4,250 lines for MPR! 2. Additional TXT files for each line type Output file: out/point_stat_360000l_ _120000v.stat Output file: out/point_stat_360000l_ _120000v_fho.txt Output file: out/point_stat_360000l_ _120000v_ctc.txt Output file: out/point_stat_360000l_ _120000v_cts.txt Output file: out/point_stat_360000l_ _120000v_mctc.txt Output file: out/point_stat_360000l_ _120000v_mcts.txt Output file: out/point_stat_360000l_ _120000v_cnt.txt Output file: out/point_stat_360000l_ _120000v_sl1l2.txt Output file: out/point_stat_360000l_ _120000v_sal1l2.txt Output file: out/point_stat_360000l_ _120000v_vl1l2.txt Output file: out/point_stat_360000l_ _120000v_val1l2.txt Output file: out/point_stat_360000l_ _120000v_mpr.txt

56 Point-Stat: CTC Output Line VERSION V3.0 MODEL WRF FCST_LEAD FCST_VALID_BEG _ FCST_VALID_END _ OBS_LEAD OBS_VALID_BEG _ OBS_VALID_END _ FCST_VAR TMP FCST_LEV Z2 OBS_VAR TMP OBS_LEV Z2 OBTYPE ADPSFC VX_MASK FULL INTERP_MTHD UW_MEAN INTERP_PNTS 1 FCST_THRESH > OBS_THRESH > COV_THRESH NA ALPHA NA LINE_TYPE CTC TOTAL 4250 FY_OY 3275 FY_ON 245 FN_OY 102 FN_ON 628

57 Point-Stat: Matched Pairs Matched Pair (MPR) line type contains 1 line for each matched pair. Data overload! TOTAL INDEX OBS_SID OBS_LAT OBS_LON OBS_LVL OBS_ELV FCST OBS CLIMO NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA

58 Stat-Analysis Tool Filtering Summarizing Aggregating of Grid-Stat, Point-Stat, Ensemble-Stat & Wavelet-Stat output Tara Jensen

59 What can Stat Analysis do? Questions to MET Help - Can I get Q: Overall statistics for all gridded observations compared to the forecasts for hours 0 through 24 together? A: Yes - using Stat Analysis Tool on Grid-Stat output Q: Long-term statistics at individual sites (e.g., mean absolute error or RMS error for daily forecasts for a month)? A: Yes - using Stat Analysis Tool on Point-Stat output Q: My contingency table statistics aggregated over multiple runs? A: Yes using Stat Analysis Tool on any output Q: Statistics aggregated for a large number (N) of individual stations in one simultaneous run? A: Yes but it would be cumbersome. You would have to configure Stat Analysis Tool to run (N) number of jobs OR use our soon to be released METViewer tool. Copyright Copyright 2012, University 2012, University Corporation Corporation for Atmospheric for Atmospheric Research, Research, all rights all reserved rights reserved

60 Stat Analysis Tool Statistics Analysis User Defined Display MODE PS MODE Analysis Wavelet Stat Grid Stat Ensemble Stat STAT PS STAT STAT Stat Analysis User Graphics Package User Graphics Package Point Stat STAT For Stat Analysis Tool: MET provides the analysis in output. You provide the graphing / plotting capability.

61 Stat Analysis Jobs Filtering filter - filters out lines from one or more stat files based on user-specified filtering options. Summarizing summary - produces summary information from a single column of data including: mean, standard deviation, min, max, and the 10th, 25th, 50th, 75th, and 90th percentiles. Customized tool for AFWA go_index - computes the GO Index, a performance statistic used primarily by the United States Air Force.

62 Stat Analysis Jobs Aggregation aggregate - aggregates stat data across multiple time steps or masking regions. Output line type is the same as input line type. aggregate_stat aggregates across multiple times/regions then calculates statistics. Output line is different from input line types. Valid line type combinations include: -line_type FHO, CTC, -out_line_type CTS -line_type MCTC, -out_line_type MCTS -line_type SL1L2, SAL1L2, -out_line_type CNT -line_type VL1L2, VAL1L2, -out_line_type WDIR (wind direction) -line_type PCT, -out_line_type PSTD, PJC, PRC -line_type NBRCTC, -out_line_type NBRCTS -line_type MPR, -out_line_type FHO, CTC, CTS, MCTC, MCTS, CNT, SL1L2, SAL1L2,PCT, PSTD, PJC, PRC

63 Stat Analysis Tool: Usage Usage: stat_analysis -lookin path [-out filename] [-tmp_dir path] [-v level] -config config_file or job at command line options with associated arguments [filter] [summary] [aggregate] [aggregate_stat] [go_index] -lookin -out -tmp_dir Path to *.stat files this can be a directory or a single file name (Use one or more times) Output name for file Folder for temporary files -v Level of logging -config filter summary aggregate aggregate_stat go_index StatAnalysisConfig file See previous 2 slides See previous 2 slides See previous 2 slides See previous 2 slides See previous 2 slides

64 Stat Analysis Tool: Configuration 37 configurable parameters only set a few: Apply NAM G212 mask vx_mask[ ] = [ G212 ]; Using only the Temperature variable fcst_var[ ] = [ TMP ]; obs_var[ ] = []; Filter on CTC lines in which fcst_var[ ] > 278 K line_type[ ] = [ CTC"]; fcst_thresh[ ] = [ >278 ]; obs_thresh[ ] = []; Dump the filtered stat data to a file AND sum contingency table count (CTC) lines of data for pressure levels between 850 and 750 jobs[ ] = ["-job filter -dump_row out/filter_job.stat", \ "-job aggregate line_type CTC \ -dump_row out/aggr_ctc_job.stat fcst_lev P "];

65 Stat Analysis Tool: Run job aggregate "-job aggregate line_type CTC -dump_row out/aggr_ctc_job.stat \ -fcst_lev P " Point Stat Output (i.e. pointstat_out.stat) V2.0 WRF ADPUPA G212 TMP P > CTC UW_MEAN 1 F C S T Y OBS N Y N V2.0 WRF ADPSFC G212 TMP P > CTC UW_MEAN 1 (NOTE: header modified to show only pertinent info) F C S T OBS Y N Y N

66 Stat Analysis Tool: Run job aggregate Stat Analysis Output (i.e. aggr_ctc_job.stat ) FILTER: -job filter -vx_mask G212 -line_type CTC fcst_thresh > var TMP -dump_row out/filter_job.stat OBS JOB_LIST: -job aggregate -vx_mask G212 -line_type CTC -fcst_thresh > var TMP -level P dump_row out/aggr_ctc_job.stat COL_NAME: TOTAL FY_OY FY_ON FN_OY FN_ON INTERP_MTHD INTERP_PNTS CTC: F C S T Y N Y N

67 Stat Analysis Tool: Run job summary "-job summary line_type CNT -fcst_var TMP \ dump_row out/job_summary_rmse.stat column RMSE" Column Number Description 1 Summary (job type) 2 Total 3-7 Mean* Includes normal and bootstrap upper and lower confidence limits 8-10 Standard deviation** Includes bootstrap upper and lower confidence limits 11 Minimum value th percentile th percentile 14 Median (50 th percentile) th percentile th percentile 17 Maximum value (stat_analysis.out cont.) JOB_LIST: -job summary - line_type CNT. COL_NAME: TOTAL MEAN MEAN_NCL MEAN_NCU MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10 P25 P50 P75 P90 MAX SUMMARY: Copyright 2012, Copyright University 2012, Corporation University for Corporation Atmospheric for Atmospheric Research, all Research, rights reserved all rights reserved

68 Use your favorite plotting software Stat Analysis User Graphics Package MET provides the analysis in ascii ouput. You provide the graphing / plotting capability.

69 User Contributed Plotting Scripts Please feel free to send your contributions to