DART Tutorial Sec'on 16: Diagnos'c Output

Transcription

1 DART Tutorial Sec'on 16: Diagnos'c Output UCAR The Na'onal Center for Atmospheric Research is sponsored by the Na'onal Science Founda'on. Any opinions, findings and conclusions or recommenda'ons expressed in this publica'on are those of the author(s) and do not necessarily reflect the views of the Na'onal Science Founda'on.

2 DART Diagnos'c Output Categories: State-Space: Values of model s state vector and infla'on. Output using netcdf format. Observa'on-Space: Values of the observa'ons. DART-specific obs_sequence format for now. Regression confidence factor: Values for state vector / observa'on pairs. Output as flat ASCII (soon to be netcdf). Program diagnos'c output: Iden'fica'on for source code version and namelist values. Error, warning, message output from modules. DART Tutorial Sec'on 16: Slide 2

3 State-Space Diagnos'c Files: Available in netcdf (a common data format) hup:// DART outputs up to four state space diagnos'c files. These files are selected by lis'ng their names for the stages_to_write entry in the &filter_nml. The stages_to_write namelist entry and resul'ng netcdf file names are: forecast forecast.nc preassim postassim postassim.nc analysis analysis.nc In addi'on, stages_to_write can also include: input input.nc Copy of ini'al condi'ons, same format as output.nc output output.nc Output file for restart of subsequent filter steps. DART Tutorial Sec'on 16: Slide 3

4 State-Space Diagnos'c Files: Loca'on of each diagnos'c file in the filter cycle. Ensemble Forecasts forecast.nc Apply Prior Infla'on analysis.nc Apply Posterior Infla'on postassim.nc Assimila'on Updates State DART Tutorial Sec'on 16: Slide 4

5 State-Space Diagnos'c Files: Contents of state space diagnos'c files are controlled by &filter_nml: &filter_nml output_mean =.true. output_sd =.true. num_output_state_members = ## output_interval = N (include ensemble mean) (include ensemble spread) (include this many individual ensemble members) (only output every N th assimila'on 'me) Note: output_interval for true_state.nc is in the &perfect_model_obs_nml namelist. In input.nml for lorenz_96, make sure all diagnos'c files are listed as stages_to_write. Run the filter to generate all files. Try some Matlab diagnos'cs. You can change the diagnos'c file for a single plot by typing the file at the prompt, or You can change the file for all subsequent plots by secng Matlab variable diagn_file. For instance, diagn_file = postassim.nc ; DART Tutorial Sec'on 16: Slide 5

6 State-Space Diagnos'c Files: Trying out different diagnos'c files: In input.nml for lorenz_96, the default has been to output the and analysis.nc diagnos'c files. You could also add postassim and forecast to the list in stages_to_write. So far, have only looked at diagnos'cs for Two ways to change the diagnos'c file in Matlab tools like plot_total_err: 1). Change for a single plot by entering diagnos'c filename at Matlab prompt: >> plot_total_err Input name of ensemble trajectory file: <cr> for analysis.nc Comparing true_state.nc and analysis.nc 2).You can change the file for all subsequent plots by secng Matlab variable diagn_file. >> diagn_file = analysis.nc ; Try looking at diagnos'cs for analysis.nc, forecast.nc, and postassim.nc Some of these will be the same unless you have both prior and posterior infla'on on. DART Tutorial Sec'on 16: Slide 6

7 DART State-Space Diagnos'c func'ons See the DART website sec'on 'tled: Configuring Matlab to work with DART hup:// ALL the DART Matlab state-space diagnos'c func'ons are in diagnos?cs/matlab This must be in your matlabpath. Only focus on the files that start with plot_ plot_bins.m plot_correl.m plot_ens_err_spread.m plot_ens_mean_'me_series.m plot_ens_'me_series.m plot_phase_space.m plot_reg_factor.m plot_sawtooth.m plot_smoother_err.m plot_total_err.m plot_var_var_correl.m Some, but not all, described here. All func'ons have a help sec'on available in the standard Matlab way. DART Tutorial Sec'on 16: Slide 7

8 Viewing the State-Space netcdf files: 1. Standard DART matlab diagnos'cs: a. plot_bins: rank histograms, 8 Lorenz_96 Variable 1 for./prior_diag.nc Lorenz_96 Variable 13 for./prior_diag.nc Lorenz_96 Variable 27 for./prior_diag.nc DART 25 Tutorial Sec'on 16: Slide 8

9 Viewing the State-Space netcdf files: 1. Standard DART matlab diagnos'cs: b. plot_correl: correla'on x(t) with all other state vars at all 'mes, state variable (index) Lorenz_96 Correlation of variable state index 2, T = 5 of./prior_diag.nc against all variables, all times, all 2 ensemble members time (timestep #) DART Tutorial Sec'on 16: Slide 9

10 Viewing the State-Space netcdf files: 1. Standard DART matlab diagnos'cs: c. plot_ens_err_spread: rms error and spread, distance 1.5 Lorenz_96 model Var 1 Ensemble Error Spread for./prior_diag.nc time mean Ensemble Mean Total Error = time mean Ensemble Spread =.2779 distance distance model time (1 timesteps) Lorenz_96 model Var 13 Ensemble Error Spread for./prior_diag.nc time mean Ensemble Mean Total Error =.3668 time mean Ensemble Spread = model time (1 timesteps) Lorenz_96 model Var 27 Ensemble Error Spread for./prior_diag.nc 1.5 time mean Ensemble Mean Total Error = time mean Ensemble Spread = model time (1 timesteps) DART Tutorial Sec'on 16: Slide 1

11 Viewing the State-Space netcdf files: 1. Standard DART matlab diagnos'cs: d. plot_ens_mean_'me_series: just like the name says, 1 5 Lorenz_96 Variable 1 of./prior_diag.nc True State Ensemble Mean model time (1 timesteps) 1 5 Lorenz_96 Variable 13 of./prior_diag.nc True State Ensemble Mean model time (1 timesteps) 1 Lorenz_96 Variable 27 of./prior_diag.nc 5 True State Ensemble Mean model time (1 timesteps) DART Tutorial Sec'on 16: Slide 11

12 Viewing the State-Space netcdf files: 1. Standard DART matlab diagnos'cs: e. plot_ens_'me_series: plots the ensemble (as available from num_output_state_members ), Lorenz_96 state varnum 1 Ensemble Members of./prior_diag.nc True State Ensemble Mean Ensemble Members (2) model time (1 timesteps) Lorenz_96 state varnum 13 Ensemble Members of./prior_diag.nc 1 5 True State Ensemble Mean Ensemble Members (2) model time (1 timesteps) Lorenz_96 state varnum 27 Ensemble Members of./prior_diag.nc 1 5 True State Ensemble Mean Ensemble Members (2) model time (1 timesteps) DART Tutorial Sec'on 16: Slide 12

13 Viewing the State-Space netcdf files: 1. Standard DART matlab diagnos'cs: f. plot_phase_space: 3D phase space 'me evolu'on. The Attractor True_State.nc true true_state.nc Lorenz_96 true_state 1 State variable #3 state variable # state variable # state variable # State variable #1 1 DART Tutorial Sec'on 16: Slide 13

14 Viewing the State-Space netcdf files: 1. Standard DART matlab diagnos'cs: g. plot_sawtooth: truth, prior and posterior 'me series. 5 Lorenz_96 Trajectories state index truth ensemble mean ensemble member 5 ensemble member 1 ensemble member model time (25 timesteps) DART Tutorial Sec'on 16: Slide 14

15 Viewing the State-Space netcdf files: 1. Standard DART matlab diagnos'cs: h. plot_total_err: total error for different fields, 3.5 Lorenz_96 Total Error over all 4 variables for./prior_diag.nc time mean Ensemble Mean Total Error = time mean Ensemble Spread Total Error = Total Error model time (1 timesteps) DART Tutorial Sec'on 16: Slide 15

16 Viewing the State-Space netcdf files: 1. Standard DART matlab diagnos'cs: i. plot_var_var_correl: x(t) correla'on to single variable, all 'mes. 1 Lorenz_96 Correlation of variable state 2, T = 5, with variable state 18 2 ensemble members Prior_Diag.nc correlation time (timestep #) DART Tutorial Sec'on 16: Slide 16

17 Viewing the State-Space netcdf files: 2. ncview: a quick and surprisingly useful netcdf viewer. hup://meteora.ucsd.edu/~pierce/ncview_home_page.html Displays spa'al slices, anima'ons, 'me series model state or fcopy time (days since -- ::) State Variable ID (indexical) jla Sun Jun 5 13:41:44 25 prior ensemble state Range of model state or fcopy: to (null) Range of State Variable ID: 1 to 4 indexical Range of time: to 1 days since -- :: Current ensemble member or copy: 1 nondimensional Frame 1 in File Prior_Diag.nc DART Tutorial Sec'on 16: Slide 17

18 Viewing the State-Space netcdf files: 3. Many other graphical/analysis programs can read netcdf. (Note that we use udunits metadata conven'on.) 4. netcdf Operator (NCO) tools allow opera'ons on netcdf files: (hup://nco.sourceforge.net) Selec'ng hyperslices of fields, Differencing netcdf file, Averaging, etc. NASA GISS: Panoply DART Tutorial Sec'on 16: Slide 18

19 Infla'on Diagnos'cs in State-Space netcdf files: In addi'on to state variables, the netcdf files also contain 'me series of the state space infla'on mean value and infla'on standard devia'on if adap've infla'on is on. These fields are called: state_priorinf_mean, state_priorinf_sd, state_pos?nf_mean, and state_pos?nf_sd DART Tutorial Sec'on 16: Slide 19

20 Prior Infla'on State-Space Diagnos'c Files: Contents of prior infla'on fields for each diagnos'c file in the filter cycle. Undamped, same as previous 'me analysis.nc Ensemble Forecasts forecast.nc Apply Prior Infla'on Same as postassim.nc analysis.nc Damped, applied at this 'me Apply Posterior Infla'on Updated by assimila'on postassim.nc Assimila'on Updates State DART Tutorial Sec'on 16: Slide 2

21 Posterior Infla'on State-Space Diagnos'c Files: Contents of posterior infla'on fields for each diagnos'c file in the filter cycle. Undamped, same as previous 'me analysis.nc Ensemble Forecasts forecast.nc Apply Prior Infla'on Updated by observa'ons. analysis.nc Undamped, same as forecast Apply Posterior Infla'on Damped, what is applied postassim.nc Assimila'on Updates State DART Tutorial Sec'on 16: Slide 21

22 Observa'on-space files: Quick recap of standard observa'on sequence file names (all names are actually specified in namelists): obs_seq.in input to perfect_model_obs obs_seq.out output from perfect_model_obs, also input to filter obs_seq.final output from filter Observa'on sequence file output by filter has prior, posterior, observed value (and truth for OSSEs). For an overview, check out the DART webpage sec'on: hup:// Contents of obs_seq.final controlled by &filter_nml: &filter_nml obs_sequence_in_name = obs_seq.out obs_sequence_out_name = obs_seq.final num_output_obs_members = ## Name of input observa'on sequence file. Name of output observa'on sequence file. Output this many individual ensemble es'mates. DART Tutorial Sec'on 16: Slide 22

23 Observa'on-space diagnos'cs: The observa'on sequence file is not in a par'cularly user-friendly format. To aid in the evalua'on and interpreta'on, a program named obs_diag must be run to produce a netcdf file with results that can be ploued in a manner of your choosing. DART has Matlab func'ons/scripts that create high-quality graphics. See tutorial sec'on 18 for full coverage of viewing, diagnosing obs sequences. Here are a few of the Matlab func'ons available in diagnos?cs/matlab plot_rank_histogram.m plot_evolu'on.m plot_rmse_xxx_evolu'on.m two_experiments_evolu'on.m (works with more than two, actually) plot_profile.m plot_bias_xxx_profile.m plot_rmse_xxx_profile.m two_experiments_profile.m (works with more than two, actually) DART Tutorial Sec'on 16: Slide 23

24 Observa'on-space diagnos'cs: SOME of the informa'on in the observa'on sequence files can be converted to netcdf and easily ploued. A program named obs_seq_to_netcdf must be run to produce the netcdf. Here are a few of the Matlab func'ons available in diagnos?cs/matlab. link_obs.m plot_obs_netcdf.m plot_obs_netcdf_diffs.m DART Tutorial Sec'on 16: Slide 24

25 Regression confidence factor output: Reminder: reg_factor α introduced in Tutorial Sec'on 13 when running the group filter (with more than 1 group!). Controlled by &reg_factor_nml: &reg_factor_nml save_reg_diagnostics =.true. reg_diagnostics_file = reg_diagnostics Should file be output? Name of output file. File size could be (model size) X (number of obs.) X (number of assim 'mes). Very big, even for small models (only first 4 obs output default). Normally, modify code in reg_factor_mod.f9 to control: Output is at end of select_regression = 1 code block. Format is ASCII: 'me in days, 'me in seconds, obs_index, state_index, α Plot with Matlab plot_reg_factor. DART Tutorial Sec'on 16: Slide 25

26 Program Diagnos'c Output: File dart_log.out All DART executables append to this file! Contains: registra'on informa'on Program start 'me, version of code for each module used Namelist values for each module* Names of output files, Diagnos'c output for modules (through error_handler()), Warnings and fatal errors from DART code. Fair Warning: This file is not cleared by DART. Can get very longgggggg You should feel free to delete/rename it before star'ng the next experiment. *may be in a separate file, depending on &utilities_nml secng DART Tutorial Sec'on 16: Slide 26

27 DART Tutorial Index to Sec'ons 1. Filtering For a One Variable System 2. The DART Directory Tree 3. DART Run?me Control and Documenta?on 4. How should observa?ons of a state variable impact an unobserved state variable? Mul?variate assimila?on. 5. Comprehensive Filtering Theory: Non-Iden?ty Observa?ons and the Joint Phase Space 6. Other Updates for An Observed Variable 7. Some Addi?onal Low-Order Models 8. Dealing with Sampling Error 9. More on Dealing with Error; Infla?on 1. Regression and Nonlinear Effects 11. Crea?ng DART Executables 12. Adap?ve Infla?on 13. Hierarchical Group Filters and Localiza?on 14. Quality Control 15. DART Experiments: Control and Design 16. Diagnos?c Output 17. Crea?ng Observa?on Sequences 18. Lost in Phase Space: The Challenge of Not Knowing the Truth 19. DART-Compliant Models and Making Models Compliant 2. Model Parameter Es?ma?on 21. Observa?on Types and Observing System Design 22. Parallel Algorithm Implementa?on 23. Loca'on module design (not available) 24. Fixed lag smoother (not available) 25. A simple 1D advec?on model: Tracer Data Assimila?on DART Tutorial Sec'on 16: Slide 27