An introduction to HYCOM

Transcription

1 CCS workshop, April 2013 An introduction to HYCOM Matthieu Le Hénaff (1) (1) RSMAS/CIMAS, Miami, FL

2 Why HYCOM? HYCOM stands for HYbrid Coordinate Ocean Model; it merges various approaches for the representation of the vertical coordinate in the model: - in the ocean interior: isopycnal coordinates, i.e. defined by the density of the ocean layer (MICOM model). - at the surface: pressure, or z- levels, for adequate representation of the mixed layer, also for highly stratified layers. - along steep bathymetry: sigma layers, which are terrain-following. model adapted for any type of study area; in particular, it is welldesigned for studying transition between the continental shelf and the ocean interior. (adapted from Schiller et al., 2011, JGR)

3 What do you need to run HYCOM? Grid: based on the horizontal resolution chosen (1/12, 1/25 etc.) Bathymetry: ETOPO, GEBCO etc Initial conditions: climatology, or output from another simulation. Forcing: atmospheric conditions from climatology, or atmospheric model; river outflow from climatology or observations; photosynthetically available radiation coefficient, from satellite measurements (used for shortwave flux penetration). Boundary conditions: relaxation to climatology, or nesting into another simulation. But also Parameters for the model simulation and the multi-processing, and a compiled model executable. All these files are organized rationally on the computer.

4 Set-up of hycom directory Main subdirectories: ALL: toolbox to pre-process, or post-process files used for, or generated during the simulation various domains where one the model is run. Examples: - ATLb GOMl GLBa etc. In these names: - ATL means Atlantic, GOM means Gulf of Mexico, GLB mean global etc. - the small letter is the version of the grid on the domain. - the number following the root name is the horizontal resolution in degree: 2.00 is 2 resolution, 0.04 is 1/25 resolution etc.

5 Set-up of hycom directory Within the domain subdirectories: src: source directory for hycom code. Specific to a domain because the code needs information about the grid (and its decomposition for parallel computing) prior to compilation. Name typically extended to show version of the Hycom code used; example: src_2.2.18_20_mpi. config: contains files with compilation options, depending on the type of machine. topo: contains the grid and bathymetry information. relax: contains files for relaxation, at the boundaries or at the surface; usually based on climatology. nest: contains files for boundary conditions, in case of nesting. Based on another model simulation over a domain encompassing the study area. NB: Files in relax and nest have to be consistent with the grid configuration (spatial resolution, definition of vertical layers)

6 Set-up of hycom directory force: contains forcing fields. Typical subdirectories: - flux: fields used to calculate air-sea fluxes. - pcip: precipitation. - wind: wind speed and wind stress vectors. - rivers: river outflow. - seawifs: photosynthetically available radiation coefficient. experiment directories: expt_01.0, ext_01.1, etc. topo directory: grid and bathymetry files regional.grid.a + regional.grid.b : grid regional.depth.a + regional.depth.b : bathymetry NB: HYCOM files in a specific format, by pair: full data file *.a (binary), and header file *.b (in ascii). regional.depth files usually have more specific name related to the grid: - depth_goml0.04_71.[a,b] - depth_atlb2.00_01.[a,b] - etc.

7 relax directory relaxation fields used for: - initial and/or boundary conditions in case of simulations based on climatology. - surface relaxation in case of temperature or salinity relaxation needed. files relax.temp.[a,b], relax.saln.[a,b] and nest_rmu.[a,b] Typically use of monthly GDEM3 climatology + executables z_gdem3 and relax_sig0 or relax_sig2, in ~/hycom/all/relax/src/ to project fields on the specific HYCOM grid (horizontal and vertical interpolation). nest_rmu.[a,b] contains information on the buffer zone at the boundary (width and time-dependence). Created by rmu, in ~/hycom/all/relax/src nest directory contains archive fields with the same format, and on the same grid, as the specific HYCOM grid for the specified experiment: the fields are typically projected from a lower resolution, greater domain simulation. In case this outer simulation is HYCOM, the executable isubregion in ~/ hycom/all/subregion/src/, to extract fields from the outer domain on the desired area, and project fields on the higher resolution grid.

8 force directory atmospheric forcing: fields projected from an atmospheric model on the HYCOM horizontal grid. It includes: in flux subdirectory, created with ap in ~/hycom/all/force/src/ : - airtmp.[a,b]: air temperature at the surface of the sea. - radflx.[a,b]: radiative flux. - shwflx.[a,b]: shortwave flux. - vapmix.[a,b]: evaporation in pcip, created with ap in ~/hycom/all/force/src/ : - pcip.[a,b]: precipition. in wind, created with wi in ~/hycom/all/force/src/ : - wndspd.[a,b]: wind speed. - tauewd.[a,b]: zonal wind stress. - taunwd.[a,b]: meridional wind stress. Variety of atmospheric models: NOGAPS, COAMPS, GFS etc.

9 force directory river forcing: based on observations; climatology or actual data: - monthly values: pcip_riv_mon in ~/hycom/all/force/src/ - daily values: pcip_riv_hf in ~/hycom/all/force/src/ radiation coefficient: kpar.[a,b], based on monthly Seawifs observations, using kp in ~/hycom/all/force/src/

10 src directory: compilation of the HYCOM code file dimensions.h : - contains information about the grid: itdm, jtdm, kdm. These must be consistent with regional.grid.[a,b]. - contains information on the grid decomposition used for parallel computing, depending on the number of processors used: - iqr, jqr: number of tiles in the x- and y-directions. - idm, jdm: size of each subdomain. - these numbers are determined by the executable partit, in ~/hycom/ ALL/topo/src/, together with the file patch.input, necessary for running HYCOM. file Makefile : procedure and files needed for compiling HYCOM. It shouldn t be modified. in../config/ : configuration files to use with the Makefile, depending on your system. This allows the Makefile to use the appropriate compiler, and find all the necessary libraries. compilation: type make ARCH=ARCH_type

11 Set-up of the experiment directory: expt_01.0, expt_01.1 etc. contains: - the executable hycom - the partition file patch.input - the topo files (regional.grid and regional.depth) - a file named limits with the initial and final days of the simulation, expressed in days since January 1 st, symbolic links to: - a /nest directory that contains the boundary conditions and nesting configuration (file nest_rmu.[a,b]). - all the files used for forcing (atmospheric and rivers). NB: symbolic links point toward files that are not present in the local directory, without copying them. Procedure: ln -s remote_file local_file!!! Caution!!! : any action that alters the file locally will equally affect the remote file (since there is no duplication). E.g., copying a file with the same name into the local directory will affect the original, remote file. Use rm to break the link (this command won t affect the original file).

12 Set-up of the experiment directory: contains also: - the initial conditions: restart_in.[a,b]. This restart file is generated from a HYCOM archive file, with archv2restart, in ~/hycom/all/archive/src/. This executable creates a restart file from an archive file, which usually comes from another simulation. - the parameter file for the HYCOM simulation, named blkdat.input. This file contains all the internal parameters for the model. Some of these parameters are: - the grid size: idm, jdm, kdm - the definition of the vertical layers target densities: sigma - frequency of outputs writing (snapshot, or time averaged): diagfq, meanfq - barotropic/baroclinic time steps: baclin, batrop. Must be reduced when going to a finer grid. Parameters to reduce also when code blows up (negative layer thickness during the vertical grid calculation). - many, many more (parameters for the z-levels, mixing, friction, diffusion etc ) - a parameter file for localizing the open boundaries, ports.input.

13 Example of running a HYCOM experiment: on Pegasus: - go to, or create a personal directory on /scratch - create a hycom directory - copy /scratch/miskand/workshop2/hycom/all into your local hycom/ - create a directory /hycom/goml copy /scratch/miskand/workshop2/hycom/goml0.04/ src_2.2.18_20_mpi and /scratch/miskand/workshop2/hycom/ GOMl0.04/config onto your local hycom/goml0.04/ - go to your local hycom/goml0.04/src_2.2.18_20_mpi - type make ARCH=intelIFC to compile the hycom executable (here on 8 procs). Needs to load module intel, then openmpi/intel. - create a directory hycom/goml0.04/expt_ copy the script /scratch/miskand/workshop2/hycom/goml0.04/ expt_01.1/script_build_link into your local hycom/goml0.04/ expt_01.1/

14 Example of running a HYCOM experiment:

15 Example of running a HYCOM experiment: - go to your local hycom/goml0.04/expt_01.1/ - run the script./script_build_link : this copies or links to the necessary files. - the file run_hycom.job contains the information for submitting a job to Pegasus. - to run hycom, type bsub x < run_hycom.job. The option -x forces the code to run on 8 processors on a single node (optimizes the communication between nodes). - check the running simulation: - bjobs - bpeek JOBID - the simulation is designed to generate 7 days of outputs. Each is in the form of archm.2009_???_12.[a,b], where??? is the number of days in the year represents the mean hours between 0 and 24 hours during which the average output (archm) is calculated. Daily snapshot would have the form archv.2009_???_00.[a,b].

16 Post-processing HYCOM outputs: visualization tools: see hycom/all/plot directory transfer of [a,b] HYCOM outputs into netcdf format: hycom/all/ archive/src/archv2ncdf3z use of other programs (Matlab, ferret ) HYCOM resources: HYCOM consortium webpage: hycom.org HYCOM manual and user guide: 63-hycom-users-manual-and-guide HYCOM for dummies: 349_HYCOM_for_Dummies.pdf