IOIPSL. The IPSL input/output library

Transcription

1 [IMAGE] IOIPSL The IPSL input/output library [IMAGE] Introduction The aim of this software project is to provide a simple but very flexible user interface for input/output and sampling operations within geophysical models. Flexibility means that for instance it should be very easy to choose the same sampling of data in the model as the one used for any in-situ observations.the final goal is to have most of these managing tasks outside of the model so the users and developers can concentrate on the physics of the model. In the design of this first version we have made a number of choices : FORTRAN 90 is the chosen programing language. The main reason is obviously to make our life simple but also because most of the models we use are in FORTRAN. netcdf is the data format. This choice should at one point only be an option as there is no reason why the interface should not be used for other formats as is done in most PCMDI software packages. The Open Source philosophy seems to us best suited for developing such software. Our main goal will always be to have a general and simple user interface and try to do all the work behind the scenes. For the moment the package contains 4 modules which should fulfill the major data management tasks in a geophysical model. Some of these packages are already well tested while other are still at an alpha level or even lower! A general introduction to the IOIPSL library was given at CERFACS on 23 Octobre The elements of IOIPSL HIST : Writes history files from a running model. It offers to the user a wide choice of sampling of the data and operations on the fly before it is written to the file. Stable version : A stable F90 version. Developments : MPP support and probably a re-write to make it more flexible (without changing the interfaces).

2 GETIN : Reads parameters from a file and configures the model for a given experiment. Stable version : A stable F90 version with a complete data base system. Developments : It should write the parameters of the run also in the restart file. FLIN : Inputs and outputs data sets from the model. Stable version : A rough version is available. Developments : It need to be refined and probably have its taks better defined. REST : Manages restart files. Stable version : A working F90 version Developments : It probably needs to be tested in more situations. Obtaining the source of IOIPSL. Installing IOIPSL. How does IOIPSL interact with the time-stepping of your model?. Examples of files produced by IOIPSL : For the moment the favored data format is netcdf using the GDT convention. Here are a few examples for headers of netcdf files produced by the HIST package : testhist1.cdl, testhist2.cdl, testhist2_bis.cdl. Where is IOIPSL used? The land-surface scheme ORCHIDEE at IPSL The ocean model OPA at IPSL The regional model PROMES at the Universidad the Complutense The GCM of the LMD. Coordination : Jan Polcher (Jan.Polcher@lmd.jussieu.fr) Marie-Alice Foujols (Marie-Alice.Foujols@ipsl.jussieu.fr)

3 Contributions IOIPSL Home Page $Date: 2001/09/24 14:51:30 $ $Revision: 1.2 $

4 HIST Introduction : This package allows to write with only a few commands data from a running model. HIST will do the sampling of the data and perform the operations needed. The storage format used is netcdf. It was designed in such a way that it is easy to use within a complex model and that it is flexible enough to be also used for debugging. The main feature are : Many different operation can be performed on the data before it is written on disk. Frequency of writing and operations can be selected by the user. This frequency can be in seconds or in month. For the last case negative values are used (more details are given in the description of the calendar description). Any number of files can be written in parallel. The software can also be used only within a parameterization of the model. It relies only on the time counter of the model. This software uses the time model of COADS. Only a zoomed part of the fields can be written to the file. HIST uses a calendar which is provided with IOIPSL Arrays which are a gathered set of points from the full 3D data can also be written. How to use HIST Three different things need to be done when implementing HIST : The file, axes and variables need to be declared : histbeg : The file is declared with a horizontal set of axes and a geographical zoom if needed. histhori & histvert : Additional horizontal axes of vertical axes can be added to the file. histdef : The variables which will be written are declared the following information are passed name of variable Title units size of the variable operation to be carried out on the variable before writing it frequency at which the operations need to be done (positive values are in units of seconds and negative values are in units of month.) frequency at which the resulting data need to be written into the file(positive values are in units of seconds and negative values are in units of month.) histend : Closing the definitions The data is given to HIST so that it does the operations or writes to the file : histwrite : the data is passed to HIST with an index table for the indexing operations. This call can come once every time step in the model as HIST will decide what needs to be done.

5 All files must be closed : histclo : to write minima and maxima for each variable and to close files. In case you fear that your program may crash or terminate abnormally you have to possibility to synchronize the buffer and the disk file at will with the function histsync. This means that you keep most of your data even if the histclo command was not executed. Some simple examples are provided in the directory example : testhist1.f90 is a simple example (in f90) which shows how to open a file and write variables. It produces a netcdf file which can then be used as a test for other applications. testhist2.f90 is another simple example (in f90) which open two files and writes some data over a longer time. Good to test the usage of the frequencies of operation and writes. Basic principals - The horizontal size (Two first indices) of the arrays to be passed to the software will be declared in histbeg. This is the size of the lon and lat arrays it is given. The horizontal structure of the zoom of data to be archived is also given to histbeg. The reason is that we want to declare the axes in the netcdf file only over the slab of data. - The vertical coordinate are declared with the subroutine histvert. It allows to include any number of axes. When the variables are defined the vertical to be used can be specified. - histdef will declare the variable, its name and units and the size of the full array in the model. This is the size of the data as it will be provided to the subroutine histwrite (Except in the case where a gathered array and the indices is given). The vertical zoom to be used and the vertical axis (from the ones defined above) that applies to the data are given. The number of bytes to store on is also passed but not yet used. Finally the operation and the frequencies are provided. - histwrite has the simplest call as we wish to keep it flexible to make it easy to move it around in the model. The variable is identified by its name, the time step is given and finally the data. A new feature has been introduced but not yet fully tested. Through a number of indices and the index field HIST write can receive the data as a gathered array. It will then expand it to the full 3D field and do the appropriate zoom if needed. Allowed operations : In histdef the user may specify for each field that is going to be archived a number of operations. 3 types may be distinguished : Vector operations, vector-scalar operations and index operations on the vector. In the description of the operations to be carried out the vector will be represented by x. For instance a gather operation will be written L "gather(x)". It must be noted that the indexing

6 operations will use as index vector the one passed to histwrite. This allows indexing to change through time (does not work yet). On top of this a time operation has to be chosen. There only one is allowed and it should be outer most operator. For instance the time averaging of the gather operation will be written as : "ave(gather(x))" Allowed time operations : ave : time average of the field inst : Instantaneous values of the field is going to be written t_min : The minimum value over the time period is going to be archived t_max : The maximum value over the time period is going to be archived l_min : The minimum value over the entire simulation is going to be archived (without time dimension) l_max : The maximum value over the entire simulation is going to be archived (without time dimension) t_sum : Sums the variable over time. once : The field is written once on the file without any time axis never : The field is never written Allowed vector operations : usage : sin(x) sin : Sinus cos : cosines tan : tangent asin : arc-sinus acos : arc-cosines atan : arc-tangent exp : exponential log : logarithm sqrt : square root chs : change sign abs : absolute value cels : Transforms the field into degrees Celsius kelv : Transforms the field into degrees Kelvin deg : puts field into degrees (from radian for instance) rad : puts field into radians ident : Identity, does nothing! Allowed vector-scalar operations : Usage : scal+x + : addition - : minus * : multiplication / : division

7 ^ : exponential min : (min(x,scal)) Minimum between x and scal max : (max(x,scal)) Maximum between x and scal Allowed indexing operations : Usage : gather(x) gather : gathers from the input data all the points listed in index and puts them onto the file. Other points in the resulting array are going to labeled as missing. This operation could be used to reduce the resolution of the history file. scatter : scatters from the input data onto the points which are listed in the index. Other points are going to be labeled as missing. This is used for instance to put a variable only available over oceans onto a global grid. coll : The same function as gather but the points which are not indexed are not altered. fill : Same function as scatter except that the points which are not indexed are left untouched only : Only the points listed in index have meaningful data and the others are changed to missing. This is useful for masking. undef : The points listed in index are set to undefined The index array has some special feature for the scatter and fill operations. We have added on top of the the standard capability of the these operation the possibility to repeat a given point. This is accomplished by using negative values in the index array. This operation is useful in cases where the model has a value for each pole which needs to be duplicated for the cylindrical projection.the negative values will not have any effect for the other operations. Thus the same index vector can be used for the scatter and the gather. Here is an example : the input vector : The index vector : The resulting vector : The first value was repeated 3 times (the index before 1 is -3) and then a normal scatter is performed. It must be noted that the repetition of the values is done after the scatter and thus if the user is not careful it can overwrite previous values. There is no test to ensure that it does not happen. Examples ave(scatter(cels(x)) : Will average the variables (temperature for instance) in celsius and scatter it onto the grid. This could be used in an ocean model where all grid points are in a vector and cover only ocean points. Thus in the history file you will have the field placed correctly on the map. ave(max(cels(x),0.)) : Average over time only the temperature (if that is the variable in Kelvin) which are above 0. t_max(x) : Computes the maximum over the time between two writes of the variable. Other examples can be tested in a bench program for the syntax analyzer. It will show you how it decomposes the string you provide. The F90 code of the bench testopp.f90 is available in the directory example. Configuration routines for HIST : ioconf_modname( model name ) : This subroutine will change the name of the model name in the netcdf file. The default value is : An IPSL model. Calendar : The calendar can be configured using the functions in the calendar module.

8 To be done : The netcdf files produced should follow the CF convention. Implementation started but some points could still be improved. GDT recommends the use of days since for the units of the time axis. The time axis should then be ot type double. Improve this documentation and write a french one. Allow variables with different grids which have different dimensions in the same file. $Date: 2001/09/24 14:51:30 $ $Revision: 1.2 $

9 GETIN Introduction : This module gives a simple mechanism to read runtime configuration files in the model. It sets up a database which can then be queried from any point within the model. Combined with the Fparser and Tk/Tcl tools a menu base configuration tool can be build automatically for the code. How to use it : The idea is the following: Each parameter in the model is identified by a keyword and provided with a default value in the model. Then a call is made to getin in order to see if this value is changed in the configuration file run.def. If this is the case then the default value is replaced. Then any later call to getin corresponding to this keyword will return exactly the same values, either the default first set or the values from the configuration file. At the end of the model the user can make a call to the subroutine "getin_dump" (it has one optional argument to replace the standard prefix of the files generated) to write out the content of the data base for all the keywords and values encountered in the code. It will signal if the value is a default or read from the configuration file. These dumped files can then be as used as they are as new configuration files. Just before the call to getin a number of comments can be written to describe the keyword. This will first help the user of the code to understand what is need but these comments can also be used by Fparser to generate the menu based configuration tool. Typically the initialization of a changeable parameter in the code would comprise the following code and comments : C Config Key = FORCING_FILE Config Desc = Name of file containing the forcing data Config Def = Cabauw.nc Config Help = This is the name of the file which should be opened Config for reading the forcing data of the dim0 model. Config The format of the file has to be netcdf and COADS Config C C C Compliant. filename= Cabauw.nc CALL getinc( FORCING_FILE,1, filename) In F90 the configuration commands are written like this :!Config Key = FORCING_FILE!Config Desc = Name of file containing the forcing data!config Def = Cabauw.nc

10 This way scalar or vector parameters can be read. It also allows for conditional parameters. This is useful if for instance a set of parameter is only needed when a certain value is set for another parameter. The code for this is taken in great part from the configuration tool of the Linux Kernel. Syntax of the run.def file The syntax of run.def is relatively simple : Empty lines and lines which have as first non blank character are considered to be comments. Each element has to be preceded by a keyword. It should be in capital letters and only the 30 first characters are significant. The value specified for the keyword can either be of type real, integer, character or logical. Arrays can be written in three different ways : 1. One keyword and a line with all elements separated by blanks. 2. The same method can also be used but separating each element by a new line. In this case you can not have an empty or commented line in the list. 3. A subscripts is added to the keyword. It is thus written as KEYWORD n. That is element n of array corresponding to keyword KEYWORD. In this case only one value can be given to the keyword. Strings with spaces can be provided as a value to a keyword if they are enclosed in quotes. You may subdivide the input over many files. Just add anywhere in the run.def (or other definition files) the keyword "INCLUDEDEF" and give as a value the name of the file you which to include. For instance : INCLUDEDEF = toto.def Here is a short example of a run.def file : This file is liked to the following files : INCLUDEDEF = toto.def A logical value DEBUG_INFO = TRUE A single string FORCING_FILE = islscp_for.nc A single real value GRAVIT = A vector of strings WORDS = here there anywhere A vector of real values

11 VECTOR= How to implement the configuration tool into the Makefile : In the main Makefile the configuration in each sub Makefile needs to be executed. Then the parser needs to be applied to all the code present in the main directory. The -source option specifies which sub-directories need to be taken into account. Finally the menus need to be build and the Tk shell executed. config : (cd ioipsl; make config) (cd sechiba; make config)./ioipsl/fparser -main -name SECHIBA_0dim -source ioipsl -source sechiba *.f./ioipsl/build_tkmenus ioipsl sechiba./kconfig.tk In each of the sub-directories the config line in the Makefile is much simpler. config :../ioipsl/fparser -name SECHIBA *.f90 echo Configuration of SECHIBA done $Date: 2001/07/06 14:12:32 $ $Revision: 1.2 $

12 FLIN Introduction In contrast to HIST this set of subroutine are designed to write and read data from netcdf files in one piece. The only slab which is allowed is along the time axis. Thus to use these routines you need to know the size of the domain covered by the data. You can read a portion of data (zooming). On the other hand the routines are made to be as convention independent as possible so that as many net CDF files as possible can be read. Currently this piece of software is still in beta version but it works. For the moment the thing it needs most is a documentation. I hope I will have to time to do it very soon. $Date: 2000/06/05 08:34:07 $ $Revision: $

13 REST Introduction This module manage the restart files and that include the following tasks : Check the presence in the file of the variable needed by the model Check that the date is correct Check that the dimensions are correct. write the variable to the restart file if it is time to do so. The restart files can be in netcdf as the IEEE format is less accurate than the representation used in netcdf for the range of values used in a numerical model. The advantage of netcdf is that the restart file is easy to manipulate and the data included can be well documented. The created files follow a simplified version of the GDT1.2 convention. $Date: 2000/06/05 08:34:08 $ $Revision: $

14 Time stepping in IOIPSL Introduction : The idea we have tried to implement in IOIPSL for the description of time is that the model only needs to know and communicate to IOIPSL the time-step at which it is and the increment between two time steps in seconds. All the rest should be handled by IOIPSL. Obviously this does not exclude the model from having its own calendar but it is not needed. Functions are available to match the IOIPSL calendar with the one of the model. To achieve this three pieces of information need to be exchanged between the model and IOIPSL : The starting date : This is the date in Julian days at which the time step counter was equal to zero. The Julian date is defined according to the calendar chosen. It is very important that this day is the one a which the time step counter was equal to zero and not to one, else all the time axes will be shifted and discontinuities may appear. Time step length : This is the time in seconds which represents the increment of the time-step counter by one. Usually there are a number of time step length and thus time step counters in a model and it does not matter too much which one is chosen but it should always be the same. The fact that the time step length is used may cause problems if one wishes to apply IOIPSL to a model with a variable time-step. Time step counter : This is an integer which is incremented as the model integrates over the period of a time step. It starts with the value zero and then increases until the end of the integration. These three pieces of information are used by the HIST and REST modules. If they are correctly defined all should go well. Example : Here is an example how the time-stepping is done and passed to IOIPSL. First of all we use two distinct time step counters. One for the current time-step (itau_c ) and another for the time-step to which the current calculations are going to lead (itau_n ). Thus the model takes at each iteration the variables from time step itau_c to itau_n which corresponds to an increment of dt in time. dt is thus the time step length. The model begins with : itau_c = 0 itau_n = 1 at the Julian day date0. This information can for instance be obtained from the restart file with a call of the type : CALL restini(..., itau_c, date0, dt,...). When the history files are defined one has to remember that the first iteration of the model starts at itau_c. As we possibly wish to archive the data produced while integrating from itau_c to itau_n the files need to be defined as follows : CALL histbeg(..., itau_c, date0, dt,...)

15 Whatever the value of itau_c, date0 will always be the Julian day at which itau_c=0. When retrieving data from the restart file the starting time step has to be used : itau_c. Thus we would write the call to restget as : CALL restget(..., itau_c,...). On the other hand it has been chosen to label the time step for the history file as the end of the current iteration. This means that all the variables computed between itau_c and itau_n will be included if it was chosen to write on disk at itau_n. Thus the call to a histwrite should have the following structure : CALL histwrite(..., itau_n,...). When the result of the iteration is written to the restart file it is also this last time step which is used. Thus we would write : CALL restput(..., itau_n,...). The time DO loop would end in this configuration with the following instructions : itau_c = itau_n itau_n = itau_n + 1. The sequencing of time-steps is relatively straight forward and should be adaptable to most models. It is just a matter of knowing where the counters start and what they mean. $Date: 2000/06/05 08:34:08 $ $Revision: $