HIRVDA(C2M) HIRVDA(MIN) HIRVDA(MDCO) HIRVDA(M2C)

Size: px
Start display at page:

Download "HIRVDA(C2M) HIRVDA(MIN) HIRVDA(MDCO) HIRVDA(M2C)"

Transcription

1 Observational data structure of the HIRLAM variational system Kristian S. Mogensen March 27, Introduction The original data structure of the HIRLAM variational code was based on ECMWF's CMA (Central Memory Array) format (ECMWF, 2000), where all observational data were packed report by report into a single 64 bit oating-point array. This format was used both for external les produced by the OBSPROC software and as the internal data structure. There have been two main problems with using CMA as the internal data structure: 1) in the ongoing development of the HIRLAM variational system, we have onseveral occasions seen that some users have had diculty in understanding the CMA format and 2) several parts of the code operate on only a limited amount of information from the observations (e.g. the fraction of land check for ships only needs to read the fraction of land and to write various ags for rejected observations) so several routines existed which extracted information from CMA, did some calculation, then stored the results in CMA. On modern computers where memory access can be the limiting factor for performance, this can degrade performance. In addition ECMWF and MeteoFrance are currently in the process of moving from CMA to ODB, so it is possible that HIRLAM will end up with a data format we have to maintain and support ourselves. This was the basis for a discussion at the 2001 HIRLAM all-sta meeting (Jarvinen et al., 2001), where we agreed on preparing for a new external data format by changing the internal structure but keeping CMA as the external le format. A proposal for a new internal structure was made shortly after (Mogensen, 2001). This newsletter describes the implemented version of the new structure with a short description of the basis for the design decisions made. 2 Typical data access patterns in the variational code An example of typical data access patterns in the variational code is as follows: do i=1,nobs norerr=obfg(i)**2/(obserr(i)**2+bgerr(i)**2) if (norerr.lt.chklim(1)) then iflag=0 98

2 elseif (norerr.lt.chklim(2)) then iflag=1 endif if (iflag.gt.1) then staflg(i)=flrejected endif enddo This chunk of code is taken from the rst guess check and illustrates that we typically only need a few variables from all observations in each step of the code. In the example we need the observation error (obserr), the rst guess error (bgerr) and the dierence between the rst guess and the observations (obfg) from the observational data in order to calculate the normalized error used for checking against specied limits. If some limits are exceeded the status ag (stag) are set to rejected. Typically only 5 to 10 variables are needed from each observation type in the calculations of the HIRLAM variational system. 3 Design of a new observational data structure Based on the discussions at the 2001 all-sta meeting the following points were drawn up: In physical les the data is often stored report by report. Atypical memory access pattern is to access a small number of values (e.g. position, time, observed value) from an observation type. It should be possible to use the original external observation le format (CMA). New or other existing external data formats should also be possible. 4D-VAR and FGAT should be taken into account in the design phase. The code should be easy to port and should be understandable for Fortran-77 programmers. The code should be designed with performance in mind. Based on these statements the following design decisions were made: Data should be stored in Fortran-90 allocatable arrays to allow dynamic memory allocation. The allocatable arrays should be collected in Fortran-90 modules in order to allow adding of new variables without changing the calling tree. By storing each variable of a report (e.g. observation status ag) in a global allocatable array for all reports of a given observation type, ecient memory access is possible both on scalar and vector machines. 99

3 The ag structure of the CMA format should be used since it is well documented. OpenMP parallelization of the observation handling should be implemented as an option. 4 Examples on observation modules In order to relate some of the design decisions to real code we willnow present some of the observation modules implemented. Each observation module is organised in a series of sections. The rst section is pure bookkeeping and contains information about how many observations currently can be stored in the module, how many observations are currently stored in the module and where each time window (for 4D-VAR and FGAT) starts and ends. The second section contains information about the station making the observation and contains things like position and observation time. For each observed variable a section follows which contains the observed value, the dierence between observed value and the background, ags etc. 4.1 Single level data The simplest observation module is for single level data such as surface data, aircraft data and satellite wind data. An example of parts of such a module are given below for SYNOP's data: module synopdata integer :: max_synop_number! Max no. of Synops integer :: synop_number! Current no. of Synops integer,allocatable :: synop_winstart(:)! Start of each time slot integer,allocatable :: synop_seqno(:)! Sequence number integer,allocatable :: synop_obsno(:)! Obs. number integer,allocatable :: synop_date(:)! Obs. date integer,allocatable :: synop_time(:)! Obs. time real,allocatable :: synop_fispres(:)! Pressure for fis obs real,allocatable :: synop_fisobs(:)! Observed fis real,allocatable :: synop_fisobfg(:)! Fgs - observed fis integer,allocatable :: synop_fisstaflag(:)! Fis status flag end module synopdata 5 Multi level data For some multi level observation types there is the complication that the number of levels for individual reports varies. An example of such observation types is radiosondes. In the present code we choose to get around this problem by using one dimensional arrays for observed values and have a start and an end pointer for each observed value for each prole. For radiosondes the structure looks like the following: 100

4 module tempdata integer :: max_temp_number! Max no. of Temps integer :: temp_number! Current no. of Temps integer,allocatable :: temp_winstart(:)! Start of each time slot integer :: max_temp_ttotalobsnumber! Max total number of T obs integer :: temp_ttotalobsnumber! Total number of T obs integer,allocatable :: temp_tstart(:)! Start of each T profile integer,allocatable :: temp_tend(:)! Start of each T profile real,allocatable :: temp_tpres(:)! Pressure for T obs real,allocatable :: temp_tobs(:)! Observed T real,allocatable :: temp_toban(:)! Analysed - observed T real,allocatable :: temp_tobfg(:)! Fgs - observed T integer,allocatable :: temp_tstaflag(:)! T status flag end module tempdata A complication with this set-up is that the start and end pointers have to be carried over into subroutine calls. An example of such a call is the call to vertical interpolation of radiosonde data which looks like the following: call vint_p_ml_mod_bg(nlev,temp_totalobsnumber, x temp_tstart,temp_tend,indext_vint_mod, x wgtt_vint_mod,t_bg,temp_tbg) where the vertical interpolation then uses the start and end pointer information when doing the vertical interpolation. An alternative would be to have a two dimensional array with dimensions given by the maximum levelsinany report and the total number of observations, but that would waste memory if large dierences between number of levels exist. 6 Data ow The data ow with the new structure is illustrated on gure 1. The observations in BUFR are converted to CMA by the OBSPROC program and read into the variational analysis code where the information is converted into the module structure described above. During the screening the modules are updated with information from the screening (SMOD on the gure). For the minimization a separate observation module for each observation type is then allocated. Originally it was planned to use only one module for each observation type, but in order to archive unit-stride memory access in the minimization a second module (CMOD in the gure) was created for each observation type which only contains observations accepted by the screening and only the information needed for the calculation of the cost function and its gradient. This module is updated with variational quality control ags and the dierences between the analysis and the observations during the minimization. After the minimization this information is stored in the full modules (now labelled AMOD) and the minimization modules deallocated. The last step is to convert the observation modules back into CMA for writing to disk and subsequent diagnostics. 101

5 FILES BUFR CMA BUFR ACMA PLOTS PROGRAM OBSPROC HIRVDA(C2M) HIRVDA(SCR) HIRVDA(MCO) HIRVDA(MIN) HIRVDA(MDCO) HIRVDA(M2C) CMASTAT DATA MOD SMOD CMOD MMOD AMOD 7 Conclusions Figure 1: Data ow with the new data structure. The new structure has been implemented and tested without diculties on many dierent computers at various HIRLAM member institutes. The performance of the new code is similar or better than the original code. The variational code currently still reads CMA as input, but any external format can be used as long as it can be converted into the format of the modules. For some new initiatives, such as Quikscat data, input data are read directly from BUFR and converted into observation modules. In general the users seem quite happy with the new structure. With the new structure we are not dependent on having the same internal stucture as the external le format and new developments such as either implementing ECMWF's ODB format or implementing direct reading of BUFR data is possible. 8 People involved in this work. The coding of the observation modules was done together with the following people: Nils Gustafsson (SMHI), Xiang-Yu Huang (DMI), Magnus Lindskog (SMHI), Frank Tveter (DNMI) and Ole Vignes (DNMI). In addition the following people to part in the dis- 102

6 cussions of the structure: Gerard Cats (KNMI), Heikki Jarvinen (FMI) and Per Unden (SMHI). References ECMWF ECMWF. IFS Documeentation. Part I: Observation Processing (CY21R4). Jarvinen, H., Mogensen, K., and Saarinen, S ASM-2001 discussion on Observation handling. HIRLAM newsletter, 189{192. Mogensen, K Proposal for a new internal observational data structure in the HIRLAM variational data assmilation code. HIRLAM newsletter, 193{

Observational DataBase (ODB*) and its usage at ECMWF

Observational DataBase (ODB*) and its usage at ECMWF Observational DataBase (ODB*) and its usage at ECMWF anne.fouilloux@ecmwf.int Slide 1 *ODB has been developed and maintained by Sami Saarinen Slide 1 Outline Observational usage over the past decades at

More information

Anne Fouilloux. Fig. 1 Use of observational data at ECMWF since CMA file structure.

Anne Fouilloux. Fig. 1 Use of observational data at ECMWF since CMA file structure. ODB (Observational Database) and its usage at ECMWF Anne Fouilloux Abstract ODB stands for Observational DataBase and has been developed at ECMWF since mid-1998 by Sami Saarinen. The main goal of ODB is

More information

Observation feedback archiving in MARS. Acknowledgement:

Observation feedback archiving in MARS. Acknowledgement: Observation feedback archiving in MARS Piotr Kuchta Data and Services Section, ECMWF P.Kuchta@ecmwf.int Acknowledgement: Slide 1 Manuel Fuentes, Baudouin Rault, Erik Andersson, Anne Fouilloux, Lars Isaksen,

More information

eccodes BUFR decoding

eccodes BUFR decoding eccodes BUFR decoding Fortran 90 and Python API part 1 Marijana Crepulja Marijana.Crepulja@ecmwf.int ECMWF March 7, 2017 Introduction: Fortran 90 subroutines to decode BUFR data Python subroutines to decode

More information

Goals for This Lecture:

Goals for This Lecture: Goals for This Lecture: Understand what modules are Understand what module procedures are and how to use them Understand explicit and implicit interfaces Understand what automatic arrays are and how to

More information

Report on the COPE technical meeting held at ECMWF, Reading 9-12, June 2014

Report on the COPE technical meeting held at ECMWF, Reading 9-12, June 2014 1 Background Report on the COPE technical meeting held at ECMWF, Reading 9-12, June 2014 Alena Trojáková The ECMWF initiated the Continuous Observation Processing Environment (COPE) project to support

More information

Programming for High Performance Computing in Modern Fortran. Bill Long, Cray Inc. 17-May-2005

Programming for High Performance Computing in Modern Fortran. Bill Long, Cray Inc. 17-May-2005 Programming for High Performance Computing in Modern Fortran Bill Long, Cray Inc. 17-May-2005 Concepts in HPC Efficient and structured layout of local data - modules and allocatable arrays Efficient operations

More information

IFS Vectorisation Improvements Using Cray Supercomputer. John Hague. Cray Consultant, Sept 2014

IFS Vectorisation Improvements Using Cray Supercomputer. John Hague. Cray Consultant, Sept 2014 IFS Vectorisation Improvements Using Cray Supercomputer John Hague. Cray Consultant, Sept 2014 1 Current System 2 clusters of approx 3500 Intel Ivybridge nodes each 24 cores/node 64 GB/node ECMWF s Integrated

More information

Cover Page. Author: Vu, Van Thieu Title: Opportunities for performance optimization of applications through code generation Issue Date:

Cover Page. Author: Vu, Van Thieu Title: Opportunities for performance optimization of applications through code generation Issue Date: Cover Page The handle http://hdl.handle.net/1887/18622 holds various files of this Leiden University dissertation. Author: Vu, Van Thieu Title: Opportunities for performance optimization of applications

More information

HARMONIE ATOVS data assimilation and coordinated impact study

HARMONIE ATOVS data assimilation and coordinated impact study HARMONIE ATOVS data assimilation and coordinated impact study ALADIN / HIRLAM 23rd Workshop / All-Staff Meeting Reykjavik, Iceland, 15-19 April, 2013 Magnus Lindskog, Mats Dahlbom, Sigurdur Thorsteinsson,

More information

III Data Structures. Dynamic sets

III Data Structures. Dynamic sets III Data Structures Elementary Data Structures Hash Tables Binary Search Trees Red-Black Trees Dynamic sets Sets are fundamental to computer science Algorithms may require several different types of operations

More information

eccodes BUFR encoding

eccodes BUFR encoding eccodes BUFR encoding Fortran 90 and Python API - part 1 Marijana Crepulja Marijana.Crepulja@ecmwf.int ECMWF February 21, 2018 Introduction: Fortran 90 subroutines to encode BUFR data Python subroutines

More information

Metview BUFR Tutorial. Meteorological Visualisation Section Operations Department ECMWF

Metview BUFR Tutorial. Meteorological Visualisation Section Operations Department ECMWF Meteorological Visualisation Section Operations Department ECMWF 05/03/2015 This tutorial was tested with Metview version 4.3.0 and some features might not work for previous versions. Copyright 2015 European

More information

Programming Languages

Programming Languages Programming Languages Tevfik Koşar Lecture - VIII February 9 th, 2006 1 Roadmap Allocation techniques Static Allocation Stack-based Allocation Heap-based Allocation Scope Rules Static Scopes Dynamic Scopes

More information

Hashing Techniques. Material based on slides by George Bebis

Hashing Techniques. Material based on slides by George Bebis Hashing Techniques Material based on slides by George Bebis https://www.cse.unr.edu/~bebis/cs477/lect/hashing.ppt The Search Problem Find items with keys matching a given search key Given an array A, containing

More information

Software Infrastructure for Data Assimilation: Object Oriented Prediction System

Software Infrastructure for Data Assimilation: Object Oriented Prediction System Software Infrastructure for Data Assimilation: Object Oriented Prediction System Yannick Trémolet ECMWF Blueprints for Next-Generation Data Assimilation Systems, Boulder, March 2016 Why OOPS? Y. Trémolet

More information

2 3. Syllabus Time Event 9:00{10:00 morning lecture 10:00{10:30 morning break 10:30{12:30 morning practical session 12:30{1:30 lunch break 1:30{2:00 a

2 3. Syllabus Time Event 9:00{10:00 morning lecture 10:00{10:30 morning break 10:30{12:30 morning practical session 12:30{1:30 lunch break 1:30{2:00 a 1 Syllabus for the Advanced 3 Day Fortran 90 Course AC Marshall cuniversity of Liverpool, 1997 Abstract The course is scheduled for 3 days. The timetable allows for two sessions a day each with a one hour

More information

Using ODB at ECMWF. Piotr Kuchta Sándor Kertész. Development Section ECMWF. Slide 1. MOS Workshop, 2013 November 18-20, ECMWF

Using ODB at ECMWF. Piotr Kuchta Sándor Kertész. Development Section ECMWF. Slide 1. MOS Workshop, 2013 November 18-20, ECMWF Using ODB at ECMWF Piotr Kuchta Sándor Kertész Development Section ECMWF Slide 1 MOS Workshop, 2013 November 18-20, ECMWF 1 History of ODB in a nutshell 1998 2008, Sami Saarinen Database of observations

More information

A problem of strong scalability?

A problem of strong scalability? Scalability of BATOR : A problem of strong scalability? Ryad El Khatib (CNRM/GMAP) Aladin Workshop / Hirlam All Staff Meeting Norrköping, 05-08 April 2011 Plan INTRODUCTION Presentation of BATOR software

More information

Instituting an observation database

Instituting an observation database Instituting an observation database capability in the NCEP GSI Tom Hamill, Jeff Whitaker, Scott Gregory NOAA / ESRL Physical Sciences Division Presentation to DAOS, Exeter England, April 2016 Our intent:

More information

Instituting an observation database (ODB) capability in the GSI

Instituting an observation database (ODB) capability in the GSI Instituting an observation database (ODB) capability in the GSI Jeff Whitaker, Scott Gregory, and Tom Hamill NOAA / ESRL Physical Sciences Division Presentation to Blueprints for Next-Generation Data Assimilation

More information

Appendix D. Fortran quick reference

Appendix D. Fortran quick reference Appendix D Fortran quick reference D.1 Fortran syntax... 315 D.2 Coarrays... 318 D.3 Fortran intrisic functions... D.4 History... 322 323 D.5 Further information... 324 Fortran 1 is the oldest high-level

More information

Cluster quality 15. Running time 0.7. Distance between estimated and true means Running time [s]

Cluster quality 15. Running time 0.7. Distance between estimated and true means Running time [s] Fast, single-pass K-means algorithms Fredrik Farnstrom Computer Science and Engineering Lund Institute of Technology, Sweden arnstrom@ucsd.edu James Lewis Computer Science and Engineering University of

More information

BUFR decoding. Dominique Lucas User Support. February Intro Bufr decoding

BUFR decoding. Dominique Lucas User Support. February Intro Bufr decoding BUFR decoding Dominique Lucas User Support February 2017 1 Content What is BUFR BUFR tools BUFR format BUFR decoding Practical examples February 2017 2 What is BUFR Binary representation of meteorological

More information

NAGWare f95 and reliable, portable programming.

NAGWare f95 and reliable, portable programming. NAGWare f95 and reliable, portable programming. Malcolm Cohen The Numerical Algorithms Group Ltd., Oxford How to detect errors using NAGWare f95, and how to write portable, reliable programs. Support for

More information

9/24/ Hash functions

9/24/ Hash functions 11.3 Hash functions A good hash function satis es (approximately) the assumption of SUH: each key is equally likely to hash to any of the slots, independently of the other keys We typically have no way

More information

dbx90: Fortran debugger March 9, 2009

dbx90: Fortran debugger March 9, 2009 dbx90: Fortran debugger March 9, 2009 1 Name dbx90 a Fortran 90/95 debugger for use with the NAG Fortran compiler. 2 Usage dbx90 [option]... executable-file 3 Description dbx90 is a Fortran 90/95 debugger

More information

OBAN Class Homework Assignment No. 4 Distributed on November 3, Due Thursday, December 1, 2016

OBAN Class Homework Assignment No. 4 Distributed on November 3, Due Thursday, December 1, 2016 OBAN Class Homework Assignment No. 4 Distributed on November 3, 2016 Due Thursday, December 1, 2016 Original document on ANALAB was written by Huang, Gustafsson and Robertson (2000), and can be found in

More information

Table-Lookup Approach for Compiling Two-Level Data-Processor Mappings in HPF Kuei-Ping Shih y, Jang-Ping Sheu y, and Chua-Huang Huang z y Department o

Table-Lookup Approach for Compiling Two-Level Data-Processor Mappings in HPF Kuei-Ping Shih y, Jang-Ping Sheu y, and Chua-Huang Huang z y Department o Table-Lookup Approach for Compiling Two-Level Data-Processor Mappings in HPF Kuei-Ping Shih y, Jang-Ping Sheu y, and Chua-Huang Huang z y Department of Computer Science and Information Engineering National

More information

Cedar Fortran Programmer's Manual 1. Jay Hoeinger. Center for Supercomputing Research and Development. Urbana, Illinois

Cedar Fortran Programmer's Manual 1. Jay Hoeinger. Center for Supercomputing Research and Development. Urbana, Illinois Cedar Fortran Programmer's Manual 1 Jay Hoeinger Center for Supercomputing Research and Development University of Illinois at Urbana-Champaign Urbana, Illinois 61801 June 14, 1993 1 This work was supported

More information

Lecture V: Introduction to parallel programming with Fortran coarrays

Lecture V: Introduction to parallel programming with Fortran coarrays Lecture V: Introduction to parallel programming with Fortran coarrays What is parallel computing? Serial computing Single processing unit (core) is used for solving a problem One task processed at a time

More information

Run Time Environment. Procedure Abstraction. The Procedure as a Control Abstraction. The Procedure as a Control Abstraction

Run Time Environment. Procedure Abstraction. The Procedure as a Control Abstraction. The Procedure as a Control Abstraction Procedure Abstraction Run Time Environment Records Procedure Linkage Name Translation and Variable Access Copyright 2010, Pedro C. Diniz, all rights reserved. Students enrolled in the Compilers class at

More information

Goals for This Lecture:

Goals for This Lecture: Goals for This Lecture: Learn about multi-dimensional (rank > 1) arrays Learn about multi-dimensional array storage Learn about the RESHAPE function Learn about allocatable arrays & the ALLOCATE and DEALLOCATE

More information

Compiler and Runtime Support for Programming in Adaptive. Parallel Environments 1. Guy Edjlali, Gagan Agrawal, and Joel Saltz

Compiler and Runtime Support for Programming in Adaptive. Parallel Environments 1. Guy Edjlali, Gagan Agrawal, and Joel Saltz Compiler and Runtime Support for Programming in Adaptive Parallel Environments 1 Guy Edjlali, Gagan Agrawal, Alan Sussman, Jim Humphries, and Joel Saltz UMIACS and Dept. of Computer Science University

More information

Metview and Python - what they can do for each other

Metview and Python - what they can do for each other Metview and Python - what they can do for each other Workshop on Python for Earth System Sciences, ECMWF Iain Russell, Fernando Ii, Sándor Kertész, Stephan Siemen Development Section, ECMWF ECMWF November

More information

Progress in the assimilation of GPS-RO at ECMWF

Progress in the assimilation of GPS-RO at ECMWF Progress in the assimilation of GPS-RO at ECMWF Sean Healy Mike Rennie, Milan Dragosavac, Paul Poli, Carla Cardinali, Peter Bauer. Impact of GPS-RO The impact of GPS-RO NWP and climate reanalysis applications

More information

Chapter 4. Fortran Arrays

Chapter 4. Fortran Arrays Chapter 4. Fortran Arrays Fortran arrays are any object with the dimension attribute. In Fortran 90/95, and in HPF, arrays may be very different from arrays in older versions of Fortran. Arrays can have

More information

Introduction to Metview

Introduction to Metview Introduction to Metview Fernando Ii, Iain Russell, Sándor Slide Kertész 1 Development Section - ECMWF 1 What is Metview? Retrieve/manipulate/visualise meteorological data Working environment for operational

More information

Bits, Bytes, and Precision

Bits, Bytes, and Precision Bits, Bytes, and Precision Bit: Smallest amount of information in a computer. Binary: A bit holds either a 0 or 1. Series of bits make up a number. Byte: 8 bits. Single precision variable: 4 bytes (32

More information

Accelerating Harmonie with GPUs (or MICs)

Accelerating Harmonie with GPUs (or MICs) Accelerating Harmonie with GPUs (or MICs) (A view from the starting-point) Enda O Brien, Adam Ralph Irish Centre for High-End Computing Motivation There is constant, insatiable demand for more performance

More information

Computer Architectures and Aspects of NWP models

Computer Architectures and Aspects of NWP models Computer Architectures and Aspects of NWP models Deborah Salmond ECMWF Shinfield Park, Reading, UK Abstract: This paper describes the supercomputers used in operational NWP together with the computer aspects

More information

The WMO table driven codes: The 21 st century universal observation codes

The WMO table driven codes: The 21 st century universal observation codes The WMO table driven codes: The 21 st century universal observation codes Luis Kornblueh Max-Planck-Institut für Meteorologie p.1/11 Introduction Unified meteorological observation data handling: BUFR

More information

Dictionaries and Hash Tables

Dictionaries and Hash Tables Dictionaries and Hash Tables Nicholas Mainardi Dipartimento di Elettronica e Informazione Politecnico di Milano nicholas.mainardi@polimi.it 14th June 2017 Dictionaries What is a dictionary? A dictionary

More information

CS 536 Introduction to Programming Languages and Compilers Charles N. Fischer Lecture 11

CS 536 Introduction to Programming Languages and Compilers Charles N. Fischer Lecture 11 CS 536 Introduction to Programming Languages and Compilers Charles N. Fischer Lecture 11 CS 536 Spring 2015 1 Handling Overloaded Declarations Two approaches are popular: 1. Create a single symbol table

More information

Worst-case running time for RANDOMIZED-SELECT

Worst-case running time for RANDOMIZED-SELECT Worst-case running time for RANDOMIZED-SELECT is ), even to nd the minimum The algorithm has a linear expected running time, though, and because it is randomized, no particular input elicits the worst-case

More information

INLINE EXPANSION FOR THE POLARIS RESEARCH COMPILER JOHN ROBERT GROUT. B.S., Worcester Polytechnic Institute, 1981 THESIS

INLINE EXPANSION FOR THE POLARIS RESEARCH COMPILER JOHN ROBERT GROUT. B.S., Worcester Polytechnic Institute, 1981 THESIS INLINE EXPANSION FOR THE POLARIS RESEARCH COMPILER BY JOHN ROBERT GROUT B.S., Worcester Polytechnic Institute, 1981 THESIS Submitted in partial fulllment of the requirements for the degree of Master of

More information

Technical Specification on further interoperability with C

Technical Specification on further interoperability with C Technical Specification on further interoperability with C John Reid, ISO Fortran Convener Fortran 2003 (or 2008) provides for interoperability of procedures with nonoptional arguments that are scalars,

More information

Empirical transfer function determination by. BP 100, Universit de PARIS 6

Empirical transfer function determination by. BP 100, Universit de PARIS 6 Empirical transfer function determination by the use of Multilayer Perceptron F. Badran b, M. Crepon a, C. Mejia a, S. Thiria a and N. Tran a a Laboratoire d'oc anographie Dynamique et de Climatologie

More information

AAPP status report and preparations for processing METOP data

AAPP status report and preparations for processing METOP data AAPP status report and preparations for processing METOP data Nigel C Atkinson *, Pascal Brunel, Philippe Marguinaud and Tiphaine Labrot * Met Office, Exeter, UK Météo-France, Centre de Météorologie Spatiale,

More information

Programming with MPI

Programming with MPI Programming with MPI p. 1/?? Programming with MPI Composite Types and Language Standards Nick Maclaren Computing Service nmm1@cam.ac.uk, ext. 34761 March 2008 Programming with MPI p. 2/?? Composite Types

More information

perform. If more storage is required, more can be added without having to modify the processor (provided that the extra memory is still addressable).

perform. If more storage is required, more can be added without having to modify the processor (provided that the extra memory is still addressable). How to Make Zuse's Z3 a Universal Computer Raul Rojas January 14, 1998 Abstract The computing machine Z3, built by Konrad Zuse between 1938 and 1941, could only execute xed sequences of oating-point arithmetical

More information

Fortran. (FORmula TRANslator) History

Fortran. (FORmula TRANslator) History Fortran (FORmula TRANslator) History FORTRAN vs. Fortran 1954 FORTRAN first successful high level language John Backus (IBM) 1958 FORTRAN II (Logical IF, subroutines, functions) 1961 FORTRAN IV 1966 FORTRAN

More information

Optimisation p.1/22. Optimisation

Optimisation p.1/22. Optimisation Performance Tuning Optimisation p.1/22 Optimisation Optimisation p.2/22 Constant Elimination do i=1,n a(i) = 2*b*c(i) enddo What is wrong with this loop? Compilers can move simple instances of constant

More information

Status of OPLACE system

Status of OPLACE system Status of OPLACE system Alena Trojáková OPLACE system The common LACE observation pre-processing system (OPLACE) - aimed to support DA implementation, avoid duplication of work on observation pre-processing

More information

InterSci. Version 2.1. Scilab Group. May All Scilab primitive functions are dened in a set of interface routines. For each function the

InterSci. Version 2.1. Scilab Group. May All Scilab primitive functions are dened in a set of interface routines. For each function the InterSci Version 21 Scilab Group May 1993 1 Introduction All Scilab primitive functions are dened in a set of interface routines For each function the interfacing code checks rst number of rhs and lhs

More information

Frama-C's metrics plug-in

Frama-C's metrics plug-in Metrics Frama-C's metrics plug-in 20120901 (Oxygen) Richard Bonichon & Boris Yakobowski CEA LIST, Software Reliability Laboratory, Saclay, F-91191 c 2011 CEA LIST CONTENTS Contents 1 Quick overview 7

More information

Analysis Methods in Atmospheric and Oceanic Science

Analysis Methods in Atmospheric and Oceanic Science Analysis Methods in Atmospheric and Oceanic Science AOSC 652 Getting to know FORTRAN: Input/Output, Data Sorting, Simple Statistics Day 2 21 Sep 2016 1 Review Mon: Call to piksrt in our code: call piksrt(iarray_in,iarray_out,npts)

More information

lisp APL Algol Fortran like like like like A+ Delphi C J Algol60 Pascal Scheme S-algol Ada Java ML Hi Haskell Fixed rank mutable array size

lisp APL Algol Fortran like like like like A+ Delphi C J Algol60 Pascal Scheme S-algol Ada Java ML Hi Haskell Fixed rank mutable array size The nature of vector types Languages can be categorized by the degree of dynamism associated with their vector types and the implications this has for how vector operations are done. Continuum Most dynamic

More information

ECMWF contribution to the SMOS mission

ECMWF contribution to the SMOS mission contribution to the SMOS mission J. Muñoz Sabater, P. de Rosnay, M. Drusch & G. Balsamo Monitoring Assimilation Remote Sensing and Modeling of Surface Properties 09-11 June 2009 slide 1 Outline Global

More information

Technical Report on further interoperability with C

Technical Report on further interoperability with C Technical Report on further interoperability with C John Reid, ISO Fortran Convener, JKR Associates and Rutherford Appleton Laboratory Fortran 2003 (or 2008) provides for interoperability of procedures

More information

Write an iterative real-space Poisson solver in Python/C

Write an iterative real-space Poisson solver in Python/C Write an iterative real-space Poisson solver in Python/C Ask Hjorth Larsen asklarsen@gmail.com October 10, 2018 The Poisson equation is 2 φ(r) = ρ(r). (1) This is a second-order linear dierential equation

More information

Using Flex 3 in a Flex 4 World *

Using Flex 3 in a Flex 4 World * OpenStax-CNX module: m34631 1 Using Flex 3 in a Flex 4 World * R.G. (Dick) Baldwin This work is produced by OpenStax-CNX and licensed under the Creative Commons Attribution License 3.0 Abstract Learn how

More information

AN INTRODUCTION TO FORTRAN 90 LECTURE 2. Consider the following system of linear equations: a x + a x + a x = b

AN INTRODUCTION TO FORTRAN 90 LECTURE 2. Consider the following system of linear equations: a x + a x + a x = b AN INTRODUCTION TO FORTRAN 90 LECTURE 2 1. Fortran 90. Arrays, functions and subroutines. 2. Scientific plotting. Gnuplot 1 Each coefficient and variable is a scalar. Lengthy and cumbersome! Program Scalar

More information

Run Time Environment. Activation Records Procedure Linkage Name Translation and Variable Access

Run Time Environment. Activation Records Procedure Linkage Name Translation and Variable Access Run Time Environment Activation Records Procedure Linkage Name Translation and Variable Access Copyright 2015, Pedro C. Diniz, all rights reserved. Students enrolled in the Compilers class at the University

More information

Interpolation of gridded data using Geostrophic Balance. Lennon O Naraigh *

Interpolation of gridded data using Geostrophic Balance. Lennon O Naraigh * Interpolation of gridded data using Geostrophic Balance Lennon O Naraigh * Date: 6 th September 2004 Abstract: In this report, a method of interpolating data from the grid of a global climate model (GCM)

More information

Convolutional Neural Networks for Object Classication in CUDA

Convolutional Neural Networks for Object Classication in CUDA Convolutional Neural Networks for Object Classication in CUDA Alex Krizhevsky (kriz@cs.toronto.edu) April 16, 2009 1 Introduction Here I will present my implementation of a simple convolutional neural

More information

Introduction. 1 Measuring time. How large is the TLB? 1.1 process or wall time. 1.2 the test rig. Johan Montelius. September 20, 2018

Introduction. 1 Measuring time. How large is the TLB? 1.1 process or wall time. 1.2 the test rig. Johan Montelius. September 20, 2018 How large is the TLB? Johan Montelius September 20, 2018 Introduction The Translation Lookaside Buer, TLB, is a cache of page table entries that are used in virtual to physical address translation. Since

More information

An Introduction to Fortran

An Introduction to Fortran An Introduction to Fortran Sylvia Plöckinger March 10, 2011 Sylvia Plöckinger () An Introduction to Fortran March 10, 2011 1 / 43 General Information Find this file on: http://homepage.univie.ac.at/nigel.mitchell/numprac/

More information

PAW: Physicist Analysis Workstation

PAW: Physicist Analysis Workstation PAW: Physicist Analysis Workstation What is PAW? A tool to display and manipulate data. Learning PAW See ref. in your induction week notes. Running PAW: 2 Versions:- PAW: 2 windows: A terminal window for

More information

From Integrated to Object-Oriented

From Integrated to Object-Oriented From Integrated to Object-Oriented Yannick Trémolet and Mike Fisher ECMWF 3 October 2012 Thanks to many people who have contributed to the project: Tomas Wilhelmsson, Deborah Salmond, John Hague, George

More information

Algorithmic "imperative" language

Algorithmic imperative language Algorithmic "imperative" language Undergraduate years Epita November 2014 The aim of this document is to introduce breiy the "imperative algorithmic" language used in the courses and tutorials during the

More information

HPF commands specify which processor gets which part of the data. Concurrency is defined by HPF commands based on Fortran90

HPF commands specify which processor gets which part of the data. Concurrency is defined by HPF commands based on Fortran90 149 Fortran and HPF 6.2 Concept High Performance Fortran 6.2 Concept Fortran90 extension SPMD (Single Program Multiple Data) model each process operates with its own part of data HPF commands specify which

More information

Metview 4 ECMWF s latest generation meteorological workstation

Metview 4 ECMWF s latest generation meteorological workstation Metview 4 ECMWF s latest generation meteorological workstation Iain Russell, Stephan Siemen, Fernando Ii, Sándor Kertész, Sylvie Lamy-Thépaut, Vesa Karhila Version 4 builds on the flexible and proven modular

More information

University of Ghent. St.-Pietersnieuwstraat 41. Abstract. Sucient and precise semantic information is essential to interactive

University of Ghent. St.-Pietersnieuwstraat 41. Abstract. Sucient and precise semantic information is essential to interactive Visualizing the Iteration Space in PEFPT? Qi Wang, Yu Yijun and Erik D'Hollander University of Ghent Dept. of Electrical Engineering St.-Pietersnieuwstraat 41 B-9000 Ghent wang@elis.rug.ac.be Tel: +32-9-264.33.75

More information

Sparse Matrix Libraries in C++ for High Performance. Architectures. ferent sparse matrix data formats in order to best

Sparse Matrix Libraries in C++ for High Performance. Architectures. ferent sparse matrix data formats in order to best Sparse Matrix Libraries in C++ for High Performance Architectures Jack Dongarra xz, Andrew Lumsdaine, Xinhui Niu Roldan Pozo z, Karin Remington x x Oak Ridge National Laboratory z University oftennessee

More information

Bitangent 3. Bitangent 1. dist = max Region A. Region B. Bitangent 2. Bitangent 4

Bitangent 3. Bitangent 1. dist = max Region A. Region B. Bitangent 2. Bitangent 4 Ecient pictograph detection Dietrich Buesching TU Muenchen, Fakultaet fuer Informatik FG Bildverstehen 81667 Munich, Germany email: bueschin@informatik.tu-muenchen.de 1 Introduction Pictographs are ubiquitous

More information

Introduction to Programming with Fortran 90

Introduction to Programming with Fortran 90 Introduction to Programming with Fortran 90 p. 1/?? Introduction to Programming with Fortran 90 Array Concepts Nick Maclaren Computing Service nmm1@cam.ac.uk, ext. 34761 November 2007 Introduction to Programming

More information

An introduction into numerical optimization with KNITRO

An introduction into numerical optimization with KNITRO An introduction into numerical optimization with KNITRO Pawel Doligalski and Dominik Thaler 15 September 2014 KNITRO fval fcount time fmincon -103.6194 2197 1.578750 knitro a'la fmincon -103.1450 144 0.094221

More information

Paralleliza(on Challenges for Ensemble Data Assimila(on

Paralleliza(on Challenges for Ensemble Data Assimila(on Paralleliza(on Challenges for Ensemble Data Assimila(on Helen Kershaw Institute for Mathematics Applied to Geophysics, National Center for Atmospheric Research Email: hkershaw@ucar.edu What am I going

More information

Programming with MPI

Programming with MPI Programming with MPI p. 1/?? Programming with MPI Miscellaneous Guidelines Nick Maclaren Computing Service nmm1@cam.ac.uk, ext. 34761 March 2010 Programming with MPI p. 2/?? Summary This is a miscellaneous

More information

Annex A (Informative) Collected syntax The nonterminal symbols pointer-type, program, signed-number, simple-type, special-symbol, and structured-type

Annex A (Informative) Collected syntax The nonterminal symbols pointer-type, program, signed-number, simple-type, special-symbol, and structured-type Pascal ISO 7185:1990 This online copy of the unextended Pascal standard is provided only as an aid to standardization. In the case of dierences between this online version and the printed version, the

More information

G. Colin de Verdière CEA

G. Colin de Verdière CEA A methodology to port production codes to GPU learned from our experiments G. Colin de Verdière CEA GCdV CEA, DAM, DIF, F-91297 Arpajon 1 Moving towards GPGPU Potentially important speedups COMPLEX(8)

More information

A Brief Introduction to Fortran of 15

A Brief Introduction to Fortran of 15 A Brief Introduction to Fortran 90 1 of 15 Data Types and Kinds Data types Intrisic data types (INTEGER, REAL,LOGICAL) derived data types ( structures or records in other languages) kind parameter (or

More information

Fortran 2008: what s in it for high-performance computing

Fortran 2008: what s in it for high-performance computing Fortran 2008: what s in it for high-performance computing John Reid, ISO Fortran Convener, JKR Associates and Rutherford Appleton Laboratory Fortran 2008 has been completed and is about to be published.

More information

Hashing. Hashing Procedures

Hashing. Hashing Procedures Hashing Hashing Procedures Let us denote the set of all possible key values (i.e., the universe of keys) used in a dictionary application by U. Suppose an application requires a dictionary in which elements

More information

Metview 5.0 and Beyond, to its Pythonic Future

Metview 5.0 and Beyond, to its Pythonic Future Metview 5.0 and Beyond, to its Pythonic Future MOS 2017, ECMWF Iain Russell, Fernando Ii, Sándor Kertész, Stephan Siemen Development Section, ECMWF ECMWF March 02, 2017 What is Metview? Workstation software

More information

Meteorology and Python

Meteorology and Python Meteorology and Python desperately trying to forget technical details Claude Gibert, Europython 2011 Background Meteorology - NWP Numerical Weather Prediction ECMWF European Centre for Medium-Range Weather

More information

Metview Introduction

Metview Introduction Metview Introduction Fernando Ii Slide 1 Meteorological Visualisation Section ECMWF Metview - Introduction June 2013 1 Outline Introduction Interactive usage demo Slide 2 Metview - Introduction June 2013

More information

Comp 335 File Structures. Hashing

Comp 335 File Structures. Hashing Comp 335 File Structures Hashing What is Hashing? A process used with record files that will try to achieve O(1) (i.e. constant) access to a record s location in the file. An algorithm, called a hash function

More information

Information technology Programming languages Fortran Part 1: Base language

Information technology Programming languages Fortran Part 1: Base language INTERNATIONAL STANDARD ISO/IEC 1539-1:2010 TECHNICAL CORRIGENDUM 2 Published 2013-06-01 INTERNATIONAL ORGANIZATION FOR STANDARDIZATION МЕЖДУНАРОДНАЯ ОРГАНИЗАЦИЯ ПО СТАНДАРТИЗАЦИИ ORGANISATION INTERNATIONALE

More information

Metview s new Python interface

Metview s new Python interface Metview s new Python interface Workshop on developing Python frameworks for earth system sciences. ECMWF, 2018 Iain Russell Development Section, ECMWF Thanks to Sándor Kertész Fernando Ii Stephan Siemen

More information

Scientific Programming in C VI. Common errors

Scientific Programming in C VI. Common errors Scientific Programming in C VI. Common errors Susi Lehtola 6 November 2012 Beginner errors If you re a beginning C programmer, you might often make off-by one errors when you use arrays: #i n c l u de

More information

Verification of Fortran Codes

Verification of Fortran Codes Verification of Fortran Codes Wadud Miah (wadud.miah@nag.co.uk) Numerical Algorithms Group http://www.nag.co.uk/content/fortran-modernization-workshop Fortran Compilers Compilers seem to be either high

More information

NAG Library Routine Document C05QBF.1

NAG Library Routine Document C05QBF.1 NAG Library Routine Document Note: before using this routine, please read the Users Note for your implementation to check the interpretation of bold italicised terms and other implementation-dependent

More information

Let the dynamic table support the operations TABLE-INSERT and TABLE-DELETE It is convenient to use the load factor ( )

Let the dynamic table support the operations TABLE-INSERT and TABLE-DELETE It is convenient to use the load factor ( ) 17.4 Dynamic tables Let us now study the problem of dynamically expanding and contracting a table We show that the amortized cost of insertion/ deletion is only (1) Though the actual cost of an operation

More information

Memory. From Chapter 3 of High Performance Computing. c R. Leduc

Memory. From Chapter 3 of High Performance Computing. c R. Leduc Memory From Chapter 3 of High Performance Computing c 2002-2004 R. Leduc Memory Even if CPU is infinitely fast, still need to read/write data to memory. Speed of memory increasing much slower than processor

More information

(Refer Slide Time 01:41 min)

(Refer Slide Time 01:41 min) Programming and Data Structure Dr. P.P.Chakraborty Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture # 03 C Programming - II We shall continue our study of

More information

GEMS: Global Earth-system Monitoring using Satellite & in-situ Data

GEMS: Global Earth-system Monitoring using Satellite & in-situ Data GEMS: Global Earth-system Monitoring using Space and in-situ data Introduction to GEMS 2006 Assembly GMES Integrated Project, 12.5MEuro, 30 Institutes, 14 Countries www.ecmwf.int/research/eu_projects/gems

More information

ISO/IEC : TECHNICAL CORRIGENDUM 2

ISO/IEC : TECHNICAL CORRIGENDUM 2 ISO/IEC 1539-1:2010 - TECHNICAL CORRIGENDUM 2 ISO/IEC/JTC1/SC22/WG5-N1957 Notes for WG5: Edits included in this document from the interpretations in N1932 as amended by 12-193 and 12-194 and in N1949 as

More information

AAL 217: DATA STRUCTURES

AAL 217: DATA STRUCTURES Chapter # 4: Hashing AAL 217: DATA STRUCTURES The implementation of hash tables is frequently called hashing. Hashing is a technique used for performing insertions, deletions, and finds in constant average

More information