Preparation. Login to mirage[0,1,2].ucar.edu with X terminal. siliu% > ssh X

Transcription

1 Preparation Login to mirage[0,1,2].ucar.edu with X terminal siliu% > ssh X username@mirage1.ucar.edu Run an X application o mirage1> xclock o mirage1> xeyes Copy the examples to your directory mirage1>scp r /glade/home/siliu/ncl mydir

2 Introduction to NCL Si Liu Consulting Services Group NCAR/CISL/OSD/USS May 25, 2012

3 Outline Introduction to NCL NCL basic syntax Data processing File input and output NCL Graphics (visualization) Learn more in the future How to get help

4 Introduction to NCL

5 What is NCL NCAR Command Language (NCL) a product of the CISL at NCAR and sponsored by the NSF A free interpreted language designed specifically for scientific data processing and visualization Base on netcdf variable model

6 NCL The power and utility of the language are evident in three main areas: Data analysis File input and output Visualization

7 NCL features Features common to modern programming languages: types, variables, operators, expressions, conditional statements, loops, and functions Support array operations and procedures Manipulation of metadata Configuration of the visualizations Import of data from a variety of data formats

8 Download and installation A one-time registration is needed at Earth System Grid website. Download binaries or source code for various UNIX systems on Earth System Grid website. Install the compiled binaries recommended way Build from the source code not trivial on some platforms Available on CISL machines (Bluefire, Mirage)

9 Run NCL on mirage Logon to mirage1.ucar.edu with X terminal siliu% > ssh X username@mirage1.ucar.edu Run NCL in interactive mode or batch mode o Interactive Mode mirage1:siliu% >ncl ncl > ncl command here o Batch Mode ncl my_script.ncl

10 Variable model NCL is based on netcdf variable model Values: Scalar or array Meta data (useful in NCL) Attributes Coordinate variables _FillValue attribute (missing_value)

11 What is netcdf? NetCDF (Network Common Data Form) Developed at the Unidata Program Center A set of interfaces for array-oriented data access A freely-distributed collection of data access libraries Supported by many libraries and languages o C/C++, Fortran, Java, o NCL, IDL, Matlab, Ferret GrADS o NCO, CDO

12 netcdf features Self-Describing Portable Scalable Appendable Sharable Archivable

13 netcdf Coordinate Variable Single-dimension arrays. The dimension must be named. Dimension name is the same as the variable name. Monotonically increasing or decreasing. Example: time, longitude, latitude, etc.

14 Accessing netcdf file ncdump(unidata utility) o ncdump data_file o ncdump h data_file ncview ncl_filedump ncl_convert2nc

15 Example mirage1:siliu% > ncdump h input.nc mirage1:siliu% > nvview input.nc mirage1:siliu% > ncl_filedump input.nc mirage1:siliu% > ncl_convert2nc hdfdata.hdf Start ncl mirage1:siliu% >ncl ncl > file1=addfile("input.nc","r") ;Open data file ncl >TS=file1->TS ;import variables via addfile function ncl >printvarsummary(ts) ;Mot useful command!

16 NCL syntax

17 Data Types Basic numeric types float, double int64, uint64, Integer, uint long, ulong short, ushortr byte, ubyte Non-numeric types string character graphic file logical list More details about: NCL data types

18 Logical Expressions Greater than Greater than or equal to Less than Less than or equal to Equal to Not equal to And Or Not Exclusive or.gt..ge..lt..le..eq..ne..and..or..not..xor.

19 Algebraic Operators Plus + Work for string concatenation Minus and Negation - Multiply * Divide / Exponentiation ^ Modulus % Matrix Multiply # Greater than selection (clipping) > T=T>E where (T.lt.E) T=E Less than selection (clipping) < T=T<E where(t.gt.e) T=E

20 Syntax symbols ; comment -> use to (im/ex)port variables via addfile reference/create attributes! reference/create named dimension & reference/create coordinate variable { } coordinate subscripting $ enclose strings when import/export variables via addfile (/ /) array construction (variable); remove meta data [/ /] list construction; [:] all elements of a list : array syntax separator for named dimensions \ continue character :: syntax for external shared objects (fortran, C, etc.)

21 Create and Assign Coordinate Variables ; create 1D array: time, lon, lat time = (/ 2006,2007,2008,2009,2010, 2011/) lon = ispan(0, 360, 5) lat = ispan(-180, 180, 10) ; create 3D variable Q with named dimensions ;! reference/create named dimension Q!0 = time Q!1 = lon Q!2 = lat ;Assign coordinate variables to x ; & reference/create coordinate variable Q&time = time Q&lon = lon Q&lat = lat ;Assign value to Q Q=.

22 Array 0-based, row major o left dimension varies slowest; right varies fastest o dimension numbering left to right [0,1,..] Whole array operators o similar to f90/f95 Subscripts o Q(0,100,100) o Q(3:4, {180:360:10}, 30:50:5 )

23 Array example I_integer = (/1,2,3,4,5,6,7,8,9/) F_float = (/ 2.3, 4.3, 2.3, 4.3, 4.3 /) S_string = (/ Tom, Steve, Peter /) M_matrix = (/ (/1,2,3/), (/4,5,6/), (/7,8,9/) /) pretty-print 2D array (table) to standard out write_matrix (M_matrix, I5, Option)

24 Control Structure If-then-end if Do loop if (a.eq.b) then statements else statements end if Do while (N.gt.0) statements end do break and continue

25 Functions and Procedures Many Powerful Built-in functions and procedures o all, any, conform, ind, ind_resolve, dimsizes o fspan, ispan o ndtooned, onedtond, o mask, ismissing, where o system, systemfunc o dim_avg_n dimsizes User defined functions and procedures o Similar to Fortran and C

26 Meta data Read values only and _FillValue a = (/ file1->a /) Read variable and all meta data a = file1->a Computation causes loss of meta data b = a+5 Built-in functions cause loss of meta data Tavg = dim_avg_n(t, 0) Variable to variable transfer b = a b = b+5 Use wrapper functions Tavg = dim_avg_n_wrap(t, 0)

27 NCL file I/O

28 NCL File Input/Output Supported formats o netcdf (network Common Data Form) o HDF4/H5 (Hierarchical Data Format) o HDF-EOS (Earth Observing System) o GRIB-1/2 (Grid in Binary) o More Binary and ASCII o Sequential, direct

29 Reading and Writing Binary/ASCII data fbinrecread: reads multiple unformatted sequential records fbinnumrec: returns the number of unformatted sequential records fbindirread: reads specified record from a Fortran direct access file fbinread: same as fbinrecread but reads only one ieee rec fbinrecwrite: write unformatted fortran sequential recs fbindirwrite: write specified record; fortran direct access fbinwrite: write a binary file containing a single record

30 File I/O f = addfile (file.ext, status ) o fin = addfile ( my_input_file.nc", "r") o fout = addfile ( my_output_file.nc", "c") o fio = addfile ( /home/myncl/result.nc", "w") io_ex1.ncl file1=addfile("input.nc","r") ;open data file TS=file1->TS fout = addfile ( output.nc", c") fout@title = Output Example" fout->q = TS ;import variables via addfile function

31 Efficient netcdf Creation NCL functions to predefine a netcdf/hdf file: ; create global attributes, specify entering define mode fileatt = True fileattdef (fout, fileatt) ; predefine coordinate variables dimnames = (/ " time", "lat", "lon"/) dimsizes = (/ -1, nlat, mlon/) ; -1 means unknown dimunlim = (/ True, False, False/) filedimdef (fout, dimnames, dimsizes, dimunlim) ; predefine variable names, type, and dimensions filevardef (fout, "time", typeof(time), getvardims(time)) filevardef (fout, "lat", typeof(lat), getvardims(lat) ) filevardef (fout, "lon", typeof(lon), getvardims(lon) ) filevardef (fout, "Q", typeof(ts), getvardims( TS) ) ; create var attributes for each variable filevarattdef (fout, "Q ", TS) ; output data values only [use (/ /) to strip meta data] fout->time = (/ time /) fout->lat = (/ lat /) fout->lon = (/ lon /) fout->q = (/ TS /)

32 What else can NCL do Functions and procedures Command line options Using external codes (f77,f90, C) NCL as a scripting language More

33 NCL Graphics

34 Graphics interfaces Over 40 plotting interfaces Contours plot(over maps) Maps Streamlines plot(over maps) Primitives Vectors plot (over maps) XY plots Many highly specialized Polygon Histograms Bar charts Pie chart Skew-T Table Wind roses Taylor diagrams Plenty of application examples

35

36

37

38 CCSM conventions recognized by gsn_csm scripts _FillValue attribute recognized as missing value ( missing_value is NOT) Data attributes such as long_name and units may be used for plot titles Coordinate arrays used for axes values If data has 1D coordinate arrays and plotting over a map, then units attribute of degrees_east or degrees_north expected

39 NCL script structure Load all necessary libraries Data input Data processing Open a workstation Create/choose a proper color table Create/set resources Realize the visualization Date output

40 xy_ex1.ncl load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl" load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl" x = ispan(0,100,1) ; 201 points (-100,-99,...,99,100) y = cos(0.0628*x) wks = gsn_open_wks("x11","xy_ex1") res = False plot = gsn_csm_xy(wks,x,y,res) ; Call the gsn_csm function for drawing the curve.

41 Libraries Load the following two libraries in order. $NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl $NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl gsn_code.ncl: generic interfaces and supplemental routines gsn_csm.ncl: interfaces that look for CCSM conventions Loading order is important!

42 Color Table Set the color map before drawing to the frame. If you use the same color map a lot, can put in.hluresfile file (show it later) Can use one of the other 130+ color maps, or create your own.

43 example: coloratable1.ncl load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl" begin wks = gsn_open_wks( x11","color") ;Open a works station (xwindow) ;wks = gsn_open_wks("ps","color) ; Open color.ps PostScript file ;gsn_define_colormap(wks,"rainbow") ; Change color map gsn_draw_colormap(wks) ; Draw color map. end

44 example: coloratable2.ncl colortable2.ncl ; Define a color map using named colors. cmap = (/"PeachPuff","BlueViolet","Burlywood3","Chartreuse","HotPink",\ "LightSteelBlue","Peru","Gray25","Khaki","Black","Yellow"/) coloratable3.ncl ; Change the color map using RGB triplets. cmap = (/(/0.,0.,0./), (/1.,1.,1./), \ (/1.,0.,0./), (/0.,1.,0./), \ (/0.,0.,1./), (/1.,0.,1./), \ (/1.,1.,0./), (/0.,1.,1./), \ (/.2,.2,.2/), (/.8,.8,.8/)/) gsn_define_colormap(wks,cmap) ; Set this new color map.

45 Resource types am annotation manager lb label bar tm tickmark app app lg legends tr transformation ca coordinate array mp maps tx text cn contour pm plot manager vc vectors ct coordinate array table pr primitives vf vector field dc data comm sf scalar field cp view port err error st streamline wk workstation gs graphic style tf transform ws workspace gsn gsn high-level interfaces ti title xy xy plots

46 xy example2 (demo) x = ispan(-100,100) ; 201 points (-100,99,...,99,100) y = cos(0.0628*x) wks = gsn_open_wks("x11","xy_ex1") res = True ; Create a variable to hold plot resources. res@xylinecolor = "Red" ; Line color (case insensitive) res@xylinethicknessf = 2 ; 2x as thick as default res@xydashpattern = 0 ; Line dash pattern 0 is default (solid) res@timainstring = "This is xy example 1" ; Main title res@tixaxisstring = "X Values" ; X axis title res@tiyaxisstring = "Y Values" ; Y axis title res@tryminf = min(y) ; Set axes limits for Y axis res@trymaxf = max(y) res@trxminf = min(x) ; Set axes limits for Y axis res@trxmaxf = max(x) plot = gsn_csm_xy(wks,x,y,res)

47 Contour example (demo) tf = addfile("meccatemp.cdf","r") T = tf->t(0,:,:) ; T will have metadata attached. wks = gsn_open_wks("x11","contour2d") gsn_define_colormap(wks,"rainbow") ; Change color map res = True ; Set some plot options res@gsnaddcyclic = False ; Don't add a cyclic point res@cnfillon = True ; Turn on contour fill res@cnlineson = False ; Turn off contour lines res@gsnspreadcolors = True ; Span full color map plot = gsn_csm_contour_map(wks,t,res) ; Call the gsn_csm function for

48 Vector example(demo) a = addfile("atmos.nc","r") ; Read in first time step u = a->u(0,15,:,:) ; and 15th level of v = a->v(0,15,:,:) ; atmospheric data. wks = gsn_open_wks("x11", "vector") gsn_define_colormap(wks,"rainbow") ; Change color map res = True ; Plot mods desired res@gsnpolar = "NH ; Northern hemisphere res@gsnleftstring = "Zonal Wind" ; Left subtitle res@gsnrightstring = "meters/second" ; Right subtitle res@vcrefmagnitudef = 20 ; Vector magnitude res@vcreflengthf = 0.09 ; Size of ref vector res@vcmindistancef = 0.02 ; Thins arrows near pole res@vcglyphstyle = "CurlyVector" ; Turns on curly vectors res@vcmonolinearrowcolor = False ; Multi-colored vectors res@gsnspreadcolors = True ; Span full color map plot = gsn_csm_vector_map(wks,u,v,res)

49 Want to learn more NCL website NCL Resources NCL functions NCL application examples NCL workshop 4 days long, 2 times a year June 12-15, 2012 Oct 9-12, 2012

50 Need help Send to CISL help Subscribe to NCL and NCAR Graphics lists and send a message to: ncl-install@ucar.edu ncl-talk@ucar.edu ncarg-talk@ucar.edu

51 References NCL Workshop Presentations (Dennis Shea, Mary Haley) NCL reference pages NCL Mini-Language Reference Manual NCL Mini Graphics Manual