Modeling Training Workshop on WRF, SMOKE and CMAQ Asia Center for Air Pollution Research (ACAP) January 2011

Transcription

1 Modeling Training Workshop on WRF, SMOKE and CMAQ Asia Center for Air Pollution Research (ACAP) January 2011 Prepared by Dr. Joshua Fu 1, Dr. Yun-Fat Lam 1 and Dr. Tatsuya Sakurai 2 1 University of Tennessee, Knoxville, U.S.A. 2 JAPAN NUS CO., LTD., Japan Organized by World Meteorological Organization (WMO)/Global Atmosphere Watch (GAW) Urban Research Meteorology and Environment (GURME), and Asia Center for Air Pollution Research (ACAP) Objectives: 1. Exchange information on current air quality modeling and emission modeling capabilities 2. Transfer a suite of USEPA's public air quality modeling tools and User's Guides, including Models- 3/CMAQ system (for simulations of multiple pollutants, e.g., ozone, PM, visibility, deposition, air toxics, etc.) 3. Test run includes two tutorial cases (27km domain and 9km domain of Asia region) of CMAQ simulations and demonstrate the model capabilities using the Models-3's visualization tool, VERDI, and post analysis for modeling result. In addition, hands on training on how to write a simple IOAPI FORTRAN program is also included. Information and Technology Transfer: 1. Weather Research Forecasting Model Advanced Research WRF (WRF-ARW v3.1/3.2) for Linux operation system. 2. The latest release of Sparse Matrix Operator Kernel Emissions (SMOKE v2.7.1) for Linux operation system. 3. The latest release of CMAQ (version 4.7.1) for Linux operation system. 4. " Visualization Environment for Rich Data Interpretation" (VERDI) visualization and analysis tool used for CMAQ and other air quality models. 5. Electronic copies of WRF, SMOKE and CMAQ science algorithm documentation, relevant papers and CMAQ4.7.1 release note. 6. Instructions on how to install Xming, PuTTY and Security Shell. Data and Resources Requirement for Running Models-3/CMAQ: 1. The CMAQ model, although an advanced model and being able to simulate multiple pollutants, is a quite complex photochemical grid model. It requires (1) elaborate emission and meteorological data and processing (e.g., SMOKE and WRF) (2) a higher learning curve, and (3) could be computationally demanding (may not be a problem anymore in the foreseeable future). 1

2 2. The resources recommended for running on the PCs are given as follows: a. High performance Linux PCs (preferred system) or SUN/SGI workstations b. PC Hardware: Hard disk 50 GB, RAM > 2 G, CPU > 500 MHz c. PC Software: 1. Operation System - Linux; 2. FORTRAN90 compiler (Portland Group Fortran for Linux recommended) Note: It is highly recommended to visit the website: to learn about the Models-3/CMAQ system and SMOKE to learn about WRF-ARW Models-3/CMAQ System Framework The next four days of hand-on training involve installation and running of several programs (marked inside the red or orange box) as shown in the figure above. Each day is going to cover one topic/model (i.e., Day 1 WRF, Day 2 SMOKE, Day 3 CMAQ & Day 4 m3tools). Moreover, we have divided each day into four units: Morning section 1 - Basic information of the model (about hours) Morning section 2 - Model installation (about hours) Afternoon section 1 - Understand and run sample scripts (about 2.0 hours) Afternoon section 2 - Visualize outputs (about 1.0 hour) Before we get into the detail of WRF in the first day, some basics operation on Linux in windows platform will be covered. 2

3 Tentative schedule Date 26-Jan 27-Jan 28-Jan Morning Afternoon Section 1 Section 2 Break Section 1 Section 2 9:00-10:00 10:00-12:00 12:00-13:30 13:30-15:30 15:30-17:00 WRF WRF Installation/ Lunch WRF WRF Visualization Lecture Runscript Tutorial Case (NCL) SMOKE Lecture CMAQ Lecture SMOKE Installation/ Runscript CMAQ Installation/ Runscript Lunch Lunch SMOKE SMOKE Visualization Tutorial Case (VERDI) CMAQ Tutorial Case CMAQ Visualization (VERDI) Morning Afternoon Break Date Section 1 Section 2 Section 1 Section 2 9:00-10:00 10:00-12:00 12:00-13:30 13:30-15:00 15:00-16:00 29-Jan M3tools IOAPI Programming Lunch Wrap-up Wrap-up 3

4 ACAP Modeling Training Workshop, Niigata, Japan, Jan , 2011 International Workshop on Regional Air Quality Modeling Dr. Joshua S. Fu 1, Dr. Yun Fat Lam 1 and Dr. Tatsuya Sakurai 2 1 University of Tennessee, Knoxville, USA 2 JAPAN NUS CO., LTD., Japan

5 International Workshop on Regional Air Quality Modeling Day 1: Linux tools for air quality modeling WRF installation WPS, OBSGRID and WRF-ARW v3 WRF simulation NCL (visualization)

6 International Workshop on Regional Air Quality Modeling Day 2: SMOKE installation IOAPI and VERDI SMOKE tutorial simulation point source and biogenic source VERDI (visualization)

7 International Workshop on Regional Air Quality Modeling Day 3: CMAQ installation MCIP, ICON, BCON and JPROC CMAQ biomass simulation Nest down scenario VERDI (visualization)

8 International Workshop on Regional Air Quality Modeling Day 4: IOAPI tools M3tools IOAPI program for CMAQ input/output

9 Background of Air Quality Modeling

10 Air Pollution Control in 21 st Century City /Town Counties State/ Multi-Counties Federal/ Multi-State International Los Angeles photochemical smog

11 Types of Air Quality Models Photochemical vs. Dispersion models (chemistry/transport) (e.g., Gaussian plume) Eulerian grid vs. Lagrangian trajectory Emissions-based (source-oriented) (e.g. OZIP/EKMA, HYSPLIT) vs. Observation-based (receptor-oriented, e.g., CMB) Prognostic vs. Statistical (e.g., Monte-Carlo) Mathematical vs. Physical (e.g., wind tunnel)

12 Evolution of Air Quality Models 1st-generation AQM (1970s s) Dispersion models (e.g., Gaussian plume models) Photochemical box models (e.g. OZIP/EKMA) 2nd-generation AQM (1980s s) Photochemical grid models (e.g., UAM, RADM) 3rd-generation AQM (1990s s) Community-Based One-Atmosphere Modeling System (e.g., U.S. EPA s Models-3/CMAQ)

13 First-Generation Air Quality Models Gaussian Dispersion Model Photochemical Box Model ISC3, CALPUFF, AERMOD (for primary pollutants) OZIP/EKMA (for ozone)

14 2nd-Generation Air Quality Models Eulerian Grid Models : UAM, RADM, REMSAD, ROM

15 Examples of 2nd-Generation Air Quality Models RADM ROM UAM Photochemical Grid Models

16 Examples of 2nd-Generation Air Quality Models Model Structure Simulation objective ROM RADM UAM OZIPM/EKMA 3-D Eulerian Grid Model Regional oxidants (O 3 ) 3-D Eulerian Grid Model Regional oxidants & acid deposition 3-D Eulerian Grid Model Urban oxidants (O 3 ) Lagrangian Trajectory (Box) Model Urban oxidants (O 3 ) Grid Resolution ~18 km 20 ~ 80 km 4 ~ 8 km 1-D box Vertical 6 ~ 15 layers Box height Structure 3 layers σ-coord 5 ~ 8 layers change with PBL Chemistry mechanism CB4 RADM2 CB4 CB4, RADM2 Meteorological inputs Objective Analysis MM4 DWM SAIMM Local observation Advection Smolarkiewicz Smolarkiewicz Advected along scheme scheme scheme wind trajectory Family models Lagrangian polynomial scheme SAQM, TAQM UAM-V, CAMx, REMSAD

17 Third-Generation Air Quality Models: U.S.EPA s Models-3/CMAQ Open-Access Community-Based Models : User-friendly, modular, common modeling framework for scientists and policy-makers. Advanced Computer Technologies : High performance hardware and software technologies (Cross-platform, GUI, distributed computing, visualization tools, etc.). One-Atmosphere Modeling : Multi-pollutant (Ozone, PM, visibility, acid and nutrients deposition, air toxics, etc.), multi-scale.

18 One-Atmosphere Approach (Cars, trucks, planes, boats, etc.) (Power plants, refineries/ chemical plants, etc.) Mobile Sources NOx, VOC, PM, Toxics Industrial Sources NOx, VOC, SOx, PM, Toxics Area Sources NOx, VOC, PM, Toxics (Residential, farming commercial, biogenic, etc.) Chemistry Meteorology Ozone PM Acid Rain Visibility Air Toxics Atmospheric Deposition Climate Change

19 Major Elements of AQM Chemistry (Gas- & Aqueous-phase, Heterogeneous) Emissions (Sources) Deposition (Sinks) Air Quality Model Meteorology (Advection, Diffusion & Cloud Processes) Predictions (Species Conc.)

20 Models-3/CMAQ System Framework Meteorology Processor MM5/WRF or RAMS or ETA MCIP Emission Processor SMOKE or Emission Gridding (GIS) Air Quality Model (CMAQ) ICON BCON JPROC CCTM PAVE/VERDI

21 Part 1 - WRF

22 ACAP Modeling Training Workshop, Niigata, Japan, Jan , 2011 International Workshop on Regional Air Quality Modeling Dr. Joshua S. Fu 1, Dr. Yun Fat Lam 1 and Dr. Tatsuya Sakurai 2 1 University of Tennessee, Knoxville, USA 2 JAPAN NUS CO., LTD., Japan

23 International Workshop on Regional Air Quality Modeling Day 1: Linux tools for air quality modeling WRF installation WPS, OBSGRID and WRF-ARW v3 WRF simulation NCL (visualization)

24 International Workshop on Regional Air Quality Modeling Day 2: SMOKE installation IOAPI and VERDI SMOKE tutorial simulation point source and biogenic source VERDI (visualization)

25 International Workshop on Regional Air Quality Modeling Day 3: CMAQ installation MCIP, ICON, BCON and JPROC CMAQ biomass simulation Nest down scenario VERDI (visualization)

26 International Workshop on Regional Air Quality Modeling Day 4: IOAPI tools M3tools IOAPI program for CMAQ input/output

27 Background of Air Quality Modeling

28 Air Pollution Control in 21 st Century City /Town Counties State/ Multi-Counties Federal/ Multi-State International Los Angeles photochemical smog

29 Types of Air Quality Models Photochemical vs. Dispersion models (chemistry/transport) (e.g., Gaussian plume) Eulerian grid vs. Lagrangian trajectory Emissions-based (source-oriented) (e.g. OZIP/EKMA, HYSPLIT) vs. Observation-based (receptor-oriented, e.g., CMB) Prognostic vs. Statistical (e.g., Monte-Carlo) Mathematical vs. Physical (e.g., wind tunnel)

30 Evolution of Air Quality Models 1st-generation AQM (1970s s) Dispersion models (e.g., Gaussian plume models) Photochemical box models (e.g. OZIP/EKMA) 2nd-generation AQM (1980s s) Photochemical grid models (e.g., UAM, RADM) 3rd-generation AQM (1990s s) Community-Based One-Atmosphere Modeling System (e.g., U.S. EPA s Models-3/CMAQ)

31 First-Generation Air Quality Models Gaussian Dispersion Model Photochemical Box Model ISC3, CALPUFF, AERMOD (for primary pollutants) OZIP/EKMA (for ozone)

32 2nd-Generation Air Quality Models Eulerian Grid Models : UAM, RADM, REMSAD, ROM

33 Examples of 2nd-Generation Air Quality Models RADM ROM UAM Photochemical Grid Models

34 Examples of 2nd-Generation Air Quality Models Model Structure Simulation objective ROM RADM UAM OZIPM/EKMA 3-D Eulerian Grid Model Regional oxidants (O 3 ) 3-D Eulerian Grid Model Regional oxidants & acid deposition 3-D Eulerian Grid Model Urban oxidants (O 3 ) Lagrangian Trajectory (Box) Model Urban oxidants (O 3 ) Grid Resolution ~18 km 20 ~ 80 km 4 ~ 8 km 1-D box Vertical 6 ~ 15 layers Box height Structure 3 layers σ-coord 5 ~ 8 layers change with PBL Chemistry mechanism CB4 RADM2 CB4 CB4, RADM2 Meteorological inputs Objective Analysis MM4 DWM SAIMM Local observation Advection Smolarkiewicz Smolarkiewicz Advected along scheme scheme scheme wind trajectory Family models Lagrangian polynomial scheme SAQM, TAQM UAM-V, CAMx, REMSAD

35 Third-Generation Air Quality Models: U.S.EPA s Models-3/CMAQ Open-Access Community-Based Models : User-friendly, modular, common modeling framework for scientists and policy-makers. Advanced Computer Technologies : High performance hardware and software technologies (Cross-platform, GUI, distributed computing, visualization tools, etc.). One-Atmosphere Modeling : Multi-pollutant (Ozone, PM, visibility, acid and nutrients deposition, air toxics, etc.), multi-scale.

36 One-Atmosphere Approach (Cars, trucks, planes, boats, etc.) (Power plants, refineries/ chemical plants, etc.) Mobile Sources NOx, VOC, PM, Toxics Industrial Sources NOx, VOC, SOx, PM, Toxics Area Sources NOx, VOC, PM, Toxics (Residential, farming commercial, biogenic, etc.) Chemistry Meteorology Ozone PM Acid Rain Visibility Air Toxics Atmospheric Deposition Climate Change

37 Major Elements of AQM Chemistry (Gas- & Aqueous-phase, Heterogeneous) Emissions (Sources) Deposition (Sinks) Air Quality Model Meteorology (Advection, Diffusion & Cloud Processes) Predictions (Species Conc.)

38 Models-3/CMAQ System Framework Meteorology Processor MM5/WRF or RAMS or ETA MCIP Emission Processor SMOKE or Emission Gridding (GIS) Air Quality Model (CMAQ) ICON BCON JPROC CCTM PAVE/VERDI

39 Part 1 - WRF

40 Instruction_26-Jan_Morning_Section2 What will cover: 1. PuTTY, SSH Secure Shell Client and Xming installation 2. WRF installation WPS, OBSGRID and WRF Training Files All the training materials are located at $WRKDIR and $SODIR directory $WRKDIR = /data/trainee0 $SODIR = /data/acap/models/ylam/source All the scripts are written in c-shell. To define environmental variable, use setenv command. PART 1 For PC to connect to Linux system through PuTTY client PART 1A: Setting up PuTTY a) Download PuTTY from b) Click putty<version>-installer.exe and save the file or Go to $SODIR to copy putty installer.exe c) Double click the program. 1) Click Next 2) Select the directory where you want the program to install and click Next 3) Click Next 4) Click Next 5) Click Finish d) Double click on PuTTY icon 21

41 e) Click on SSH tab f) Make sure the X11 forwarding is check g) Type the Host Name: , Put the Saved Sessions Name: ACAP, and Click Save 22

42 e) Click on ACAP, click Load Click Open f) Type in your login name and password Now you are successfully log into a Linux system through PuTTY 23

43 For PC to connect to Linux system through SSH Secure Shell Client PART 1B: Setting up SSH Secure Shell a) Download SSH from b) Click Download Now (5.7 MB) and save the file or Go to $SODIR/sshsecureshellclient exe c) Double click the program. 1) Click Next 2) Click Yes 3) Select the directory where you want the program to install and click Next 4) Select the program folders and Click Next 5) Click Next for all components 24

44 6) Click Next 7) Click Finish d) Double click on SSH Secure Shell Client icon e) Click on SSH Secure Shell Client 1) Click space bar 2) Put in host name and user name 3) Click connect 4) Type in your password 5) Click 6) This allows you transfer data between your laptop and the server 25

45 For PC to connect to X Windows Server PART 1C: Setting up Xming a) Download Xming from b) Click Xming-mesa and save the file or Go to $SODIR/Xming-mesa setup.exe c) Double click the program. 1) Click Next 2) Select the directory where you want the program to install and Click Next 3) Select Don t install an SSH client and click Next and click Yes 4) Click Next 5) Click Next 6) Click Install 7) Click Finish d) Double click on Xming icon 26

46 e) The Windows Task bar shows Xming is running f) To test the X Windows, run PuTTY, select Training_address, click load, and click Open Type in login name and password, and connect to Linux system g) Type xterm or xclock to test the X connections h) If the X Windows pops up on the screen, the Xming is working properly. Note: we will need PART 1A&B&C to access Linux system and execute VERDI (WRF/CMAQ visualization tools) to view air quality outputs. 27

47 PART 2 For Linux or Windows or Mac PART 2A: 1. Download WRF-ARW, Navigating the users page 2. Upload WRF-ARW into the server 1) Go Internet Explore/FireFox 2) Type in 3) Click on User Resources 4) Click on the Advanced Research WRF (ARW, Users page) 5) Click on Download and select WRF 6) Click on New Users 28

48 7) Fill out the form and click Submit 8) Click click here to get to the download page. 9) Download WRF v3.1.1, OBSGRID v3.2, WPS v3.1 and geog.tar.gz 29

49 30

50 You can also get to the download page by Due to the time constraint, we will copy these files from $SODIR (WRFV3.1.1.TAR.gz, WPSV3.2.TAR.gz, OBSGRID.tar.gz and geog.tar.gz ) 10) Upload all three files into the Server using SSH Secure Shell Filer Transfer. Put it under $WRKDIR directory. (click on the icon, click space bar, type in Host Name and User Name, type in password ) 11) In this case, $WRKDIR directory is /data/trainee0. 31

51 12) Drag the downloaded files from left side of the screen and drop it to right side of the screen. Note: PART 2A ends here. Now we will show you how to navigating the WRF users page. See 32

52 For Linux system only PART 2B: 1. Installation of WPS/WRF/OBSGRID 2. Installation of NetCDF and NCL 1) Click on SSH Secure Shell Client 2) Click space bar, type in Host Name and User Name, type in password 3) Go to $WRKDIR directory by typing cd $WRKDIR and hit Enter. This is the location where WRF and WPS are uploaded to. Note: $WRKDIR = /data/trainee0 in UTK server 4) Type mkdir WRF_Folder, hit Enter 5) Type mv *.gz WRF_Folder, hit Enter 6) Type cd WRF_Folder, hit Enter 7) Choose the version you want to install. In ACAP, we choose to install WRFV3.1.1 due to the consideration of version of compiler. Type tar zxvf WRFV3.1.1.TAR.gz, hit Enter Type tar zxvf WPSV3.2.1.TAR.gz, hit Enter Type tar zxvf OBSGRID.tar.gz, hit Enter Type tar zxvf geog.tar.gz, hit Enter 8) Type mv OBSGRID./WPS/., hit Enter 9) Okay, now we have WPS and WRFV3. Before we do any compilation, we have to compile netcdf and ncarg graphic first. Go to Internet Explore/FireFox For NETCDF a) Click here It will redirect you to b) Click on Downloads 33

53 c) Click on netcdf d) Select on e) Select the gzipped version of and download it or get it from $SODIR directory f) Upload netcdf p1.tar.gz to the server using SSH Secure Shell File Transfer. See PART 2A Step for instruction or get it from $SODIR. Put the file under your $WRKDIR directory. g) Go to $WRKDIR directory from SSH Secure Shell Client. 34

54 h) Type mkdir Lib i) Type mv $WRKDIR/netcdf p1.tar.gz./Lib/. j) Type cd Lib k) Type tar zxvf netcdf p1.tar.gz to untar the file l) Type cd netcdf p1/src m) To install netcdf, you will have to define four variables, which are FC (FORTRAN compiler), F90 (FORTRAN90 compiler) and CC (cc compiler). Now our case, we only have Portland Group Fortran (pgf90) For pgf Type setenv FC /opt/pgi/linux86-64/2010/bin/pgf90 Type setenv F77 /opt/pgi/linux86-64/2010/bin/pgf90 Type setenv F90 /opt/pgi/linux86-64/2010/bin/pgf90 Type setenv CC /usr/bin/gcc For ifort Type setenv FC /data/apps/intel/ /bin/intel64/ifort Type setenv F77 /data/apps/intel/ /bin/intel64/ifort Type setenv F90 /data/apps/intel/ /bin/intel64/ifort Type setenv CC /usr/bin/gcc" Type setenv FFLAGS '-i-static -mp' Type setenv CPPFLAGS '-fpic -DpgiFortran' In addition, you have to tell configure file where you want to install. To do so, prefix flag is used. For pgf Type./configure --prefix=$wrkdir/lib/netcdf_3.6.0_install_pgf90" For ifort Type./configure --prefix=$wrkdir/lib/netcdf-3.6.0_install_ifort 35

55 Type make check Check for any error Type mkdir $WRKDIR/Lib/netcdf_3.6.0_install_pgf90 Type mkdir $WRKDIR/Lib/netcdf_3.6.0_install_pgf90/lib Type make install Type cd../../netcdf_3.6.0_install_pgf90/lib n) If you successfully install netcdf, you will see libnetcdf.a 36

56 For NCL a) Click here It will redirect you to b) Click on the webpage to register c) Click on Create Account 37

57 d) Fill out all the information and click Next e) Go back to the previous page, click on NCL:NCAR Command Language f) Click on NCL Version 5.1.1/Version

58 g) Click on NCL Version precompiled binaries and click on Download files for. h) Log-in based on your user ID and password i) Click on Accept License j) Select ncl_ncarg linux_x86_64_gcc412.tar.gz and click on download button 39

59 k) Upload the file using SSH Secure Shell File Transfer and place it under $WRKDIR/Lib or copy directly from $SODIR l) Go back to SSH Secure Shell Client under the directory of $WRKDIR/Lib m) Type cd $WRKDIR/Lib n) Type mkdir NCL_install_gcc o) Type mv ncl_ncarg linux_x86_64_gcc412.tar.gz./ncl_install_gcc/. p) Type cd NCL_install_gcc q) Untar the file by typing tar zxvf ncl_ncarg linux_x86_64_gcc412.tar.gz Type $WRKDIR/Lib/NCL_install_gcc/bin/ncl and Type quit r) Now you have all the ncarg library for WPS installation. Note: It is not necessary to have ncarg library for WPS compilation. 10) After netcdf and Ncarg are installed, now we can continue installing WPS and WRF 11) It is necessary to install WRF before WPS since some of the libraries from WRF are required by WPS. 12) Installation of WRF. 13) Type cd $WRKDIR/WRF_Folder/WRFV3 40

60 14) Type setenv NETCDF $WRKDIR/Lib/netcdf_3.6.0_install_pgf90 Note: a)wrf requires both the lib and include directories b) Make sure both netcdf and MPI libraries are compiled using the pgf90 compiler. 41

61 15) Additional Information for specific issues. 16) Before configuration, we would like to set up Large File Support Type setenv WRFIO_NCD_LARGE_FILE_SUPPORT 1 17) Type./configure 18) Smpar -> Shared-Memory Parallelism OpenMP DMPar -> Distributed-Memory Parallelism MPI 42

62 19) Type cp configure.wrf configure.wrf_org 20) Type setenv WRF_EM_CORE 1 21) Type limit datasize unlimited or ulimit s unlimited 22) Type./compile h 23) Type./compile em_real >& compile_wrf.log It will take about an hour for compilation. You will find several executable files under $WRKDIR/WRF_Folder/WRFV3/test/em_real WRF compilation is completed. 43

63 WPS compilation 24) Type cd $WRKDIR/WRF_Folder/WPS 25) Type setenv NCAR_ROOT $WRKDIR/Lib/NCL_install_gcc 26) Type vi configure 27) Type./configure, select 5, it will create a file called configure.wps 28) Type vi configure.wps 29) Pound out line#18-21 and 26, since we will not use any compression library 30) Pound out line#45 and #46 31) Replace line #54 and #55 to CC = gcc SCC = gcc 32) Pound out line#57, #59 and #60 33) Type cp configure.wps configure_wps_work 34) Type./compile >& compile_wps.log It will take about 1-2 minutes for compilation. You will find a executable file (ungrib.exe, geogrid.exe, metgrid.exe) under $WRKDIR/WRF_Folder/WPS/ 44

64 35) WPS compilation is completed. OBSGRID compilation 36) Type cd $WRKDIR/WRF_Folder/WPS/OBSGRID 37) Type./configure 38) Type./compile >& compile_obsgrid.log It will take about 1-2 minutes for compilation. You will find a executable file (obsgrid.exe) under $WRKDIR/WRF_Folder/WPS/OBSGRID Note: PART 2B ends here. Now we are ready to run WPS/OBSGRID/WRF. Let s go for Lunch!!!! 45

65 Instruction_26-Jan_Afternoon_Section1&2 What will cover: 1. Download WPS and OBSGRID input data 2. WRF simulation for biomass study a. WPS, b. OBSGRID and c. WRF 3. Visualize input/output using NCL Training Files All the training materials are located at $WRKDIR and $SODIR directory $WRKDIR = /data/trainee0 $SODIR = /data/acap/models/ylam/source All the scripts are written in c-shell. To define environmental variable, use setenv command. In this exercise, we will simulate a 3-day biomass burning episode with a 10-day WRF/CMAQ simulation. Note: WRF requires a minimum of 12 hours of spinning-off time, where CMAQ requires a minimum of 6 days of spinning-off time. Therefore, for this exercise, we will have to simulate a total of 10 days of WRF simulation. All WRF simulations are one-way nested. For information of two-way nesting, you may see the example at 46

66 PART 1 Download WPS input data (used for initial and boundary conditions and FDDA) WPS-UNGRIB input data 1) Go to 2) Select Download, then select input Data from NCAR 3) Select NCEP Final Analysis 4) Instead of the link, use this one: 5) Click DATA ACCESS on the bottom of the page 6) Choose Register Now if you don t have an account. If you have one, go ahead log it in. 47

67 7) Fill out all the questions in the registration form. Once it is completed, use your and password to login to ds083.2 data 48

68 8) Create a new directory called Data under $WRKDIR. Upload the Unix csh script to $WRKDIR/Data and named it to DS083.2_script or you can get it from $SODIR/DATA 9) Type chmod u+x DS083.2_script 10) Modify the passwd and =ylam@utk.edu in DS083.2_script 11) Type./DS083.2_script to start downloading the data OBSGRID input data 12) Go to for upper air data, and for surface observation 49

69 13) Doing the same step as DS083.2 data. After that, select the time period of 05 April to 17 April, 2006, select Draw Box and draw the box on the study domain. 14) In this particular exercise, we only have one. So, we can t create the Unix.csh directly. (no create button is available. We have to go through two or three extract steps. 15) Click the Show All Files 16) Click Next 50

70 17) Put a check box on the dataset that you want. In our case, it is Y93126.gz 18) Click Create button 19) Save the Unix csh as DS353.4_script and put it in $WRKDIR/Data or you can get it from $SODIR/DATA 20) Type chmod u+x DS353.4_script 21) Modify the passwd and =ylam@utk.edu in DS353.4_script. Type./DS353.4_script to start downloading the data 22) Type gzip d Y93126.gz 23) Doing the same step as DS464.0 data, select the time period of 05 April to 17 April, 2006, select Draw Box and draw the box on the study domain. 51

71 LIST_E & LIST_F 24) Save the Unix csh as DS464.0 _script and put it in $WKDIR/Data or you can get it from $SODIR/DATA 25) Type gzip d *.gz 26) Unlike the DS083.2 data (fnl_data), both DS464.0 and requires preprocess before it can be used. 27) Go to $WRKDIR/WRF_Folder/WPS/OBSGRID/util 28) Click on 29) Click FETCH or ftp://ftp.ucar.edu/mesouser/mm5v3/fetch.tar.gz or $SODIR/FETCH.TAR.gz 30) Type wget ftp://ftp.ucar.edu/mesouser/mm5v3/fetch.tar.gz in $SODIR/WRF_Folder/WPS/OBSGRID/util 52

72 31) Type tar zxvf FETCH.TAR.gz a. Type cd FETCH b. Type cp adp_to_little-r.csh adp_to_little-r_org.csh c. Type vi adp_to_little-r.csh d. Modify those values to set DataDir = $WRKDIR/Data set InUpperData = ( $DataDir/Y93126 ) set InSFCData = ( $DataDir/Y93117 $DataDir/Y93118 $DataDir/Y93119 $DataDir/Y93120 $DataDir/Y93125) set starting_date = set ending_date = set lon_e = 174 set lon_w = 57 set lat_s = -2 set lat_n = 71 32) Before running.csh file, we have to add a path location for pgf90. The makefile for the.csh is only configure to pgf90. So, we will use pgf90 for adp_to_little-r.csh. First, we have to put the pgf90 location into the path variable. You can type which pgf90 setenv PATH /opt/pgi/linux86-64/2010/bin:$path (location your pgf90 location) Type./adp_to_little-r.csh Note: Compilations of FETCH subprograms are included in the script 33) Type mkdir -p $WRKDIR/Data/Data_FETCH 34) Type mv obs* $WRKDIR/Data/Data_FETCH/. 53

73 35) Type mv surf* $WRKDIR/Data/Data_FETCH/. 36) Type cd $WRKDIR/Data/Data_FETCH 37) Type ls -la surface_obs_r\: * grep surface_ wc l. You should get 289 lines (24 hours * 12 days + 1) 38) Type ls -la obs* grep obs wc l. You should get 49 lines (24 hours/6 hours * 11 days + 1) Note: we have set up the OBSGRID for every three hours. Therefore we have to create additional input data. To do so, we have utilize the surface_obs data. So, we have to add extra hours 03, 09, 15, 21 39) Type vi extra_hour.csh #! /bin/csh ln -s surface_obs_r: _03 obs: _03 ln -s surface_obs_r: _03 obs: _09 ln -s surface_obs_r: _03 obs: _15 ln -s surface_obs_r: _03 obs: _21.. Other days 40) Type chmod u+x extra_hour.csh 41) Type./extra_hour.csh 42) Type ls -la obs* grep obs wc l. You should get 97 lines (24 hours/3 hours * 12 days + 1) Now we are ready to run WPS and WRF!!! 54

74 PART 2 2a. WPS - GEOGRID, UNGRIB, METGRID For more information, WPS-GEOGRID For 3.1, no need data For 3.2, Get Terrestrial Input Data The latest version of the WPS processes fields for gravity wave drag schemes in both the ARW and the NMM. Regardless of whether a GWD scheme will be used, the static fields must be available in the directory where other static geographical fields are found. 1) Go to $WRKDIR/WRF_Folder/WPS 2) Type cp $SODIR/WRF_WPS_scripts/namelist.wps_Biomass_tuitorial $WRKDIR/WRF_Folder/WPS/. 3) Type vi namelist.wps_biomass_tuitorial 55

75 a) active_grid => A list of MAX_DOM logical values specifying, for each grid, whether that grid should be processed by geogrid and metgrid. Default value is.true. b) subgrid_ratio_x c) subgrid_ratio_y These two variables only are useful when subgrid=yes (For WRF-Fire only) d) io_form_geogrid => the type of format use to output geogrid We have to change the directory to 4) To run GEOGRID, it requires 1) Geographical data, 2) namelist.wps Type ls $WRKDIR/WRF_Folder/geog 56

76 Type rm namelist.wps Type ln s namelist.wps_biomass_tuitorial namelist.wps Note: the namelist is ready. 5) Type./geogrid.exe >& geogrid.log 6) GEOGRID output includes 1) geogrid.log, 2) geo_em.d0*.nc Type tail geogrid.log Type ls la geo_* Note: 1. GEOSGRID only needs to run once for the same domain configuration. 2. All geo_em.d0* are netcdf files. WPS-UNGRIB The purpose of UNGRIB is to extract fields from an external grib file and write it to WPS recognizable format. UNGRIB - is NOT dependant on any WRF model domain. - is NOT dependant on GEOGRID. - Different dataset will use different Vtables. 7) Go to $WRKDIR/WRF_Folder/WPS 8) Type cp $SODIR/WRF_WPS_scripts/namelist.wps_Biomass_tuitorial $WRKDIR/WRF_Folder/WPS/. 9) Type vi namelist.wps_biomass_tuitorial 57

77 10) To run UNGRIB, it requires 1) External grib files, 2) namelist.wps, 3) Vtable Type ls $WRKDIR/Data, the fnl_* are the grib files Type vi link_grib.csh Type./link_grib.csh Type./link_grib.csh $WRKDIR/Data/fnl* Type rm namelist.wps Type ln s namelist.wps_biomass_tuitorial namelist.wps Type vi namelist.wps, Modify line#55 to FNL Note: the namelist is ready Type ln -sf ungrib/variable_tables/vtable.gfs Vtable 58

78 11) Type./ungrib.exe >& ungrib.log 12) UNGRIB output includes 1) FNL:date, 2) ungrib.log Type tail ungrib.log Type ls la FNL* Type rm rf GRIBFILE.* WPS-METGRID The purpose of METGRID is to interpolate external meteorological input horizontally to defined grid points 13) Go to $WRKDIR/WRF_Folder/WPS 14) Type cp $SODIR/WRF_WPS_scripts/namelist.wps_Biomass_tuitorial $WRKDIR/WRF_Folder/WPS/. 15) Type vi namelist.wps_biomass_tuitorial 59

79 16) To run, it requires 1) FNL:date files, 2) geo_em.d0*, 3) namelist.wps Type ls $WRKDIR/WRF_Folder/WPS/FNL* Type ls $WRKDIR/WRF_Folder/WPS/geo_em* Type vi namelist.wps_biomass_tuitorial Modify line#59 to FNL Note: the namelist is ready. 17) Type./metgrid.exe >& metgrid.log 18) METGRID output includes 1) met_em.d0*.date.nc, 2) metgrid.log Type tail metgrid.log 60

80 Type ls la met_em* Note: 1. All 3 WPS programs are run inside WPS directory 2. Make sure to clean up before each new WPS run 3. An integrated script (wrf_wps_biomass.script1) is located in $SODIR/WRF_WPS_scripts 2b. OBSGRID preprocessor OBSGRID The purpose of OBSGRID is to process observational data for WRF, which will be used to improve meteorological analyses (the first guess) on the mesoscale grid. 1) Go to $WRKDIR/WRF_Folder/WPS/OBSGRID 2) Type cp $SODIR/WRF_WPS_scripts/namelist.oa $WRKDIR/WRF_Folder/WPS/OBSGRID 3) Type vi namelist.oa 61

81 4) To run, it requires 1)obs:date file, 2) met_em.*, 3)namelist.oa Type ls $WRKDIR/Data/Data_FETCH/obs* Type ln -s../met_em.d0*. Type ls $WRKDIR/WRF_Folder/WPS/met_em* Type vi namelist.oa Modify line#14 to Note: the namelist is ready. 5) Type./obsgrid.exe >& obsgrid.log 6) OBSGRID output includes 1) metoa_em*, 2) plotobs_out*, 3)qc_obs_raw*, 4) obsgrid.log, 5) wrfsfdda_d01 Type tail obsgrid.log Type ls -la metoa_em.d0* 62

82 Type ls la wrfsfdda_d01 Note: other outputs are text files for summary of the run.2c. REAL and WRF REAL The purpose of real is to interpolate meteorological data to vertical model levels. The outputs of real can be used as input files for WRF 1) Go to $WRKDIR/WRF_Folder/WRFV3/test/em_real 2) Type cp $SODIR/WRF_WPS_scripts/all_clean.csh $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 3) Type cp $SODIR/WRF_WPS_scripts/link_metoa_met $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 4) Type cp $SODIR/WRF_WPS_scripts/namelist.input_BIOMASS_D1_wrf $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 5) Type rm namelist.input 6) Type ln -sf namelist.input_biomass_d1_wrf namelist.input 7) Type cp $SODIR/WRF_WPS_scripts/run_script $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 8) Type vi namelist.input_biomass_d1_wrf 63

83 We will go all the parameters by visiting Namelist.input contains 5 parts. 1) time_control 2) domains 3) physics 4) dynamics 5) bdy_control 9) Type vi link_metoa_met 64

84 This script is used to help to link all metoa data file. User has to set up environmental variables, WPS_root and WRF_root before running the script 10) Type setenv WPS_root $WRKDIR/WRF_Folder/WPS 11) Type setenv WRF_root $WRKDIR/WRF_Folder/WRFV3/test/em_real 12) Type./link_metoa_met 13) To run real, it requires 1)met_em.d*, 2)namelist.input Type./run_script to run real.exe 65

85 7) real output includes 1) wrfbdy_d01*, 2) wrfinput_d01*, 3) wrffdda_d0*, 4) rsl.out.00*, 5) rsl.error.00*type tail rsl.out.0000 Type tail rsl.error.0000 Note: real, wrf and ndown also produce the same output. It can be used to check for error. WRF The purpose of wrf is to numerically approximate the solutions to the model equations to produce a meteorological condition 14) Go to $WRKDIR/WRF_Folder/WRFV3/test/em_real 15) Type ln -sf $WRKDIR/WRF_Folder/WPS/OBSGRID/wrfsfdda_d01. 16) Type ls la *d01 17) WRF input includes 1) wrfbdy_d01*, 2) wrfinput_d01*, 3) wrffdda_d0*, 4) wrfsfdda_d01 18) Type cp $SODIR/WRF_WPS_scripts/run_script_2 $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 19) Type ln -sf namelist.input_biomass_d1_wrf namelist.input 20) To run WRF, type./run_script_2 21) WRF output includes 1) wrfout_d*, 2) rsl.out.00*, 3) rsl.error.00* Note: the wrfout* is netcdf format, and the contents can be viewed with ncdump h filename. ncdump can be found in $WRKDIR/Lib/netcdf_3.6.0_install_pgf90/bin/ncdump Due to the time limitation, we have to kill the run. Now we will copy output from $SODIR directory. Let remove the wrfout_d01_ _00:00:00 by typing rm wrfout_d01_ _00:00:00 Let copy output from $SODIR by typing cp $SODIR/WRF_Out/D1/*./. Type ls la wrfout* 66

86 Type tail rsl.out.0000 Type tail rsl.error.0000 ndown The purpose of ndown is to provide the initial and boundary conditions for a nest. Only one-way nests required ndown. Before running ndown, we have to create a real output for the nest domain 22) Go to $WRKDIR/WRF_Folder/WRFV3/test/em_real 23) Type rm -f met_em* (we need to use the met_em from nest domain domain 2) 24) Type cp $SODIR/WRF_WPS_scripts/link_nest_down_d02_met_d01 $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 25) Type vi link_nest_down_d02_met_d01 (you will have to set WPS_root and WRF_root) 26) Type./link_nest_down_d02_met_d01 27) Type rm -f namelist.input 28) Type cp $SODIR/WRF_WPS_scripts/namelist.input_BIOMASS_D2_init $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 29) Type diff namelist.input_biomass_d1_wrf namelist.input_biomass_d2_init 67

87 30) Type ln -sf namelist.input_biomass_d2_init namelist.input 31) Type mkdir tmp_input_wrfd01 32) Type mv *d01./tmp_input_wrfd01 33) Type./run_script (now we are running real using d02 information) Now we should have wrfbdy_d01 and wrfinput_d01. Both files actually contain data of d02. 34) To run ndown.exe, we need three files. 1) wrfndi_d02 2) namelist.input on nestdown setting 3) wrfout_d01 (from domain 1) 35) Type mv wrfinput_d01 wrfndi_d02 (ndown.exe recognizes wrfndi_d02 as input) 36) Type rm -f namelist.input 37) Type cp $SODIR/WRF_WPS_scripts/namelist.input_BIOMASS_D2_nd $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 68

88 38) Type ln -sf namelist.input_biomass_d2_nd namelist.input 39) Type rm wrfbdy_d01 (this wrfbdy comes from original ungrib data, for nested domain, we want to use the nest down output.) 40) Now we are ready to run ndown.exe. The output of ndown includes 1) Type cp $SODIR/WRF_WPS_scripts/run_script_3 $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 2) Type./run_script_3 41) The output of nestdown should includes 1) wrfinput_d02 2) wrfbdy_d02 3) rsl.* 42) Type mv wrfndi_d02 wrfndi_d02_t (we have no use of wrfndi_d02, but we will save a temporary copy) 43) Type mv wrfinput_d02 wrfinput_d01 44) Type mv wrfbdy_d02 wrfbdy_d01 45) Type mkdir -p../../../wrf_output/d1 46) Type mv wrfout_d01_*../../../wrf_output/d1/. (Move all the domain 1 outputs to provide room for domain 2 output) 47) Type ls *d01* 69

89 Note: for one-way nest, all inputs have to name it to d01 for wrf to recognize. Although all d01 data contains domain 2 information. 48) Type rm -f namelist.input 49) Type cp $SODIR/WRF_WPS_scripts/namelist.input_BIOMASS_D2_wrf $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 50) Type diff namelist.input_biomass_d2_wrf namelist.input_biomass_d1_wrf 51) Type ln -sf namelist.input_biomass_d2_wrf namelist.input 52) To run WRF, type./run_script_2 (check rsl.out.0000 for completion) => we will not run that because it is going to take too long 53) WRF output includes 1) wrfout_d*, 2) rsl.out.00*, 3) rsl.error.00* 54) Now we are no longer need the domain 2 input, Type mv wrfinput_d01 wrfinput_d02_t Type mv wrfbdy_d01 wrfbdy_d02_t 55) Do the same step for domain 3 initial condition, domain 2 nestdown,domain 3 wrf. 56) Instead manually step by step walking through the nest down process, we have created a script to do the entire process. 70

90 57) Type cp $SODIR/WRF_WPS_scripts/wrf_execution.csh $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 58) To use this script, we have to copy all na melist.input_biomass*, link* and run_script* and all_clean.csh to $WRKDIR/WRF_Folder/WRFV3/test/em_real/. 59) Type vi wrf_execution.csh WRF-ARW V3 User s Guide is found at: WPS and WRF end here!!! 71

91 PART What is NCL? The NCAR Command Language (NCL) is a free interpreted language designed specifically for scientific data processing and visualization. NCL has robust file input and output. It can read in netcdf, HDF4, HDF4-EOS, GRIB, binary and ASCII data. The graphics are world class and highly customizable. (from WRF-ARW tutorial) For information of NCL scripts using in WRF, WRF NCL In this exercise, we are going to run two examples using NCL 1) Preview domain plot 2) Basic surface plots Visualize input/output using NCL 1) We finished installing NCL this morning, now we are going to use pre-defined NCL script from WRF-ARW website. 2) The first thing we have to do is to set $NCARG_ROOT 3) Type setenv $NCARG_ROOT $WRKDIR/Lib/NCL_install_gcc 4) The second thing is to create.hluresfile file. 5) Go to $WRKDIR/Lib/NCL_install_gcc 6) Type cp $SODIR/WRF_WPS_scripts/.hluresfile $HOME/.hluresfile 7) This is a basic structure of a NCL script 8) Note: our NCL is installed at $WRKDIR/Lib/NCL_install_gcc/bin To run NCL, Type $WRKDIR/Lib/NCL_install_gcc/bin/ncl 72

92 Preview domain plot 9) To take advantage of what it has been developed, we will first download the ncl function given in TAL/wrf_show_wps_som_namelist.htm 10) Type cd $WRKDIR 11) Type mkdir NCL_script, Type cd NCL_script 12) Type wget TAL/WRFUserARW_add.ncl or copy it from $SODIR/NCL_scripts Type wget TAL/wrf_wps_show_domain_namelist.ncl or copy it from $SODIR/NCL_scripts Note: the WRFUserARW_add.ncl contains many functions. You may learn more in 13) Type vi wrf_wps_show_domain_namelist.ncl Modify line#10, so it points to $WRKDIR/NCL_script/WRFUserARW_add.ncl Modify line#17, so it will output pdf file 14) Type ln -s $WRKDIR/WRF_Folder/WPS/namelist.wps./. 15) Type $NCARG_ROOT/bin/ncl wrf_wps_show_domain_namelist.ncl 16) Open wps_show_dom_namelist.pdf from SSH Secure File Transfer 73

93 wps_show_dom_namelist-example1.pdf 17) Let try to modify few things and plot it again 18) Type vi wrf_wps_show_domain_namelist.ncl Modify line #42 from Test Domain to Biomass Study Go to #123, we will find the script called wrf_wps_dom function 19) I would like to add the country line. To do that, we have to change the base map data. Type vi WRFUserARW_add.ncl Find wrf_wps_dom or go line#2776 Go down to line# 2850, we want to add some scripts to modify the graph before plotting mpres@mpdatasetname = "Earth..4" ; This new database contains mpres@mpdatabaseversion = "MediumRes" ; Medium resolution database mpres@mpoutlineon = True ; Turn on map outlines mpres@mpnationallinethicknessf = 1.2 mpres@mpoutlineboundarysets = "National" mpres@mpoutlinespecifiers = (/"Brazil:states","China:states","India:states"/) For more information on modify the base map, go to 20) Type $NCARG_ROOT/bin/ncl wrf_wps_show_domain_namelist.ncl wps_show_dom_namelist-example2pdf 74

94 Basic surface plots 21) Go to wrf_surface1.htm 22) Type cd $WRKDIR/NCL_script 23) wget wrf_surface1.ncl 24) Type vi wrf_surface1.ncl See the line#6 25) Type ln -s $WRFDIR/WRF_Folder/WRF_OUTPUT/D1/wrfout_d01_ _00:00:00./. 75

95 26) Type vi wrf_surface1.ncl Modify line#6 to $WRKDIR/NCL_script/WRFUserARW_add.ncl Modify line#12 to wrfout_d01_ _00:00:00 Remove ; from line#16 27) Type $NCARG_ROOT/bin/ncl wrf_surface1.ncl 28) I don t like the color, let try to change it. Type vi wrf_surface1.ncl Add gsn_define_colormap(wks,"blwhre") ; choose color map to line# 28 Note: you can find more color option from 29) Type $NCARG_ROOT/bin/ncl wrf_surface1.ncl 76

96 Note: We can also use VERDI to plot WRF output. However, NCL gives a full flexibility on the appearance of a plot. NCL end here!!! 77