NAREGI Middleware Mediator

Transcription

1 Administrator's Guide NAREGI Middleware Mediator October, 2008 National Institute of Informatics

2 Documents List Administrator s Guide Group Administrator s Guide, NAREGI Middleware IS(Distributed Information Service) Administrator s Guide, NAREGI Middleware IS(Distributed Information Service) - LRPSConfig - Administrator s Guide, NAREGI Middleware SS(Super Scheduler) Administrator s Guide, NAREGI Middleware GridVM(Grid Virtual Machine) Administrator s Guide, NAREGI Middleware Portal Administrator s Guide, NAREGI Middleware PSE(Problem Solving Environment) Administrator s Guide, NAREGI Middleware WFT(GUI Workflow Tool) Administrator s Guide, NAREGI Middleware GVS (Grid Visualization System) Administrator s Guide, NAREGI Middleware DataGrid Administrator s Guide, NAREGI Middleware CA(Certification Authority) Administrator s Guide, NAREGI Middleware UMS(User Management Server) Administrator s Guide, NAREGI Middleware Authorization Service Administrator s Guide, NAREGI Middleware Renewal Service Administrator s Guide, NAREGI Middleware SBC(Synchronous Data Transfer Library) Administrator s Guide, NAREGI Middleware Mediator i

3 User s Guide Group User s Guide, NAREGI Middleware IS(Distributed Information Service) User s Guide, NAREGI Middleware Portal User s Guide, NAREGI Middleware PSE(Problem Solving Environment) User s Guide, NAREGI Middleware PSE(Problem Solving Environment) - Command line Interface - User s Guide, NAREGI Middleware WFT(GUI Workflow Tool) Programming Guide, NAREGI Middleware WFT(GUI Workflow Tool) User s Guide, NAREGI Middleware GVS(Grid Visualization System) - Client Module - User s Guide, NAREGI Middleware GVS(Grid Visualization System) - Parallel Visualization Module - User s Guide, NAREGI Middleware CA(Certification Authority) User s Guide, NAREGI Middleware UMS(User Management Server) User s Guide, NAREGI Middleware Authorization Service User s Guide, NAREGI Middleware Renewal Service User s Guide, NAREGI Middleware SBC (Synchronous Data Transfer Library) User s Guide, NAREGI Middleware Mediator Copyright National Institute of Informatics, Japan. All rights reserved. This file or a portion of this file is licensed under the terms of the NAREGI Public License, found at If you redistribute this file, with or without modifications, you must include this notice in the file. ii

4 Table of Contents Chapter 1. Installing the Mediator Summary of Installation Running Environment Hardware Requirements Software requirement Supported compilers...1 Chapter 2. Installation of the Mediator Installation of the mediator Preparation for installation Notes on installation Extract from the distributed package Setting up the Environment Variables Installation operation Compiling and linking Verification of installation Installation failure Set the access permission How to confirm your installation Confirmation operations using MPI Confirmation operations using both MPI and the SBC Note Partial Insallation Under the environment without SBC Under the environment without the NAREGI Middleware...14 Chapter 3. Providing the information of the running environment Chapter 4. Start/Stop and Maintenance Rebuilding with upgrading the prerequisite softwares Chapter 5. Uninstallation of the Mediator Before uninstallation iii

5 5.2. Unistallation operation Verify uninstallation Chapter 6. Upgrade installation of the Mediator iv

6 Chapter 1. Installing the Mediator 1.1. Summary of Installation This document explains how to install and operate the Mediator Running Environment Hardware Requirements This section describes minimum requirements of hardware for execution of the Mediator. Expansion of the memory capacity is required in proportion to calculation scale of simulation. [Linux environment] CPU: IA-32 Memory: 10MB or more Hard disk: 10MB or more [AIX environment] CPU: PowerPC (POWER4) Memory: 10MB or more Hard disk: 10MB or more Software requirement [Linux environment] OS: Linux (Centos5, OpenSUSE 10) Others: glibc2.5, GlobusToolkit 4.0.5, j2sdk 1.5, gcc3.2 and later, Synchronous Data Transfer Library (SBC) released at the same time with this [AIX environment] OS: AIX Version5.2 Others: GlobusToolkit 4.0.5, j2sdk 1.5, compilers in section 1.2.3, Synchronous Data Transfer Library (SBC) released at the same time with this Supported compilers [Linux environment] gcc3.2 and later [AIX environment] IBM XL C Version 6 IBM XL Fortran Version 8 release 1 There are no guarantees with these operations. 1

7 (1) Compiling and linking with other compiler listed above. (2) Linking the Mediator and the object compiled by other compiler listed avove. (3) Execution the Mediator and the program linked with the object(s) by case (1). (4) Execution of the program linked by case (2). AIX, PowerPC and Power4 are trademarks of IBM Corporation in the US and other countries. RED HAT is a registered trademark of Red Hat, Inc. Globus and Globus Toolkit are trademarks held by the University of Chicago. NFS is a trademark of Sun Microsystems, Inc. Java is a trademark or registered trademark of Sun Microsystems, Inc. in the US and other countries. 2

8 Chapter 2. Installation of the Mediator 2.1. Installation of the mediator Install the mediator by compiling the source code in your environment. The mediator executable, library and header files will be created by installation operation. [Important] To install all componets of the mediator, the NAREGI SBC is required. Install the SBC before installing the mediator. You can select the partial installation of the mediator for an environment without the SBC. You can do the partial installation with reference section Preparation for installation Notes on installation 1. The mediator package should be installed on each 'computation node *1 in a cluster. 2. If the running environments of each node in a cluster are same, the Mediator can be installed on the NFS storage, which can be read from any node on the cluster. *1 In this document 'computation node' means a host or a node in a cluster to execute the user program Extract from the distributed package Install the Mediator on the nodes, which execute the user programs. Make an installation directory and extract the distributed package 'mediator.tar.gz' in it. (The name of the distributed package can be different from 'mediator.tar.gz') In this document, we assume the package is stored at /tmp and the install directory of the mediator (DEX_ROOT) is the directory 'mediator' under /opt/naregi-mdl/tools # cd /opt/naregi-mdl/tools # mkdir mediator # cd mediator # tar zxvf /tmp/mediator.tar.gz./bin/./bin/mediator # Setting up the Environment Variables Set up these environment variables. 3

9 (1) DEX_ROOT Set 'DEX_ROOT' (2) MPIR_HOME,MPIROOT MPI install directory (3) GLOBUS_LOCATION Globus install directory (4) SBC_ROOT The SBC install directory (5) JAVA_HOME Specify the path to the installation directory of the JAVA. For AIX, specify the path to the installation directory of the JAVA for 32bit mode. (6) PATH Prepend $JAVA_HOME/bin and $MPIR_HOME/bin Installation operation Compiling and linking Do the 'make' command under the $DEX_ROOT. Both compilation and installation will be done. Note: No need to do 'make install'. $ cd $DEX_ROOT $ make.. $ 2.4. Verification of installation (1) Check whether any error messages are displayed during compilation and linking. (2) Check whether the two log files given below are created in the 'DEX_ROOT' (2-1) log.build This is a log file of compilation and linkage of the binaries for the GridMPI execution. (2-2) log.build.sbc This is a log file of compilation and linkage of the binaries for the SBC execution. (2-3) log.build.is This is a log file of compilation and linkage of the binaries for the SBC execution from the NAREGI Workflow Tool. 4

10 (3) Check whether log.build* files contains any error messages. (4) Check whether below files are created (execute following 2 commands). $ cd $DEX_ROOT $ ls -1 bin lib include bin: mediator mediator_g mediator_s include: SBC.h SBC_proto.h dexlibrary.h dex_msgtag.h dpp_define.h dpp_extern.h dpr_bucket.h dpr_define.h dpr_extern.h dpr_finddest.h dpr_shlibmng.h version.h lib: libdex.a libdex_g.a libdex_s.a 2.5. Installation failure If the installation fails, check your execution environment and environment variables. Execute below commands after correcting the problems. $ make clean $ rm log.build* Do the installation again. 5

11 2.6. Set the access permission Change the access permission of $DEX_ROOT, the directories under $DEX_ROOT lib,islib,include and handle and files under those directories as readable by the user. # cd $DEX_ROOT # chmod 755. bin lib include bin/* lib/* include/* 2.7. How to confirm your installation You can confirm your installation by following operations after installing on all calculation nodes Confirmation operations using MPI Prepare following things before confirmation. (a) an user account for all calculation nodes (b) environment variable setting in chapter.3 on (a) (1) Login Login to a terminal as the user. Then confirm the DEX_ROOT environment variable is exist. If it is not exist, then set up it into the shell profile (.profile or other) of the user and then login again. And add the $DEX_ROOT/bin in the PATH variable in the shell profile. ( sh or bash ) PATH=$DEX_ROOT/bin:$PATH (2) Making a copy of the test program Make a copy of the test program "$DEX_ROOT/test/pmtest_is" into suitable directory. coping into ~/work $ cp -r $DEX_ROOT/test/pmtest ~/work (3) Compilation of the test program Change the current directory into the copied directory. $ cd ~/work/pmtest (3-1) Compilation of particle program Compile the particle program by 'make particle' command when you will execute it on Linux. (Linux) 6

12 $ make particle Compile the mesh program by 'make -f Makefile.aix particle' command when you will execute it on AIX. (AIX) $ make -f Makefile.aix particle (3-2) Compilation of mesh program Compile the mesh program by 'make mesh' command when you will execute it on Linux. (Linux) $ make mesh Compile the mesh program by 'make -f Makefile.aix mesh' command when you will execute it on AIX. (AIX) $ make -f Makefile.aix mesh (4) Setup Edit these files. (a) Specify the name of the computation node to execute the 'mesh' program in the 'm_conf' file. m_conf sig0001.naregi.org (b) Specify the name of the computation node to execute the 'particle' program in the 'p_conf' file. p_conf pbg2010.naregi.org (c) Specify the name of the computation node to execute the mediator program in the 'med_conf' file. Specify the same two names as two lines. med_conf png1009.naregi.org png1009.naregi.org (5) Execution Run this command. $./run 7

13 (6) Verification of result Confirm the presence of below two outputs file. (1) mout (2) pout To confirm the contents of these files, compare it with the same file name under "$DEX_ROOT/test/pmtest" directory Confirmation operations using both MPI and the SBC Prepare following things before confirmation. (a) an user account for all calculation nodes (b) environment variable setting in chapter.3 on (a) Prepare the running environment as below. (c) The 'globus-url-copy' command *1 is available between any two computation nodes. *1 One of the command of the Globus Tooklit. (1) Login Login to a terminal as the user. Then confirm the DEX_ROOT environment variable is exist. If it is not exist, then set up it into the shell profile (.profile or other) of the user and then login again. And add the $DEX_ROOT/bin in the PATH variable in the shell profile. ( sh or bash ) PATH=$DEX_ROOT/bin:$PATH (2) Making a copy of the test program Make a copy of the test program "$DEX_ROOT/test/pmtest_is" into suitable directory. coping into ~/work $ cp -r $DEX_ROOT/test/pmtest_is ~/work (3) Compilation of the test program Change the current directory into the copied directory. $ cd ~/work/pmtest_sbc (3-1) Compilation of particle program Compile the mesh program by 'make particle' command when you will execute it on Linux. 8

14 (Linux) $ make particle Compile the mesh program by 'make -f Makefile.aix particle' command when you will execute it on AIX. (AIX) $ make -f Makefile.aix particle (3-2) Compilation of mesh program Compile the mesh program by 'make mesh' command when you will execute it on Linux. (Linux) $ make mesh Compile the mesh program by 'make -f Makefile.aix mesh' command when you will execute it on AIX. (AIX) $ make -f Makefile.aix mesh (4) Setup (4-1) Setting of MPI Edit these files. (a) Specify the name of the computation node to execute the 'mesh' program in the 'm_conf' file. (b) Specify the name of the computation node to execute the 'particle' program in the 'p_conf' file. (c) Specify the name of the computation node to execute the mediator program in the 'med_conf' file. Specify the same two names as two lines. (4-2) Setting of the SBC Edit these lines in the 'sbc.conf' file. (a) Second Line: Specify the name of the computation node to execute the 'mesh' program and the path to the communication buffer. It specifies it in the form of the following. MESH file:// the name of the computation node/ the path to the communication buffer (b) Third Line: Specify the name of the computation node to execute the 'particle' program and the path to the communication buffer.. It specifies it in the form of the following. 9

15 PARTICLE file:// the name of the computation node/ the path to the communication buffer (c) Fourth Line: Specify the name of the computation node to execute the mediator program and the path to the communication buffer.. It specifies it in the form of the following. MEDIATOR file:// the name of the computation node/ the path to the communication buffer [BUFFER] MESH file://sig0001.naregi.org//home/user1/work/pmtest_sbc/wk2 PARTICLE file://pbg2010.naregi.org//home/user1/work/pmtest_sbc/wk1 MEDIATOR file://png1030.naregi.org//home/user1/work/pmtest_sbc/wk0 [/BUFFER] [CONNECTION] PARTICLE MEDIATOR GRID_GT MEDIATOR MESH GRID_GT [/CONNECTION] [Explanation] The 'mesh' program changes to the setting executed with sig0001.naregi.org in the second line. The communication buffer in that case is assumed to be/home/user1/work/pmtest_sbc/wk2. The 'particle' program changes to the setting executed with pbg2010.naregi.org in the third line. The communication buffer in that case is assumed to be/home/user1/work/pmtest_sbc/wk1. The mediator program changes to the setting executed with pbg1030.naregi.org in the fourth line. The communication buffer in that case is assumed to be/home/user1/work/pmtest_sbc/wk0. (5) Execution [Note] The programs in (5-1), (5-2) and (5-3) do not run independently. They will exchange the data with each other in execution. So do these operations consecutively. (5-1) Execution of the mesh program (a) Log in the computation node to execute the mesh program. $ rlogin sig0001.naregi.org (b) Create your Globus credential by the 'grid-proxy-init' command. (c) Go to the directory the test program is stored. $ cd ~/work/pmtest_sbc (d) Execute the following command. 10

16 $./run.mesh >log.mesh & (5-2) Execution of the particle program (a) Log in the computation node to execute the particle program. $ rlogin pbg2010.naregi.org (b) Create your Globus credential by the 'grid-proxy-init' command. (c) Go to the directory the test program is stored. $ cd ~/work/pmtest_sbc (d) Execute the following command. $./run.particle >log.particle & (5-3) Execution of the mediator program (a) Log in the computation node to execute the mediator program. $ rlogin png1009.naregi.org (b) Create your Globus credential by the 'grid-proxy-init' command. (c) Go to the directory the test program is stored. $ cd ~/work/pmtest_sbc (d) Execute the following command. $./run.mediator (6) Verification of result (6-1) Verification of the output file Confirm the presence of below two outputs file. (1) mout (2) pout To confirm the contents of these files, compare it with the same file name under "$DEX_ROOT/test/pmtest" directory. (6-2) Confirm the log of the SBC communication Confirm that the SBC communication log files 'wk?/_sbclog*' contain the text 'SBC finish' at last line. 11

17 $ tail -1 wk?/_sbclog* ==> wk0/_sbclog_2_0 <== MEDIATOR(0) SBC finish time= ==> wk0/_sbclog_2_1 <== MEDIATOR(1) SBC finish time= ==> wk1/_sbclog_0_0 <== PARTICLE(0) SBC finish time= ==> wk2/_sbclog_1_0 <== MESH(0) SBC finish time= $ (7) How to clean up When your job is terminated as 'Exception' by configuration mistake or other reasons, you should do the following clean up operations before execute the job again. (7-1) Confirmation of remained processes Login to the calculation node that may be executed the particle or mesh program, then confirm that there are no remained processes of particle and mesh program. $ rlogin png1038.naregi.org $ ps x (7-2) Kill the remained processes Kill the remained processes if exists. kill the process which process id is $ kill (7-3) Clean up the SBC communication buffers Remove files under the SBC communication buffer. And remove also the directories named as _sbc* if these are exist. 12

18 $ rm wk?/* $ rm r wk?/_sbc* 2.8. Note Partial Insallation Under the environment without SBC If you plan to install the Mediator under the environment without SBC or any user program aren't executed with SBC, then you can do the partial installation of the Mediator. To do the partial installation, do the operations in this chapter according to following rules. (1) You need not prepare Java and related environment variables as JAVA_HOME (2) Remove or rename the file both $DEX_ROOT/config.sbc and $DEX_ROOT/config.is before the operations in section 2.3. (3) Confirm 'bin', 'lib' and 'include' directory contains only these files after the operations of section 2.3. bin: mediator include: dexlibrary.h dex_msgtag.h dpp_define.h dpp_extern.h dpr_bucket.h dpr_define.h dpr_extern.h dpr_finddest.h dpr_shlibmng.h version.h lib: libdex.a 13

19 Under the environment without the NAREGI Middleware If you plan to install the Mediator under the environment without the NAREGI Middleware or any user program aren't executed on the NAREGI Workflow Tool, then you can do the partial installation of the Mediator. To do the partial installation, do the operations in this chapter according to following rules. (1) You need not prepare Java and related environment variables as JAVA_HOME (2) Remove or rename the file $DEX_ROOT/config.is before the operations in section 2.3. (3) Confirm 'bin', 'lib' and 'include' directory contains only these files after the operations of section 2.3. bin: mediator mediator_s include: dexlibrary.h dex_msgtag.h dpp_define.h dpp_extern.h dpr_bucket.h dpr_define.h dpr_extern.h dpr_finddest.h dpr_shlibmng.h SBC.h SBC_proto.h version.h lib: libdex.a libdex_s.a 14

20 Chapter 3. Providing the information of the running environment These environment variables should be provided by the administrator of the Mediator to the user. (1) DEX_ROOT Specify the path to the Installation directory of the Mediator. (2) SBC_ROOT Specify the path to the Installation directory of the SBC. (3) MPIR_HOME, MPIROOT Specify the path to the installation directory of the GridMPI. (4) GLOBUS_LOCATION Specify the path to the installation directory of the Globus Toolkit. (5) JAVA_HOME Specify the path to the installation directory of the JAVA. (6) PATH Prepend $JAVA_HOME/bin and $MPIR_HOME/bin. 15

21 Chapter 4. Start/Stop and Maintenance The end user starts and stops the user programs with his/her permission, so there are no operations to start/stop anything for the administrator Rebuilding with upgrading the prerequisite softwares When the GridMPI or the SBC is upgraded, rebuilding ( = recompiling and re-linking) the Mediator and/or its applications may be required. Necessity of upgrading is depends on the influence to the applications of GridMPI or the SBC by its upgrading. (1) Case that the SBC need to be recompiled and re-linked When an application of GridMPI or the SBC need to recompile and re-link, the Mediator should be rebuild. An application using both the SBC and the Mediator should be recompile and re-link. (2) Case that the SBC need to be re-linked When an application of GridMPI or the SBC need to re-link, the Mediator should be rebuild. An application using both the SBC and GridMPI should be re-link. To rebuild the SBC, do the following operations. # cd $DEX_ROOT # make clean # make Chapter 5. Uninstallation of the Mediator 5.1. Before uninstallation If the mediator process (es) is/are left on a host, which share the mediator installation directory by NFS, Uninstallation could fail to remove the mediator installation directory. Kill all the mediator processes before uninstallation Unistallation operation Remove the items under the installation directory and the directory where you installed the mediator Verify uninstallation Confirm 'DEX_ROOT' is does not exists. 16

22 Chapter 6. Upgrade installation of the Mediator To upgrade the mediator, uninstall old version first according to chapter 5, then install the latest version according to this document into the same installation directory as old version's. Or if you want to keep old installation, make another installation directory and install the latest version into it. 17