Grid Examples. Steve Gallo Center for Computational Research University at Buffalo

Transcription

1 Grid Examples Steve Gallo Center for Computational Research University at Buffalo

2 Examples COBALT (Computational Fluid Dynamics) Ercan Dumlupinar, Syracyse University Aerodynamic loads on helicopter rotors Virial (Chemistry) Andrew Schultz (University at Buffalo) Virial coefficients and applications to thermodynamic properties and molecular clustering

3 Traditional Job Workflow Identify required files Submit job (qsub) Stage data and executables to compute cluster (scp, cp) Query resource for job completion (qstat) Retrieve results (scp, cp) Clean up (rm)

4 Grid Job Workflow Identify required files and gather in one location Stage data and executables to grid resource (globus-url-copy) Submit job (globus-job-submit) Query resource for job completion (globus-jobstatus) Retrieve results (globus-url-copy) Clean up (globus-job-run)

5 COBALT nysgrid.job script Submitted to resource manager (PBS) Set environment Create input file Run CoMPIRUN script CoMPIRUN script Create working directory Copy input files into working directory Runs executable using MPI task launcher

6 COBALT qstat License Server PBS CoMPIRUN cobalt.linux.dp qsub nysgrid.job Create Input mpirun cobalt.linux.dp cobalt.linux.dp Results cobalt.stdin Co_workspace Co_stdin bc_file.bc grid_file.grd

7 COBALT Why can t I just submit my script to the grid? Data files need to be staged to a remote resource Grid job managers run MPI task launcher for you using the script / executable submitted Grid Job Manager nysgrid.job globus-job-submit nysgrid.job mpirun nysgrid.job nysgrid.job

8 COBALT Rework COBALT submission into 2 scripts create_job_package script Generate input file Create job directory to manage files Copy data files into job directory submit_job script Stage job package Submit job Poll for status Retrieve Results

9 COBALT Local create_job_package submit_job Create Input Create Package Push Job Package Submit Job Poll Job Status Pull Job Results cobalt.stdin Co_workspace bc_file.bc Co_stdin grid_file.grd globus-url-copy (scp, cp) globus-job-submit (qsub) globus-job-status (qstat) globus-url-copy (scp, scp) License Server Remote Remote Storage Grid Job Manager cobalt cobalt mpirun cobalt cobalt

10 COBALT Job Submission Script Single script that performs workflow All work will be done in directory $OSG_APP/VO/APPLICATION Arguments: num procs, walltime, work dir, job name Assumptions OSG_APP=/san/scratch/grid/grid-tmp/grid-apps OSG_DATA=/san/scratch/grid/grid-tmp/grid-data OSG_HOST=u2-grid.ccr.buffalo.edu VO=nysgrid JOB_POLL_INTERVAL=60 QUEUE_NAME=

11 Create Job Package Create all COBALT input in a single script Create working directory Create input file Copy all data files to a single directory create_job_package.sh ${NPROCS} ${INPUT_DIR} ${CASE_NAME}

12 Staging Data REMOTE_URL=gsiftp://${OSG_HOST}${OSG_DATA}/${VO}/cobalt/cases/${CASE}/ SOURCE_URL=file://${INPUT_DIR}/grid_files/ globus-url-copy -cd -v -r ${SOURCE_URL} ${REMOTE_URL}

13 Submit Job queue=${queue_name:+"-q $QUEUE_NAME"} output=`globus-job-submit ${OSG_HOST}/jobmanager-pbs ${queue} -np ${NPROCS} -maxtime ${WALLTIME} -x '&(jobtype=mpi) -dir ${OSG_DATA}/${VO}/cobalt/cases/${CASE} -env COBALT_LIC="${COBALT_LIC} ${OSG_APP}/${VO}/cobalt/bin/cobalt.linux.dp 2>&1` if [ $? -eq 0 ]; then echo job_descriptor=${output} echo "Job descriptor: ${job_descriptor}" else echo "Error submitting job: ${output}" globus-job-run ${OSG_HOST}/jobmanager-fork /bin/rm -rf ${OSG_DATA}/${VO}/cobalt/cases/${CASE} 2>&1 exit 1 fi

14 Poll For Status status=`globus-job-status ${job_descriptor} 2>&1` echo " Status "`date +%r`" = ${status}" while [ "FAILED"!= "${status}" -a "DONE"!= "${status}" ]; do sleep ${JOB_POLL_INTERVAL} proxy_seconds=`grid-proxy-info -timeleft` proxy_hrs_left=`echo "$proxy_seconds 3600 / p" dc` echo " Proxy expires in ${proxy_hrs_left} hours" status=`globus-job-status ${job_descriptor} 2>&1` echo " Status "`date +%r`" = ${status}" done

15 Retrieve Results # Retrieve the job standard output and error globus-job-get-output -out ${job_descriptor} globus-job-get-output -err ${job_descriptor} # Copy back the results and clean up globus-url-copy -cd -v -r ${REMOTE_URL} ${RESULT_URL} globus-job-clean -f ${job_descriptor} output=`globus-job-run ${OSG_HOST}/jobmanager-fork /bin/rm -rf ${OSG_DATA}/${VO}/cobalt/cases/${CASE} 2>&1` if [ $? -eq 0 ]; then fi echo "Output: ${output}"

16 Script Output :>./submit_job.sh 2 30 ~/projects/grid_cobaltcfd/cases/ramp ramp Number of processors: 2 Case: 'ramp' Input directory:'/san/user/smgallo/u2/projects/grid_cobaltcfd/cases/ramp' Stage out job files Source: file:///san/user/smgallo/u2/projects/grid_cobaltcfd/cases/ramp/grid_files/ Dest: gsiftp://u2-grid.ccr.buffalo.edu/san/scratch/grid/grid-tmp/grid-data/nysgrid/cobalt/cases/ramp/ Co_stdin grid_file.grd bc_file.bc viz/ ramp.cnvrg/ Submit grid job Job descriptor: Status 04:44:58 PM = PENDING Status 04:45:21 PM = PENDING Status 04:45:44 PM = PENDING Status 04:46:09 PM = DONE Stdout globus-job-get-output -out #############################PBS Prologue############################## PBS prologue script run on host c15n32 at Wed May 14 16:47:03 EDT 2008 PBSTMPDIR is /scratch/ bono.ccr.buffalo.edu #############################PBS EPILOGUE############################## PBS epilogue script run on host c15n32 at Wed May 14 16:47:05 EDT 2008 for 2 processes. Cleaning up processes on node c15n32 for user nysgrid Removing /scratch/ bono.ccr.buffalo.edu on node c15n32 Cleaning up processes on node c15n31 for user nysgrid Removing /scratch/ bono.ccr.buffalo.edu on node c15n31 #############################PBS EPILOGUE############################## Retrieve results to file:///san/user/smgallo/u2/projects/grid_cobaltcfd/cases/ramp/results/

17 Hands On Virial (Chemistry) Andrew Schultz, University at Buffalo

18 Hands-On Log onto submit host ssh mygatekeeper.edu Source setup scripts. /opt/grid/setup.sh Generate proxy grid-proxy-init Download example wget

19 Hands-On Adapt COBALT scripts for Virial Modify COBALT submission scripts to run virial example. There is no single correct solution No MPI required

20 Conclusion Questions? Other Examples?