Parameter searches and the batch system

Transcription

1 Parameter searches and the batch system Scientific Computing Group Parameter searches and the batch system Scientific Computing Group 1st of October

2 Contents 1 Parameter searches 2 Test programs 3 Job arrays 4 Environment variables 5 Batch script generating program Parameter searches and the batch system Scientific Computing Group 1st of October

3 Parameter searches lots of program runs each with different parameters individual runs take in the order of hours or days a.k.a. parameter study e.g. Monte Carlo simulation; multivarate function optimisation Batch system options: job arrays (#PBS -t) shell script with environment variables (#PBS -v) custom script or program which generates the required batch scripts and optionally submits them more info: $ man qsub Parameter searches and the batch system Scientific Computing Group 1st of October

4 Types of parameter searches Jobs using random values as input Very easy, just run lots of jobs e.g. to run 1000 jobs: qsub -t simulation.qsub Search over one or more specific parameter values Use a job array to initialise the parameter values Use environment variables to set the parameter values Use a script to create batch jobs with the desired parameter values Parameter searches and the batch system Scientific Computing Group 1st of October

5 Test program: random_num.py 1 #!/usr/bin/env python 2 3 import sys,random 4 5 # print a random number, optionally with seed 6 if len(sys.argv) == 2: 7 seed = float(sys.argv[1]) 8 random.seed(seed) 9 random_num = random.random() 0 print "seed = %d, random number = %g" % (seed, random_num) 1 else: 2 random.seed() 3 random_num = random.random() 4 print "random number = %g" % (random_num) Parameter searches and the batch system Scientific Computing Group 1st of October

6 Test program: vol_calc.py 1 #!/usr/bin/env python 2 3 import sys 4 5 # calculate the volume of the given input parameters 6 xwidth = float(sys.argv[1]) 7 ywidth = float(sys.argv[2]) 8 zwidth = float(sys.argv[3]) 9 0 vol = xwidth*ywidth*zwidth 1 2 print "Volume = %g" % vol Parameter searches and the batch system Scientific Computing Group 1st of October

7 Job arrays Run an array of jobs from one batch script. Advantages: one batch script; many jobs; simple job submission Disadvantages: inflexible; only one parameter can be varied; floating point values not so easily used Job arrays use the -t PBS option. In a script or on the command line. Ranges and/or individual values can be specified. e.g. #PBS -t 1,10, or #PBS -t Produces jobs with the environment variable PBS_ARRAYID containing the relevant array id. The PBS_ARRAYID variable would contain the values 1, 10, and 50 through 100 (inclusive), each of which appearing in a different batch job automatically generated by the batch system. Parameter searches and the batch system Scientific Computing Group 1st of October

8 Job arrays: random number example 1 #!/bin/bash -login 2 #PBS -N job_array_random_example 3 #PBS -M ich@meine.mail.adresse.de 4 #PBS -m ae 5 #PBS -j oe 6 #PBS -l mem=2gb 7 #PBS -l nodes=1:ppn=1 8 #PBS -l walltime=00:30:00 9 #PBS -t 1,10,50, # show which computer the job ran on 12 echo "Job ran on:" $(hostname) # change to working directory 15 cd $PBS_O_WORKDIR # load the relevant modules 18 module load python # set the seed with the array id 21 SEED=$PBS_ARRAYID # run the simulation with a seed 24 python random_num.py $SEED # run the simulation without a predefined seed 27 python random_num.py Parameter searches and the batch system Scientific Computing Group 1st of October

9 Job arrays: volume calculator example 1 #!/bin/bash -login 2 #PBS -N job_array_volume_example 3 #PBS -M ich@meine.mail.adresse.de 4 #PBS -m ae 5 #PBS -j oe 6 #PBS -l mem=2gb 7 #PBS -l nodes=1:ppn=1 8 #PBS -l walltime=00:30:00 9 #PBS -t 1,10,50, # show which computer the job ran on 12 echo "Job ran on:" $(hostname) # change to working directory 15 cd $PBS_O_WORKDIR # load the relevant modules 18 module load python # range over the xwidth values 21 XWIDTH=$PBS_ARRAYID 22 YWIDTH=10 23 ZWIDTH=25 24 echo "XWIDTH = $XWIDTH" 25 echo "YWIDTH = $YWIDTH" 26 echo "ZWIDTH = $ZWIDTH" # run the simulation 29 python vol_calc.py $XWIDTH $YWIDTH $ZWIDTH Parameter searches and the batch system Scientific Computing Group 1st of October

10 Environment variables Pass parameters to script via environment variables. Advantages: one batch script; control multiple variables Disadvantages: batch jobs submitted with script; job submission more complex than with job arrays Use the #PBS -v <variable_name_list> option. 1 $ export XWIDTH=10 2 $ export YWIDTH=5 3 $ for ZWIDTH in $(seq 1 5) 4 do 5 export ZWIDTH 6 qsub env_var_example.qsub 7 done Parameter searches and the batch system Scientific Computing Group 1st of October

11 Environment variables: example 1 #!/bin/bash -login 2 #PBS -N job_array_example 3 #PBS -M ich@meine.mail.adresse.de 4 #PBS -m ae 5 #PBS -j oe 6 #PBS -l mem=2gb 7 #PBS -l nodes=1:ppn=1 8 #PBS -l walltime=00:30:00 9 #PBS -v XWIDTH,YWIDTH,ZWIDTH # show which computer the job ran on 12 echo "Job ran on:" $(hostname) # change to working directory 15 cd $PBS_O_WORKDIR # load the relevant modules 18 module load python # print the variables just to see what was used 21 echo "XWIDTH = $XWIDTH" 22 echo "YWIDTH = $YWIDTH" 23 echo "ZWIDTH = $ZWIDTH" # run the simulation 26 python vol_calc.py $XWIDTH $YWIDTH $ZWIDTH Parameter searches and the batch system Scientific Computing Group 1st of October

12 Batch script generating program Write a program or script which generates separate batch scripts each containing the required input parameters. Advantages: most flexible option; user has complete control; floating point values can be used directly Disadvantages: very easy to make mistakes; more programming experience required Parameter searches and the batch system Scientific Computing Group 1st of October

13 Batch script generating program: example 1 #!/usr/bin/env perl 2 3 use warnings; 4 use strict; 5 6 my $xwidth = 2; 7 my $zwidth = 3.5; 8 = qw( ); 9 for my $ywidth ) { 10 my $batch_script_name = "prog_generated_example_$ywidth.qsub"; 11 open my $fh, ">", $batch_script_name or die "$!"; 12 print $fh get_batch_script_text($xwidth, $ywidth, $zwidth); 13 close $fh; # optionally submit the script 16 #qx{qsub $batch_script_name}; 17 } # subroutine to return the required batch script text 20 sub get_batch_script_text { 21 my $xwidth = shift; 22 my $ywidth = shift; 23 my $zwidth = shift; my $batch_script_text = <<"EOF"; 26 #!/bin/bash -login 27 #PBS -N job_array_example 28 #PBS -M ich\@meine.mail.adresse.de 29 #PBS -m ae 30 #PBS -j oe 31 #PBS -l mem=2gb 32 #PBS -l nodes=1:ppn=1 33 #PBS -l walltime=00:30: # show which computer the job ran on 36 echo "Job ran on:" \$(hostname) # change to working directory 39 cd \$PBS_O_WORKDIR # load the relevant modules 42 module load python # run the simulation 45 python vol_calc.py $xwidth $ywidth $zwidth 46 EOF 47 return $batch_script_text; 48 } Parameter searches and the batch system Scientific Computing Group 1st of October

14 Batch script generating program: example 1 #!/usr/bin/env perl 2 3 use warnings; 4 use strict; 5 6 my $xwidth = 2; 7 my $zwidth = 3.5; 8 = qw( ); 9 for my $ywidth ) { 10 my $batch_script_name = "prog_generated_example_$ywidth.qsub"; 11 open my $fh, ">", $batch_script_name or die "$!"; 12 print $fh get_batch_script_text($xwidth, $ywidth, $zwidth); 13 close $fh; # optionally submit the script 16 #qx{qsub $batch_script_name}; 17 } # subroutine to return the required batch script text 20 sub get_batch_script_text { 21 my $xwidth = shift; 22 my $ywidth = shift; 23 my $zwidth = shift; my $batch_script_text = <<"EOF"; 26 #!/bin/bash -login 27 #PBS -N job_array_example 28 #PBS -M ich\@meine.mail.adresse.de 29 #PBS -m ae 30 #PBS -j oe 31 #PBS -l mem=2gb 32 #PBS -l nodes=1:ppn=1 33 #PBS -l walltime=00:30: # show which computer the job ran on 36 echo "Job ran on:" \$(hostname) # change to working directory 39 cd \$PBS_O_WORKDIR # load the relevant modules 42 module load python # run the simulation 45 python vol_calc.py $xwidth $ywidth $zwidth 46 EOF 47 return $batch_script_text; 48 } Parameter searches and the batch system Scientific Computing Group 1st of October

15 Don t run too many jobs! WARNING: running thousands of jobs can overload the batch system. Consequently, your jobs might not run at all. Use the advice in these slides with utmost caution! Parameter searches and the batch system Scientific Computing Group 1st of October

16 Thanks! Thank you! :-) Parameter searches and the batch system Scientific Computing Group 1st of October