Introduction to parallel computing

Transcription

1 Introduction to parallel computing using R and the Claudia Vitolo 1 1 Department of Civil and Environmental Engineering Imperial College London Civil Lunches,

2 Outline 1 Parallelism What s parallel computing? 2 3

3 Outline Parallelism What s parallel computing? 1 Parallelism What s parallel computing? 2 3

4 Ideal case: Master/Slave Model What s parallel computing? Start with a simple problem: Divide jobs among slave processes and collect results: Ideally the process becomes N times faster with N slaves!

5 What s parallel computing? A more realistic picture Jobs vary in complexity Machines vary in speed/load Communication takes times Dividing up jobs and collecting results takes time

6 What s parallel computing? Parallel computing Parallel code is convenient only when: the code contains independent tasks the processing time of each task is much longer then communication time. This is usually true when working on a single machine but not always true when working on a network. Useful tip: Before embarking in complicated parallel coding, make sure this is beneficial!

7 Outline Parallelism What s parallel computing? 1 Parallelism What s parallel computing? 2 3

8 What s parallel computing? Explicit Parallelism: the user controls the parallelization (execution more efficient, but code development more complicated) Implicit Parallelism: the system abstracts it away (execution and debug less efficient, but easier code development) We will look only at Explicit Parallelism using a single machine, a cluster of machines and supercomputers.

9 By default R runs in sequential mode but can also run in parallel mode using additional packages. The most used packages are snow & snowfall.

10 R packages: Snow and Snowfall snow (Simple Network Of Workstations) is the package that implements parallel computing in R snowfall is a wrapper package for snow that diminishes the learning curve a multi-core system standard for recent computers: dual-core, quad-core, or cluster of machines with a common mounted disk space (home directory) Assumption: Cluster is UNIX-based

11 Snowfall functions sfinit: initialize the cluster (machines and number of CPUs) sflibrary: libraries to call on each node sfsource: files to source on each node sfexport: objects to export to each node sfexportall: export all objects in workspace to each node

12 Example As an example, let s write a script to: 1 Assign a constant: a = 3 2 Create a vector of elements (e.g. sequence from 1 to ) 3 Sum each element of the vector and a The result is the vector: (4,5,6,...,100003)

13 Outline 1 Parallelism What s parallel computing? 2 3

14 Sequential Run R console a <- 3 system.time(sapply(1:100000, sum, a)) Processing time: 39 seconds

15 Parallel Run R console library( snowfall ) sfinit(parallel=true, cpus=4, type="sock") a <- 3 sfexport("a") system.time(sfsapply(1:100000, sum, a)) sfstop() Processing time: 0.58 seconds

16 Outline 1 Parallelism What s parallel computing? 2 3

17 If you have more machines available (hosts), add the list of hosts and the type of connection to the initialization settings: R console sfinit(sockethosts=c("localhost","machine1","machine2"), parallel=true, cpus=4, type="sock") Note: type="sock" means that we are connecting via SOCKET but there are other methods (MPI,NWS,PVM).

18 Outline 1 Parallelism What s parallel computing? 2 3

19 Supercomputers From the Wikipedia article on Linux adoption of supercomputers: If you want to use supercomputers, you need to learn UNIX/LINUX.

20 Description of the ICL The College has a central computer room, specially designed to support High Performance Computing (HPC) The total quantity of CPU resources available across the systems is now over 10,000 cores. Many modules are available (to run R, Matlab and much more) The system is Unix-based (only command line) More info: highperformancecomputing

21 Access and Use Access via SSH (using your college credentials) your console ssh login.cx1.hpc.ic.ac.uk Activate R using the command HPC console module load R/2.15 intel-suite/2012 copy across all the input files and scripts you need. write a batch file to specify: CPUs, memory, walltime, scriptfile, logfile, etc.

22 Example of Batch File Each line starting with R CMD BATCH will be executed by one core

23 Queing system The HPC system uses the Portable Batch System (PBS) to manage the execution of jobs on the compute resources. All jobs must be run through the queuing system and not directly from the command line. So, when your batch file is ready... Submit your jobs with the command qsub Check the status of your jobs with the command qstat Cancel the execution of a job with the command qdel HPC console qsub BatchFile.sh qstat qdel 92200

24 Code Parallelization can help, but not all the times. R is supposed to work in a sequential mode but there are some packages to overcome this limitation. HPC service, thousands of processors at your fingertips!

25 Appendix For Further Reading For Further Reading I J. Knaus and C. Porzelius Tutorial: Parallel computing using R package snowfall. Statistical Computing 2009, Reisensburg. Altair PBS Professional User s Guide.