An Introduction to Cluster Computing Using Newton Jason Harris and Dylan Storey March 25th, 2014 Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 1 / 26
Workshop design. Slides are going to be up here mostly for you to stare at and so that I remember what I m supposed to be talking about. Mostly you will be getting onto newton and submitting a few trivial jobs so that you ll be more comfortable in the future. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 2 / 26
What is a cluster? Simply put, a cluster is a number of computers that are connected together over a LAN that act like one large computer by being able to talk with each other. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 3 / 26
When should I use the cluster? You should look to using the cluster when computing becomes a bottle neck. You have many of the same processes to run serially. A few intensive processes to run. A process that needs to use more CPUs than you have on your current computer. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 4 / 26
Why should I be using the cluster? Free support Technical advising You don t need to buy your own hardware. You don t need to maintain your own hardware and software. Basically, you get a resource without having to pay for and maintain it yourself. None of this is free, but as a student you do get non priority access to the equipment and individuals can invest in the cluster to get more computing resources. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 5 / 26
Newton Specs In total Newton has: 4,200 cores 8TB of RAM 50TB of Lustre Storage 50TB of Network Attached Storage 50TB off site backup Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 6 / 26
Access to Newton All students and faculty have access to this resource. How much priority you have depends on money. To register : Go To: https://newton.utk.edu/manager/ Register for an account. Wait If you remember everyone who didn t know what Newton was gave me their UT username several months ago. Your accounts are already made. Some of you didn t write legibly you ll need to do this yourself. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 7 / 26
Connecting to Newton Connecting to Newton is a trivial matter Linux/UNIX : type ssh username@newlogin.newton.utk.edu into your terminal PuTTy : Hostname newlogin.newton.utk.edu Username NetID Password NetID password Remote port to connect to 22 A successful connection will open up a bash shell for you to use. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 8 / 26
Linux Refresher Linux Bash Shell Cheat Sheet Basic Commands Basic Terminal Shortcuts Basic file manipulation CTRL L = Clear the terminal cat <filename> = show content of file CTRL D = Logout (less, more) SHIFT Page Up/Down = Go up/down the terminal head = from the top CTRL A = Cursor to start of line -n <#oflines> <filename> CTRL E = Cursor the end of line CTRL U = Delete left of the cursor tail = from the bottom CTRL K = Delete right of the cursor -n <#oflines> <filename> CTRL W = Delete word on the left CTRL Y = Paste (after CTRL U,K or W) mkdir = create new folder TAB = auto completion of file or command mkdir mystuff.. CTRL R = reverse search history mkdir mystuff/pictures/..!! = repeat last command CTRL Z = stops the current command (resume with fg in foreground or bg in background) cp image.jpg newimage.jpg = copy and rename a file Basic Terminal Navigation cp image.jpg <foldername>/ = copy to folder cp image.jpg folder/sameimagenewname.jpg ls -a = list all files and folders cp -R stuff otherstuff = copy and rename a folder ls <foldername> = list files in folder cp *.txt stuff/ = copy all of *<file type> to folder ls -lh = Detailed list, Human readable ls -l *.jpg = list jpeg files only mv file.txt Documents/ = move file to a folder ls -lh <filename> = Result for file only mv <foldername> <foldername2> = move folder in folder mv filename.txt filename2.txt = rename file cd <foldername> = change directory mv <filename> stuff/newfilename if folder name has spaces use mv <foldername>/.. = move folder up in hierarchy cd / = go to root cd.. = go up one folder, tip:../../../ rm <filename>.. = delete file (s) rm -i <filename>.. = ask for confirmation each file du -h: Disk usage of folders, human readable rm -f <filename> = force deletion of a file du -ah: files & folders, Human readable rm -r <foldername>/ = delete folder du -sh: only show disc usage of folders touch <filename> = create or update a file pwd = print working directory ln file1 file2 = physical link man <command> = shows manual (RTFM) ln -s file1 file2 = symbolic link Courtesy of cli.learncodethehardway.org Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 9 / 26
Transfering Files Of course logging into Newton isn t enough. You ll need to get data to and from the cluster as well. There are a number of tools for this that exist. Command line: scp - Secure Copy. Its just like cp but over SSH. wget - gets content from web servers. (HTTP,HTTPS,FTP) rsync - Linux/Unix synchronization utility. Good for lots of files or if only a few changes are being made between files. GUI: FileZilla, Drag and drop file queuing and transfer program. Fugu, another popular file transfer interface for Mac OSX. Most Linux file managers have built in functionality for this as well. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 10 / 26
Screen Shots, Because it Happened Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 11 / 26
Screen Shots, Because it Happened Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 12 / 26
Screen Shots, Because it Happened Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 13 / 26
Screen Shots, Because it Happened Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 14 / 26
Screen Shots, Because it Happened Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 15 / 26
Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 16 / 26
The Sun Gride Engine Once you connect to the cluster you ll be able to do a few basic things like: edit files compile code run basic programs But keep in mind that you re resource limited and if you try to run any resource intensive jobs you ll anger the admin gods. In order to use the cluster effectively you ll need to submit jobs to the Grid Engine. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 17 / 26
Submitting a Job to SGE This process goes as follows: Create a file that tells the scheduler what you need and how longtable you ll need it for. Grid Engine finds the resources on the cluster for you to use and reserves them. Once all of the resources are collected the Grid Engine runs your job. While the job is running, the Grid Engine collects stderr and stdout and writes them to log files. Upon completion, resources are freed. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 18 / 26
Queues on the Newton Cluster The queue you request tells Newton how long you want a job to run for. If your job runs longer than the queue you re in, Newton will kill the job before it finishes so make sure to pick an appropriate queue. Queue selection also changes the number of resources you can request for a job as well. short- 256 cpus, 2 hours max run time medium - 128 cpus, 24 hours max run time long - priority groups only Don t just put everything in the medium queue, it will effect how fast your jobs are executed. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 19 / 26
A Basic Job, Single CPU! Create your first SGE file by opening a new basic text document on your computer Type the following lines into the file and save. #$ N Test #$ cwd #$ q s h o r t uname a s l e e p 30 Copy this file to your home directory on Newton (/home/username/) Submit your job by typing the following : qsub Basicjob.sge Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 20 / 26
Monitoring your Job(s) Type qstat to get a list of your currently running or submitted jobs. You can check the state column to find if your job is queued and waiting (qw), running(r), or in error (e). You can also check the direct status of running jobs by looking at output generated during the run. These file are named by the Name you submitted in your script, followd by an e or o for each stream. Pro Tip : Typing qstat repeatedly is annoying, try typing watch qstat and it will check the command regularly for you! Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 21 / 26
Requesting More Resources Obviously if you re using this resource you probably have software capable of using multiple CPUs. In order to do this you need to ask for those CPUs before you can use them. Pro Tip: It s also important to tell the program you re running how many CPUs it is allowed to use in order to make sure you re not using resources that aren t allocated to you. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 22 / 26
Multiple CPUs on a Single Node If you want to run a job in which you ll be using multiple CPUs on a single node your job file might look something like this: #$ pe t h r e a d s 4 #$ N Test #$ cwd #$ q s h o r t / l u s t r e /home/ d s t o r e y / W o r k s h o p F i l e s / t h r e a d s. p l You ll notice a new line requesting 2 threads. This tells the scheduler to find you two CPUs on the same node. The files and an example script for this available at : /lustre/home/dstorey/threads/ Remember, you can only ask for resources that exist. If no nodes exist with the number of threads you request your job can t run. Try out the qhost command and see what it says! Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 23 / 26
Trivially Parallel Jobs Sometimes you need to execute the same job repeatedly. The Grid Engine has a specific way of managing these tasks called array jobs. The files and an example script for this available at : /lustre/home/dstorey/array/ To use an array job: Break your data in to x number of files with the names having sequential integers. Submit your job file with the option #$-t 1-x The job then accesses an enviroment variable $SGE TASK ID to determine the file to use. Your Job File should look something like this : #$ N Test #$ cwd #$ q s h o r t #$ t 1 100 / l u s t r e /home/ d s t o r e y / W o r k s h o p F i l e s / Array / a v e r a g e. p l $SGE TASK ID. dat Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 24 / 26
Trivially Parallel Jobs I m sure you can already see that this is not always a feasible method for submitting jobs. Thankfully you can use scripts to write and submit job files for you. I ve included and example script for to do this if you need any ideas(/lustre/home/dstorey/workshopfiles/generate and submit jobs.pl). Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 25 / 26
Other Methods There are other methods of requesting resources: MPI jobs request many CPUs from many Nodes. Hybridized jobs request specific number of CPUs on a node across a number of Nodes. This is all I m going to say about these. Know that they exist and you may want to use them one day. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 26 / 26
Modules on Newton The module system provides you with a way of managing all of the enviromental variables in your session. Each module corresponds to a specific application. Use the module system with the module command. For example if you wanted to submit a job that uses the signalp module and calls signalp your sun grid engine would look something like this: #$ N Test #$ cwd #$ q s h o r t module l o a d s i g n a l p s i g n a l p m y p e p t i d e. f a s You can get a complete list of modules available by typing module avail. Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 27 / 26