LaPalma User's Guide
|
|
- Maximilian Lewis
- 5 years ago
- Views:
Transcription
1 LaPalma User's Guide Copyright 2018 Table of contents 1. Introduction System Overview Online Documentation Man pages Connecting to LaPalma Login node Transferring files Graphical applications File Systems Root Filesystem Lustre Filesystem Local Hard Drive Running Jobs Queues (QOS) Submitting jobs SLURM commands Job directives Examples Software Environment Modules C Compilers Distributed Memory Parallelism Shared Memory Parallelism Automatic Parallelization bit addressing Optimization FORTRAN Compilers Distributed Memory Parallelism Shared Memory Parallelism bit Addressing Optimization Debuggers Software in LaPalma Getting help FAQ's Acknowledgements...14 APPENDIX...16 A. SSH...16 A.1. Generating an SSH Key pair on Linux...16 A.1.1. Using the ssh-agent in Linux...17 A Using the ssh-agent in a Terminal Session...17 A Using ssh-agent in an X Session...17 A.2. Generating an SSH Key pair on Windows...17 A.2.1. Using the ssh-agent in Windows
2 1. Introduction This user's guide for the LaPalma Supercomputer (v3) is intended to provide the minimum amount of information needed by a new user on this system. As such, it assumes that the user is familiar with many of the standard aspects of supercomputing as the Unix operating system. We hope you can find most of the information you need to use our computing resources: from applications and libraries to technical documentation about LaPalma; how to include references in publications and so on. Please read carefully this document and if any doubt arises do not hesitate to contact our support group at res_support@iac.es 2. System Overview LaPalma comprises 252 IBM dx360 M4 compute nodes. Every node has sixteen E cores at 2.6 GHz, running Linux operating system with 32 GB of memory RAM and 500GB of local disk storage. Two Bull R423 servers are connected to a pair of Netapp E5600 storage systems providing a total amount of 346 TB of disk storage accessible from every node through Lustre Parallel File System. The networks that interconnect the LaPalma are: Infiniband Network: High bandwidth network used by parallel applications communications. Gigabit Network: Ethernet network used by the nodes to mount remotely their root file system from the servers and the network over which Lustre works. 3. Online Documentation 3.1. Man pages Information about most commands and software installed is available via the standard UNIX man command. For example, to read about command-name just type on a shell inside LaPalma: usertest@login1:~> man command-name which displays information about that command to the standard output. If you don't know the exact name of the command you want but you know the subject matter, you can use the -k flag. For example: usertest@login1:~> man -k compiler This will print out a list of all commands whose man-page definition includes the word 'compiler'. Then you could execute the exact man command line to know about the exact command you were looking for. Just to know more about the man command itself, you could also type: usertest@login1:~> man man 4. Connecting to LaPalma You must use Secure Shell (ssh) tools to login into or transfer file into LaPalma. We do not accept incoming connections from protocols as telnet, ftp, rlogin, rcp, or rsh commands. Once you are logged into LaPalma you cannot make outgoing connections for security reasons. To get more information about the secure shell version supported and how to get ssh for your system (including windows systems) see Appendix A. 2
3 LaPalma does not support authentication based on user and password, but a key-based authentication mechanism. In order to get access to LaPalma you have to provide us your public ssh key via (res_support@iac.es). Take a look at Appendix B for generating you own public/private key pair. Once you have provided your public ssh key you can get into LaPalma system, connecting to the login node: lapalma1.iac.es Here you have an example of logging into LaPalma from a UNIX environment: localsystem$ ssh -l usertest lapalma1.iac.es Welcome to LaPalma * Please contact res_support@iac.es for questions at any time * User Guide located at /storage/lapalmauserguide.pdf usertest@login1:~> If you are on a Windows system, you need to download and install a Secure Shell client to perform the connection to the machine (See appendix A for more information). Most of these applications are graphical and you will have to fill some information in some of the fields offered, in the field 'Host name' or 'Remote Host name' you will need to introduce: lapalma1.iac.es. After this procedure you may be logged into LaPalma. The first time that you connect to the LaPalma system secure shell needs to interchange some initial information to establish the communication. This information consists of the acceptance of the RSA key of the remote host, you must answer 'yes' or 'no' to confirm the acceptance of this key. If you cannot get access to the system after following this procedure, first consult Appendix A for an extended information about Secure Shell, or you can contact us, (see section 9 to know how to contact us) Login node Once you are connected into the machine, you will be presented with a UNIX shell prompt and you will normally be in your home ($HOME) directory. If you are new to UNIX, you will have to learn the basics before you could do anything useful. The machine in which you will be logged in will be LaPalma (login1). This machine acts as front ends, and are used typically for editing, compiling, preparation/submission of batch executions and as a gateway for copying data inside or outside LaPalma. It is not permitted the execution of cpu-bound programs on this node, if some compilation needs much more cputime than the permitted, this needs to be done through the batch queue system. It is not possible to connect directly to the compute nodes from the login nodes, all resource allocation is done by the batch queue system Transferring files For transferring files to LaPalma from Unix systems, you can use secure copy (scp) or secure ftp (sftp), both tools have the same syntax as the old and insecure tools such as rcp (remote copy) and ftp. As have been said before no connections are allowed from inside LaPalma to the outside world so all scp and sftp commands have to be executed from your local machines and not inside LaPalma. 3
4 Here there are some examples of each of this tools transferring files to LaPalma: localsystem$ scp localfile localsystem$ sftp sftp> put localfile These are the ways to retrieve files from LaPalma to your local machine: localsystem$ scp localdir localsystem$ sftp sftp> get remotefile On a Windows system, most of the secure shell clients comes with a tool to make secure copies or secure ftp's. There are several tools that accomplishes the requirements, please refer to the Appendix A, where you will find the most common ones and examples of use Graphical applications You could execute graphical applications from the login node, to do that the only way is tunnelling all the graphical traffic through the Secure shell connection established. You will need to have an Xserver running on your local machine to be able to show the graphical information. Most of the UNIX flavors have an X server installed by default. In a Windows environment, you will probably need to download and install some type of X server emulator (see appendix A). The second step in order to be able to execute graphical applications is to enable in your secure shell connection the forwarding of the graphical information through the secure channel created. This is normally done adding the -X flag to your normal ssh command used to connect to LaPalma. Here you have an example: localsystem$ ssh -X -l usertest lapalma1.iac.es Welcome to LaPalma * Please contact res_support@iac.es for questions at any time * User Guide located at /storage/lapalmauserguide.pdf usertest@login1:~> For Windows systems, you will have to enable the 'X11 forwarding', that option normally resides on the 'Tunneling' or 'Connection' menu of the client configuration window (see appendix A for further details). 5. File Systems IMPORTANT: It is your responsibility as a user of the LaPalma system to backup all your critical data. NO backup of user data will be done in any of the filesystems of LaPalma. Each user has several areas of disk space for storing files. These areas may have size or time limits, please read carefully all this section to know about the policy of usage of each of these filesystems. There are 3 different types of storage available inside a node: 4
5 Root filesystem: Is the filesystem where the operating system resides Lustre filesystems: Lustre is a distributed networked filesystem which can be accessed from all the nodes Local hard drive: Every node has an internal hard drive Let's see them in detail Root Filesystem The root file system, where the operating system is stored doesn't reside in the node, this is a NFS filesystem mounted from a Network Attached Storage (NAS). As this is a remote filesystem only data from the operating system has to reside in this filesystem. It is NOT permitted the use of /tmp for temporary user data. The local hard drive can be used for this purpose as you could read in section 5.3. Furthermore, the environment variable $TMPDIR is already configured to force the normal applications to use the local hard drive to store their temporary files Lustre Filesystem Lustre is an open-source, parallel file system that can provide fast, reliable data access from all nodes of the cluster to a global filesystem, with a remarkable scale capacity and performance. Lustre allows parallel applications simultaneous access to a set of files (even a single file) from any node that has the Lustre file system mounted while providing a high level of control over all file system operations. These filesystems are the recommended to use with most jobs, because Lustre provides high performance I/O by "striping" blocks of data from individual files across multiple disks on multiple storage devices and reading/writing these blocks in parallel. In addition, Lustre can read or write large blocks of data in a single I/O operation, thereby minimizing overhead. Even though there is only one Lustre filesystem mounted on LaPalma, there are different locations for different purposes: /storage/home: This location has the home directories of all the users. When you log into LaPalma you start in your home directory by default. Every user will have their own home directory to store the executables, own developed sources and their personal data. /storage/projects: In addition to the home directory, there is a directory in /storage/projects for each group of users of LaPalma. For instance, the group iac01 will have a /storage/projects/iac01 directory ready to use. This space is intended to store data that needs to be shared between the users of the same group or project. All the users of the same project will share their common /storage/projects space and it is responsibility of each project manager to determine and coordinate the better use of this space, and how it is distributed or shared between their users. /storage/scratch: Each LaPalma user will have a directory over /storage/scratch, you must use this space to store temporary files of your jobs during its execution. The previouse three locations share the same quota in order to limit the amount of data that can be saved by each group. Since the locations /storage/home, /storage/projects and /storage/scratch are in the same filesystem, the quota assigned is the sum of Disk Projects and Disk Scratch established by the access committee. The quota and the usage of space can be consulted via the quota command: usertest@login1:~> lfs quota -g <GROUP> /storage For example, if your group has been granted the following resources: Disk Projects: 1000GB 5
6 Disk Scrach: 500GB The command quota will report the sum of the two values: lfs quota -g usergroup /storage Disk quotas for grp usergroup (gid 666): Filesystem kbytes quota limit grace files quota limit grace /storage/ The amount of files is limited as well. By default the quota for files is set to file. If you need more disk space or number of files, the responsible of your project has to make a request for this extra space needed, specifying the requested space and the reasons why it is needed. The request can be sent by or any other way of contact to the user support team as it is explained in section 9 of this document. /storage/apps: Over this location will reside the applications and libraries that have already been installed on LaPalma. Take a look at the directories or to section 7 of this document to know the applications available for general use. Before installing any application that is needed by your project, first check if this application is already installed on the system. If some application that you need is not on the system, you will have to ask our user support team to install it. Check section 9 how to contact us. If it is a general application with no restrictions in its use, this will be in stalled over a public directory, that is over /storage/apps so all users on LaPalma could make use of it. If the application needs some type of license and its use must be restricted, a private directory over /storage/apps will be created, so only the required users of LaPalma could make use of this application. All applications installed on /storage/apps will be installed, controlled and supervised by the user support team. This doesn't mean that the users could not help in this task, both can work together to get the best result. The user support can provide his wide experience in compiling and optimizing applications in the LaPalma platform and the users can provide his knowledge of the application to be installed. All that general applications that have been modified in some way from its normal behavior by the project users' for their own study, and may not be suitable for general use, must be installed over /storage/projects or /storage/home depending on the usage scope of the application, but not over /storage/apps Local Hard Drive Every node has a local hard drive that can be used as a local scratch space to store temporary files during executions of one of your jobs. This space is mounted over /scratch directory. The amount of space within the /scratch filesystem varies from node to node (depending on the total amount of disk space available). All data stored in these local hard drives at the compute nodes will not be available from the login nodes. Local hard drive data is not automatically removed, so each job should have to remove its data when finishes. 6. Running Jobs Slurm is the utility used at LaPalma for batch processing support, so all jobs must be run through it. This document provides information for getting started with job execution at LaPalma. In order to keep the login nodes in a proper load, a 10 minutes limitation in the cpu time is set for processes running interactively in these nodes. Any execution taking more than this limit should be carried out through the queue system Queues (QOS) The user's limits are assigned automatically to each particular user (depending on the resources granted by the Access Committee). Anyway you are allowed to use the special queue: "debug" in order to perform some fast short tests. To use the "debug" queue you need to include the #SBATCH --qos=debug directive. Table 1. Queues Queues Max CPUs Wall time limit class_a hours 6
7 class_b hours class_c housr debug min interactive 1 1 hour The specific limits assigned to each user depends on the priority granted by the access committee. Users granted with "high priority hours" will have access to a maximum of 4032 CPUs and a maximum wall_clock_limit of 72 hours. For users with "low priority hours" the limits are 1200 CPUs and 24 hours. If you need to increase these limits please contact the support group. class_a, class_b, class_c: Queues assigned by the access committee and where normal jobs will be executed, no special directive is needed to use these queues. debug: This queue is reserved for testing the applications before submitting them to the 'production' queues. Only one job per user is allowed to run simultaneously in this queue, and the execution time will be limited to 10 minutes. The maximum number of nodes per application is 32. Only a limited number of jobs may be running at the same time in this queue. To use this queue add the directive #SBATCH --qos=debug interactive: Jobs submitted to this queue will run in the interactive (login) node. It is intended to run GUI applications that may exceed the interactive cpu time limit. Note that only sequential jobs are allowed. To use this queue launch the following command from login1: salloc -p interactive 6.2. Submitting jobs A job is the execution unit for the SLURM. A job is defined by a text file containing a set of directives describing the job, and the commands to execute SLURM commands These are the basic commands to submit jobs: sbatch <job_script> submits a job script to the queue system (see below for job script directives). squeue shows all the jobs submitted. scancel <job_id> remove his/her job from the queue system, cancelling the execution of the processes, if they were already running. scontrol show job <job_id> obtains detailed information about a specific job, including the assigned nodes and the possible reasons preventing the job from running. scontrol hold <job_id> sets a lock to the specified job. To release a job, the same command must be run with -r option. 7
8 Job directives A job must contain a series of directives to inform the batch system about the characteristics of the job. These directives appear as comments in the job script, with the following syntax: #SBATCH -directive=<value> Some come directives have a shorter version, you can use both forms: #SBATCH -d <value> Additionally, the job script may contain a set of commands to execute. If not, an external script must be provided with the 'executable' directive. Here you may find the most common directives: #SBATCH -J <name_of_job> The job name that will be appear when the command squeue is executed. This name is established by the user and it is different from the job_id (assigned by the batch system) #SBATCH --qos <queue_name> The queue where the job is to be submitted. Let this field empty unless you need to use debug queue. #SBATCH -t <wall_clock_limit> The limit of wall clock time. This is a mandatory field and you must set it to a value greater than the real execution time for your application and smaller than the time limits granted to the user. Notice that your job will be killed after the elapsed period. Shorter limits are likely to reduce the waiting time. Format is HH:MM:SS or DD-HH:MM:SS #SBATCH -D <pathname> The working directory of your job (i.e. where the job will run). If not specified, it is the current working directory at the time the job was submitted. #SBATCH -o <output_file> The name of the file to collect the standard output (stdout) of the job. It is recommended to use %j in the name of the file, then slurm will automatically include the job Id to avoid overwriting the output files if you submit several jobs. #SBATCH -e <error_file> The name of the file to collect the stderr output of the job. It is recommended to use %j in the name of the file, then slurm will automatically include the job Id to avoid overwriting the error files if you submit several jobs. #SBATCH -N <number_nodes> The number of nodes you are asking for. Bear in mind that each node has 16 cores, so by default each node will execute 16 tasks, one per core. #SBATCH --cpus-per-task=<number> The number of cpus allocated for each task. This is useful for hybrid MPI+OpenMP applications, where each 8
9 process will spawn a number of threads. The number of cpus per task must be an integer between 1 and 16, since each node has 16 cores (one for each thread). #SBATCH --tasks_per_node=<number> The number of tasks allocated in each node. When an application uses more than 1.7 GB of memory per process, it is not possible to have 16 processes in the same node and its 32 GB of memory. It can be combined with the --cpus-per-task to allocate the nodes exclusively. The number of tasks per node must be an integer between 1 and 16 (bear in mind that some cores will stay idle when setting a number lower than 16, so if it is not possible for you to use all 16 available cores, try to minimize the number of cores that will be wasted). #SBATCH -array=ini:end:step This will enable Array Jobs. Array jobs and task generation can be used to run applications over different inputs like you could also do with GREASY in past versions of LaPalma or MareNostrum. It will create as many jobs as you specify from ini to end and the step (step by default is 1). You can get the current task using $SLURM_ARRAY_TASK_ID. There are also a few SLURM environment variables you can use in your scripts: Variable SLURM_JOBID SLURM_NPROCS SLURM_NNODES SLURM_PROCID Meaning Specifies the job ID of the executing job Specifies the total number of processes in the job Is the actual number of nodes assigned to run your job Specifies the MPI rank (or relative process ID) for the current process. The range is from 0-(SLURM_NPROCS-1) SLURM_NODEID Specifies relative node ID of the current job. The range is from 0- SLURM_LOCALID SLURM_NODELIST SLURM_ARRAY_TASK_ID SLURM_ARRAY_JOB_ID (SLURM_NNODES-1) Specifies the node-local task ID for the process within a job Specifies the list of nodes on which the job is actually running Task ID inside the array job Job ID (it will be the same for all array jobs, the same as the SLURM_JOBID of the first task) Examples Example for a sequential job: #!/bin/bash # #SBATCH -J test_mpi #SBATCH -n 1 #SBATCH -t 00:02:00 #SBATCH -o test_serial-%j.out #SBATCH -e test_serial-%j.err #SBATCH -D../serial_binary The job would be submitted using: usertest@login1:~/slurm/serial> sbatch slurm_serial.cmd 9
10 Example for a parallel (MPI) job with 64 tasks (4 nodes): #!/bin/bash # #SBATCH -J test_mpi #SBATCH -N 4 #SBATCH -t 00:30:00 #SBATCH -o test_mpi-%j.out #SBATCH -e test_mpi-%j.err #SBATCH -D. module purge module load gcc module load openmpi mpirun./mpi_binary 7. Software Environment 7.1. Modules All installed software (compilers, applications, numerical and graphical libraries, tools, etc.) can be found at /storage/apps/ or directly /apps. There is a directory for each application (uppercase letters) and then a subdirectory with the installed versions. To use any of this software remember to previously load the environment with modules. Get help % module help List all available software % module avail Load a specific software (default version) % module load gcc Load a specific software and version % module load gcc/7.2.0 List all loaded software % module list Unload a specific software % module unload gcc Unload all software % module purge Change version of software % module load gcc/4.8.5 % module switch gcc/ C Compilers In LaPalma you can find the next C/C++ compilers : gcc /g++ -> GNU Compilers for C/C++, versions and (by default). You can choose the compiler you want to use with modules. % module load gcc % module load gcc/
11 % module load gcc/4.8.5 % man gcc % man g++ All invocations of the C or C++ compilers follow these suffix conventions for input files:.c,.cc,.cpp, or.cxx C++ source file..c C source file.i preprocessed C source file.so shared object file.o object file for ld command.s assembler source file By default, the preprocessor is run on both C and C++ source files Distributed Memory Parallelism MPI compilations will be done using OpenMPI compilers (at this moment no Intel license is available for parallel compilers). There are several versions of OpenMPI, by default will be used. Invoking mpicc enables the program for running across several nodes of the SP. Of course, you are responsible for using a library such as MPI to arrange communication and coordination in such a program. Any of the MPI compilers sets the include path and library paths to pick up the MPI library. % module load openmpi % mpicc a.c -o a.exe Shared Memory Parallelism The GCC C and C++ compilers support a variety of shared-memory parallelism. OpenMP directives are fully supported when using -fopenmp. % gcc -fopenmp -o exename filename.c Automatic Parallelization The GCC compiler will attempt to automatically parallelize simple loop constructs. Use the option -ftreeparallelize-loops=n where N is the number of threads you want to use. % gcc -ftree-parallelize-loops=16 filename.c bit addressing By default all compilers will use the 64 bit addressing Optimization The level optimization that we recommend for LaPalma (E Sandy Bridge processors) is : -O3 -march=native Applications compiled with -march=native might be executed properly only on those machines where they were compiled. 11
12 7.3. FORTRAN Compilers In LaPalma you can find this GCC compilers for Fortran, version or man gfortran Distributed Memory Parallelism The scripts mpifort (OpenMPI) allows to use the MPI calls to get parallelism (there are other scripts like mpif77 or mpif90, but they are just symlinks to the same wrapper. % module load openmpi % mpifort a.f -o a.exe Shared Memory Parallelism OpenMP directives will be fully supported when using -fopenmp. % gfortran -fopenmp -o exename filename.f bit Addressing By default all compilers will use the 64 bit addressing 7.4. Optimization The level optimization that we recommend for LaPalma (Intel E5-2670) is: -O3 -march=native 7.5. Debuggers GDB ( GNU DEBUGGER): /usr/bin/gdb 7.6.Software in LaPalma There is a number of different software installed in LaPalma (compilers, applications, libraries, tools, etc.). Please, use modules to see all installed software. If you need a particular software or version that is not already installed, please, contact us. List all available software % module avail Load a specific software and version % module load gcc/
13 8. Getting help User questions and support are handled at: If you need assistance, please supply us with the nature of the problem, the date and time that the problem occurred, and the location of any other relevant information, such as output files. 9. FAQ's 9.1. How can I get some help The best way to get help is to us at: <res_support@iac.es> 9.2. How do I know the position of my first job in queue? You can use the command: scontrol show job <job_id> field) shows information about estimated time for the specified job to be executed (check value of StartTime 9.3. How can I see the status of my jobs in queue? Next command will provide you information about your jobs in queues: squeue To obtain detailed information about a specific job: scontrol show job <job_id> 9.6. What version of MPI is currently available at LaPalma? Currently we have installed OpenMPI with support for MPI versions up to MPI 3.1 standard Which compilers are available at LaPalma? You can find GCC and OpenMPI compilers available in La Palma 9.8.What options are recommended to compile in LaPalma? The recommended options to compile in LaPalma are: -O3 -march=native 9.9. Should I be careful wih the memory consumption of my jobs? Yes, you should. Each one of the LaPalma nodes has 32 Gb of RAM shared by 16 cores. Up to 90 % of this memory can be consumed for user jobs, while, at least, 10 % has to be available for the Operating System and daemons. According to that, you must limit the memory consumption of your job to 1.8 Gb per Process (which is 28.8 Gb per node when there is one task per processor) Where should I install programs common to all the members of my project group? Your should install programs accessible to all your group members in the filesystem /storage/projects/ 13
14 (All the groups will have a directory such as /storage/projects/<gropu_id>) Where should I store temporary data? You can use the local hard disk of the node (/scratch) to store temporary data for your jobs Is there any way to make my jobs wait less in queue before running? You must tune the directive #SBATCH -t wall_clock_limit to the expected job duration. This value will be used by to decide when to schedule your job, so, shorter values are likely to reduce waiting time; However, notice that when a job exceeds its wall_clock_limit will be cancelled, so, it is recommended to work with an small security margin Should I be careful with the Input / Output over parallel filesystem (Lustre)? Parallel Filesystem can be a bottleneck when different processes of one job are writing to Lustre along the execution. In this kind of jobs, one possible way to improve the job performance is to copy the needed data for each job to the local scratch at the beginning and copy back to lustre at the end, (with this scheme, most of I/O will be performed locally). This scheme is also recommended for massive sets of sequential jobs. 10. Acknowledgements You should mention it in the acknowledgements of your papers or any other publications where you have used LaPalma: The author thankfully acknowledges the technical expertise and assistance provided by the Spanish Supercomputing Network (Red Española de Supercomputación), as well as the computer resources used: the LaPalma Supercomputer, located at the Instituto de Astrofísica de Canarias." 14
15 A.SSH APPENDIX SSH is a program that enables secure logins over an insecure network. It encrypts all the data passing both ways, so that if it is intercepted it cannot be read. It also replaces the old an insecure tools like telnet, rlogin, rcp, ftp,etc. SSH is a client-server software. Both machines must have ssh installed for it to work. We have already installed a ssh server in our machines. You must have installed an ssh client in your local machine. SSH is available without charge for almost all versions of Unix. IAC recommend the use of OpenSSH client that can be download from but any client compatible with SSH version 2 can be used. To accomplish a login in LaPalma with SSH you have to provide a public key. If you have not got already one you can generate a public/private key pair with the following instructions. A.1.Generating an SSH Key pair on Linux In your private linux workstation enter the command: my-private-user@mymachine> ssh-keygen -b t rsa Generating public/private rsa key pair. Accept the default location to store the key (~/.ssh/id_rsa) by pressing Enter (strongly recommended) or enter an alternative location. Enter file in which to save the key (/home/my-private-user/.ssh/id_rsa): Created directory '/home/my-private-user/.ssh'. Enter a passphrase consisting of 10 to 30 characters. The same rules as for creating safe passwords apply. It is strongly advised to refrain from specifying no passphrase. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/my-private-user/.ssh/id_rsa. Your public key has been saved in /home/my-private-user/.ssh/id_rsa.pub. The key fingerprint is: SHA256:9HoaWWfUmiM+uk1l9VrAX5cxR2CKd5YPqGvpOpCu5bY my-privateuser@mymachine The key's randomart image is: +---[RSA 4096]----+ o=o. *.=.. X S.o.. + * *o.f = O = o = O. o...o O. oo.x..oe..+o +----[SHA256] You should make absolutely sure that the private key (~/.ssh/id_rsa) is not accessible by anyone other than yourself (always set its permissions to 0600). The private key must never fall into the hands of another person. To change the password of an existing key pair, use the command: my-private-user@mymachine> ssh-keygen -p. 15
16 Once the public/private key pair is generated you have to send the public key file ~/.ssh/id_rsa.pub to A.1.1.Using the ssh-agent in Linux When doing lots of secure shell operations it is cumbersome to type the SSH passphrase for each such operation. Therefore, the SSH package provides another tool, ssh-agent, which retains the private keys for the duration of an X or terminal session. All other windows or programs are started as clients to the ssh-agent. By starting the agent, a set of environment variables is set, which will be used by ssh, scp, or sftp to locate the agent for automatic login. See the ssh-agent man page for details. After the ssh-agent is started, you need to add your keys by using ssh-add. It will prompt for the passphrase. After the password has been provided once, you can use the secure shell commands within the running session without having to authenticate again. A Using the ssh-agent in a Terminal Session In a terminal session you need to manually start the ssh-agent and then call ssh-add afterward. There are two ways to start the agent. The first example given below starts a new Bash shell on top of your existing shell. The second example starts the agent in the existing shell and modifies the environment as needed. ssh-agent -s /bin/bash eval $(ssh-agent) After the agent has been started, run ssh-add to provide the agent with your keys. A Using ssh-agent in an X Session To invoke ssh-add to add your keys to the agent at the beginning of an X session, do the following: - Log in as the desired user and check whether the file ~/.xinitrc exists. - If it does not exist, use an existing template or copy it from /etc/skel: if [ -f ~/.xinitrc.template ]; then mv ~/.xinitrc.template ~/.xinitrc; \ else cp /etc/skel/.xinitrc.template ~/.xinitrc; fi - If you have copied the template, search for the following lines and uncomment them. If ~/.xinitrc already existed, add the following lines (without comment signs). # if test -S "$SSH_AUTH_SOCK" -a -x "$SSH_ASKPASS"; then # ssh-add < /dev/null # fi - When starting a new X session, you will be prompted for your SSH passphrase. A.2. Generating an SSH Key pair on Windows In windows systems IAC recommend the use of putty. It is a free SSH client that you can download from But you can also, any client compatible with SSH version 2 can be used. In the next lines we will describe how to install, configure and use a ssh client under Windows systems. Once the client has been installed, launch PuTTygen in order to generate the ssh key: 16
17 Select RSA as Type of key and introduce 4096 in the field of number of bits and click on Generate : 17
18 The tool requires that you move the mouse randomly: Click on Save public key to store the public key and send it to Enter a passphrase consisting of 10 to 30 characters. The same rules as for creating safe passwords apply. It is strongly advised to refrain from specifying no passphrase. After introducin the Key passphrase click on Save private key 18
19 Keep the private key file in a safe location and do not share with anyone. A.2.1.Using the ssh-agent in Windows As in the case of Linux, the package Putty provides a tool to avoid typing the SSH passphrase for each connection. The name of the tool is Pageant and when you launch it an icon is displayed in the taskbar. Double-click on the icon an the list of keys is shown: 19
20 Push Add Key and select the private key previously generated: Enter the passhphrase: The list is updated and you can click on Close : 20
21 Now it is possible to connect to lapalma1.iac.es with Putty. Launch the application and specify the remote machine and click on Open : If it is your first connection to the machine, your will get a Warning telling you that the host key from the server is unknown, and will ask you if you are agree to cache the new host key, press Yes: IMPORTANT: If you see this warning another time and you haven't modified or reinstalled the ssh client, please, don't log in and contact with LaPalma support. 21
22 Finally, a new window will appear asking for your username: To transfer files to or from LaPalma you need a secure ftp (sftp) o secure copy (scp) client. There are several different clients, but as previously mentioned, IAC recommends the use of putty clients for transferring files: psftp and pscp. You can find it at the same web page as putty ( Other possible tool for users requiring graphical file transfers could be: WinSCP: Freeware Sftp and Scp client for Windows ( In order to start remote X applications you need and X-Server running in your local machine. Here is a list of most common X-servers for windows: Cygwin/X: MobaXterm: Once the X-Server is running run putty with X11 forwarding enabled: 22
Power9 CTE User s Guide
Power9 CTE User s Guide Barcelona Supercomputing Center Copyright c 2017 BSC-CNS June 25, 2018 Contents 1 Introduction 2 2 System Overview 2 3 Compiling applications 2 4 Connecting to CTE-POWER 2 4.1 Password
More informationTirant User s Guide. Valencia, April 21, 2008
Tirant User s Guide Tirant User s Guide Valencia, April 21, 2008 1 CONTENTS Tirant User s Guide Contents 1 Introduction 4 2 System Overview 4 3 Connecting to Tirant 5 3.1 Login nodes............................
More informationUsing a Linux System 6
Canaan User Guide Connecting to the Cluster 1 SSH (Secure Shell) 1 Starting an ssh session from a Mac or Linux system 1 Starting an ssh session from a Windows PC 1 Once you're connected... 1 Ending an
More informationIntroduction to High-Performance Computing (HPC)
Introduction to High-Performance Computing (HPC) Computer components CPU : Central Processing Unit cores : individual processing units within a CPU Storage : Disk drives HDD : Hard Disk Drive SSD : Solid
More informationExercises: Abel/Colossus and SLURM
Exercises: Abel/Colossus and SLURM November 08, 2016 Sabry Razick The Research Computing Services Group, USIT Topics Get access Running a simple job Job script Running a simple job -- qlogin Customize
More informationIntroduction to the Cluster
Follow us on Twitter for important news and updates: @ACCREVandy Introduction to the Cluster Advanced Computing Center for Research and Education http://www.accre.vanderbilt.edu The Cluster We will be
More informationIntroduction to the Cluster
Introduction to the Cluster Advanced Computing Center for Research and Education http://www.accre.vanderbilt.edu Follow us on Twitter for important news and updates: @ACCREVandy The Cluster We will be
More informationSlurm and Abel job scripts. Katerina Michalickova The Research Computing Services Group SUF/USIT October 23, 2012
Slurm and Abel job scripts Katerina Michalickova The Research Computing Services Group SUF/USIT October 23, 2012 Abel in numbers Nodes - 600+ Cores - 10000+ (1 node->2 processors->16 cores) Total memory
More informationSlurm and Abel job scripts. Katerina Michalickova The Research Computing Services Group SUF/USIT November 13, 2013
Slurm and Abel job scripts Katerina Michalickova The Research Computing Services Group SUF/USIT November 13, 2013 Abel in numbers Nodes - 600+ Cores - 10000+ (1 node->2 processors->16 cores) Total memory
More informationGraham vs legacy systems
New User Seminar Graham vs legacy systems This webinar only covers topics pertaining to graham. For the introduction to our legacy systems (Orca etc.), please check the following recorded webinar: SHARCNet
More informationCryptography Application : SSH. Cyber Security & Network Security March, 2017 Dhaka, Bangladesh
Cryptography Application : SSH Cyber Security & Network Security 20-22 March, 2017 Dhaka, Bangladesh Issue Date: [31-12-2015] Revision: [v.1] What is Safely Authentication I am Assured of Which Host I
More informationIntroduction to Joker Cyber Infrastructure Architecture Team CIA.NMSU.EDU
Introduction to Joker Cyber Infrastructure Architecture Team CIA.NMSU.EDU What is Joker? NMSU s supercomputer. 238 core computer cluster. Intel E-5 Xeon CPUs and Nvidia K-40 GPUs. InfiniBand innerconnect.
More informationIntroduction to PICO Parallel & Production Enviroment
Introduction to PICO Parallel & Production Enviroment Mirko Cestari m.cestari@cineca.it Alessandro Marani a.marani@cineca.it Domenico Guida d.guida@cineca.it Nicola Spallanzani n.spallanzani@cineca.it
More informationIntroduction to the NCAR HPC Systems. 25 May 2018 Consulting Services Group Brian Vanderwende
Introduction to the NCAR HPC Systems 25 May 2018 Consulting Services Group Brian Vanderwende Topics to cover Overview of the NCAR cluster resources Basic tasks in the HPC environment Accessing pre-built
More informationCompiling applications for the Cray XC
Compiling applications for the Cray XC Compiler Driver Wrappers (1) All applications that will run in parallel on the Cray XC should be compiled with the standard language wrappers. The compiler drivers
More informationCryptography - SSH. Network Security Workshop May 2017 Phnom Penh, Cambodia
Cryptography - SSH Network Security Workshop 29-31 May 2017 Phnom Penh, Cambodia What is Safely Authentication I know who I am talking with Our communication is Encrypted Telnet Servers Terminal Routers
More informationMareNostrum4 User s Guide
MareNostrum4 User s Guide Barcelona Supercomputing Center Copyright c 2017 BSC-CNS June 18, 2018 Contents 1 Introduction 2 2 System Overview 2 2.1 Compilation for the architecture...........................
More informationIntel Manycore Testing Lab (MTL) - Linux Getting Started Guide
Intel Manycore Testing Lab (MTL) - Linux Getting Started Guide Introduction What are the intended uses of the MTL? The MTL is prioritized for supporting the Intel Academic Community for the testing, validation
More informationCryptography Application : SSH. 7 Sept 2017, Taichung, Taiwan
Cryptography Application : SSH 7 Sept 2017, Taichung, Taiwan What is Safely Authentication I am Assured of Which Host I am Talking With Authentication - The Host Knows Who I Am The Traffic is Encrypted
More informationCryptography - SSH. Network Security Workshop. 3-5 October 2017 Port Moresby, Papua New Guinea
Cryptography - SSH Network Security Workshop 3-5 October 2017 Port Moresby, Papua New Guinea 1 What is Secure Authentication I know who I am talking to Our communication is Encrypted Telnet Servers Terminal
More informationIntroduction to High-Performance Computing (HPC)
Introduction to High-Performance Computing (HPC) Computer components CPU : Central Processing Unit cores : individual processing units within a CPU Storage : Disk drives HDD : Hard Disk Drive SSD : Solid
More informationBeginner's Guide for UK IBM systems
Beginner's Guide for UK IBM systems This document is intended to provide some basic guidelines for those who already had certain programming knowledge with high level computer languages (e.g. Fortran,
More informationTo connect to the cluster, simply use a SSH or SFTP client to connect to:
RIT Computer Engineering Cluster The RIT Computer Engineering cluster contains 12 computers for parallel programming using MPI. One computer, cluster-head.ce.rit.edu, serves as the master controller or
More informationNew User Seminar: Part 2 (best practices)
New User Seminar: Part 2 (best practices) General Interest Seminar January 2015 Hugh Merz merz@sharcnet.ca Session Outline Submitting Jobs Minimizing queue waits Investigating jobs Checkpointing Efficiency
More informationSSH. What is Safely 6/19/ June 2018 PacNOG 22, Honiara, Solomon Islands Supported by:
SSH 25-29 June 2018 PacNOG 22, Honiara, Solomon Islands Supported by: Issue Date: Revision: 1 What is Safely Authentication I am Assured of Which Host I am Talking With Authentication - The Host Knows
More informationSlurm basics. Summer Kickstart June slide 1 of 49
Slurm basics Summer Kickstart 2017 June 2017 slide 1 of 49 Triton layers Triton is a powerful but complex machine. You have to consider: Connecting (ssh) Data storage (filesystems and Lustre) Resource
More informationHPC Introductory Course - Exercises
HPC Introductory Course - Exercises The exercises in the following sections will guide you understand and become more familiar with how to use the Balena HPC service. Lines which start with $ are commands
More informationThe cluster system. Introduction 22th February Jan Saalbach Scientific Computing Group
The cluster system Introduction 22th February 2018 Jan Saalbach Scientific Computing Group cluster-help@luis.uni-hannover.de Contents 1 General information about the compute cluster 2 Available computing
More informationAdobe Marketing Cloud Using FTP and sftp with the Adobe Marketing Cloud
Adobe Marketing Using FTP and sftp with the Adobe Marketing Contents Using FTP and sftp with the Adobe Marketing...3 Setting Up FTP Accounts Hosted by Adobe...3 Classifications...3 Data Sources...4 Data
More informationHow to Use a Supercomputer - A Boot Camp
How to Use a Supercomputer - A Boot Camp Shelley Knuth Peter Ruprecht shelley.knuth@colorado.edu peter.ruprecht@colorado.edu www.rc.colorado.edu Outline Today we will discuss: Who Research Computing is
More informationFor Dr Landau s PHYS8602 course
For Dr Landau s PHYS8602 course Shan-Ho Tsai (shtsai@uga.edu) Georgia Advanced Computing Resource Center - GACRC January 7, 2019 You will be given a student account on the GACRC s Teaching cluster. Your
More informationIT Services Security. The Dark Arts Of SSH. Author: John Curran Version: 0.1
IT Services Security The Dark Arts Of SSH Author: John Curran Version: 0.1 STATUS\ REVISION HISTORY Date Version Description 0.1-0.9 Review preparation 1.0 Presented to business & retained by Information
More informationBefore We Start. Sign in hpcxx account slips Windows Users: Download PuTTY. Google PuTTY First result Save putty.exe to Desktop
Before We Start Sign in hpcxx account slips Windows Users: Download PuTTY Google PuTTY First result Save putty.exe to Desktop Research Computing at Virginia Tech Advanced Research Computing Compute Resources
More informationHigh Performance Computing Cluster Basic course
High Performance Computing Cluster Basic course Jeremie Vandenplas, Gwen Dawes 30 October 2017 Outline Introduction to the Agrogenomics HPC Connecting with Secure Shell to the HPC Introduction to the Unix/Linux
More informationIntroduction to GALILEO
Introduction to GALILEO Parallel & production environment Mirko Cestari m.cestari@cineca.it Alessandro Marani a.marani@cineca.it Domenico Guida d.guida@cineca.it Maurizio Cremonesi m.cremonesi@cineca.it
More informationMVAPICH MPI and Open MPI
CHAPTER 6 The following sections appear in this chapter: Introduction, page 6-1 Initial Setup, page 6-2 Configure SSH, page 6-2 Edit Environment Variables, page 6-5 Perform MPI Bandwidth Test, page 6-8
More informationSupercomputing environment TMA4280 Introduction to Supercomputing
Supercomputing environment TMA4280 Introduction to Supercomputing NTNU, IMF February 21. 2018 1 Supercomputing environment Supercomputers use UNIX-type operating systems. Predominantly Linux. Using a shell
More informationOBTAINING AN ACCOUNT:
HPC Usage Policies The IIA High Performance Computing (HPC) System is managed by the Computer Management Committee. The User Policies here were developed by the Committee. The user policies below aim to
More informationComputing with the Moore Cluster
Computing with the Moore Cluster Edward Walter An overview of data management and job processing in the Moore compute cluster. Overview Getting access to the cluster Data management Submitting jobs (MPI
More informationHPC Workshop. Nov. 9, 2018 James Coyle, PhD Dir. Of High Perf. Computing
HPC Workshop Nov. 9, 2018 James Coyle, PhD Dir. Of High Perf. Computing NEEDED EQUIPMENT 1. Laptop with Secure Shell (ssh) for login A. Windows: download/install putty from https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
More informationTech Computer Center Documentation
Tech Computer Center Documentation Release 0 TCC Doc February 17, 2014 Contents 1 TCC s User Documentation 1 1.1 TCC SGI Altix ICE Cluster User s Guide................................ 1 i ii CHAPTER 1
More informationIntroduction to GALILEO
Introduction to GALILEO Parallel & production environment Mirko Cestari m.cestari@cineca.it Alessandro Marani a.marani@cineca.it Alessandro Grottesi a.grottesi@cineca.it SuperComputing Applications and
More informationIntroduction to Discovery.
Introduction to Discovery http://discovery.dartmouth.edu The Discovery Cluster 2 Agenda What is a cluster and why use it Overview of computer hardware in cluster Help Available to Discovery Users Logging
More informationTITANI CLUSTER USER MANUAL V.1.3
2016 TITANI CLUSTER USER MANUAL V.1.3 This document is intended to give some basic notes in order to work with the TITANI High Performance Green Computing Cluster of the Civil Engineering School (ETSECCPB)
More informationCluster Clonetroop: HowTo 2014
2014/02/25 16:53 1/13 Cluster Clonetroop: HowTo 2014 Cluster Clonetroop: HowTo 2014 This section contains information about how to access, compile and execute jobs on Clonetroop, Laboratori de Càlcul Numeric's
More informationIntroduction to GALILEO
November 27, 2016 Introduction to GALILEO Parallel & production environment Mirko Cestari m.cestari@cineca.it Alessandro Marani a.marani@cineca.it SuperComputing Applications and Innovation Department
More informationIntroduction to SLURM & SLURM batch scripts
Introduction to SLURM & SLURM batch scripts Anita Orendt Assistant Director Research Consulting & Faculty Engagement anita.orendt@utah.edu 6 February 2018 Overview of Talk Basic SLURM commands SLURM batch
More informationUsing Cartesius and Lisa. Zheng Meyer-Zhao - Consultant Clustercomputing
Zheng Meyer-Zhao - zheng.meyer-zhao@surfsara.nl Consultant Clustercomputing Outline SURFsara About us What we do Cartesius and Lisa Architectures and Specifications File systems Funding Hands-on Logging
More informationIntroduction to High Performance Computing at UEA. Chris Collins Head of Research and Specialist Computing ITCS
Introduction to High Performance Computing at UEA. Chris Collins Head of Research and Specialist Computing ITCS Introduction to High Performance Computing High Performance Computing at UEA http://rscs.uea.ac.uk/hpc/
More informationUoW HPC Quick Start. Information Technology Services University of Wollongong. ( Last updated on October 10, 2011)
UoW HPC Quick Start Information Technology Services University of Wollongong ( Last updated on October 10, 2011) 1 Contents 1 Logging into the HPC Cluster 3 1.1 From within the UoW campus.......................
More informationOur new HPC-Cluster An overview
Our new HPC-Cluster An overview Christian Hagen Universität Regensburg Regensburg, 15.05.2009 Outline 1 Layout 2 Hardware 3 Software 4 Getting an account 5 Compiling 6 Queueing system 7 Parallelization
More informationWhat is Secure. Authenticated I know who I am talking to. Our communication is Encrypted
Crypto App - SSH 1 What is Secure Authenticated I know who I am talking to Our communication is Encrypted Telnet clear text Servers Terminal clear text Routers SSH encrypted channel encrypted text Servers
More informationDuke Compute Cluster Workshop. 3/28/2018 Tom Milledge rc.duke.edu
Duke Compute Cluster Workshop 3/28/2018 Tom Milledge rc.duke.edu rescomputing@duke.edu Outline of talk Overview of Research Computing resources Duke Compute Cluster overview Running interactive and batch
More informationCNAG Advanced User Training
www.bsc.es CNAG Advanced User Training Aníbal Moreno, CNAG System Administrator Pablo Ródenas, BSC HPC Support Rubén Ramos Horta, CNAG HPC Support Barcelona,May the 5th Aim Understand CNAG s cluster design
More informationIntroduction to the SHARCNET Environment May-25 Pre-(summer)school webinar Speaker: Alex Razoumov University of Ontario Institute of Technology
Introduction to the SHARCNET Environment 2010-May-25 Pre-(summer)school webinar Speaker: Alex Razoumov University of Ontario Institute of Technology available hardware and software resources our web portal
More informationSubmitting and running jobs on PlaFRIM2 Redouane Bouchouirbat
Submitting and running jobs on PlaFRIM2 Redouane Bouchouirbat Summary 1. Submitting Jobs: Batch mode - Interactive mode 2. Partition 3. Jobs: Serial, Parallel 4. Using generic resources Gres : GPUs, MICs.
More informationMinoTauro User s Guide
MinoTauro User s Guide Barcelona Supercomputing Center Copyright c 2017 BSC-CNS December 20, 2017 Contents 1 Introduction 2 2 System Overview 2 3 Connecting to MinoTauro 2 3.1 Password Management................................
More informationSherlock for IBIIS. William Law Stanford Research Computing
Sherlock for IBIIS William Law Stanford Research Computing Overview How we can help System overview Tech specs Signing on Batch submission Software environment Interactive jobs Next steps We are here to
More informationIntroduction to High-Performance Computing (HPC)
Introduction to High-Performance Computing (HPC) Computer components CPU : Central Processing Unit CPU cores : individual processing units within a Storage : Disk drives HDD : Hard Disk Drive SSD : Solid
More informationIntroduction to Discovery.
Introduction to Discovery http://discovery.dartmouth.edu The Discovery Cluster 2 Agenda What is a cluster and why use it Overview of computer hardware in cluster Help Available to Discovery Users Logging
More informationXSEDE New User Training. Ritu Arora November 14, 2014
XSEDE New User Training Ritu Arora Email: rauta@tacc.utexas.edu November 14, 2014 1 Objectives Provide a brief overview of XSEDE Computational, Visualization and Storage Resources Extended Collaborative
More informationBatch Usage on JURECA Introduction to Slurm. May 2016 Chrysovalantis Paschoulas HPS JSC
Batch Usage on JURECA Introduction to Slurm May 2016 Chrysovalantis Paschoulas HPS group @ JSC Batch System Concepts Resource Manager is the software responsible for managing the resources of a cluster,
More informationIntroduction to Linux Workshop 2. The George Washington University SEAS Computing Facility
Introduction to Linux Workshop 2 The George Washington University SEAS Computing Facility Course Goals SSH and communicating with other machines Public/Private key generation,.ssh directory, and the config
More informationTELE301 Lab16 - The Secure Shell
TELE301 Lab16 - The Secure Shell Department of Telecommunications May 7, 2002 Contents 1 Introduction 2 2 OpenSSH 2 3 Replacing Telnet 2 4 Logging in without a password 2 5 SSH Agent 3 5.1 SSH Add..............................
More informationHigh Performance Computing Cluster Advanced course
High Performance Computing Cluster Advanced course Jeremie Vandenplas, Gwen Dawes 9 November 2017 Outline Introduction to the Agrogenomics HPC Submitting and monitoring jobs on the HPC Parallel jobs on
More informationIntroduction to SLURM & SLURM batch scripts
Introduction to SLURM & SLURM batch scripts Anita Orendt Assistant Director Research Consulting & Faculty Engagement anita.orendt@utah.edu 23 June 2016 Overview of Talk Basic SLURM commands SLURM batch
More informationIntroduction to SLURM & SLURM batch scripts
Introduction to SLURM & SLURM batch scripts Anita Orendt Assistant Director Research Consulting & Faculty Engagement anita.orendt@utah.edu 16 Feb 2017 Overview of Talk Basic SLURM commands SLURM batch
More informationIntroduction to NCAR HPC. 25 May 2017 Consulting Services Group Brian Vanderwende
Introduction to NCAR HPC 25 May 2017 Consulting Services Group Brian Vanderwende Topics we will cover Technical overview of our HPC systems The NCAR computing environment Accessing software on Cheyenne
More informationProgramming Tools. Computer Organization Tools. Most of the programming assignments will require using the C language.
Programming Tools Tools 1 Most of the programming assignments will require using the C language. We will use a current version of the GCC C compiler. What s GCC? - the Gnu Compiler Collection (see gcc.gnu.org)
More informationRHRK-Seminar. High Performance Computing with the Cluster Elwetritsch - II. Course instructor : Dr. Josef Schüle, RHRK
RHRK-Seminar High Performance Computing with the Cluster Elwetritsch - II Course instructor : Dr. Josef Schüle, RHRK Overview Course I Login to cluster SSH RDP / NX Desktop Environments GNOME (default)
More informationIntroduction to Cheyenne. 12 January, 2017 Consulting Services Group Brian Vanderwende
Introduction to Cheyenne 12 January, 2017 Consulting Services Group Brian Vanderwende Topics we will cover Technical specs of the Cheyenne supercomputer and expanded GLADE file systems The Cheyenne computing
More informationUsing the IBM Opteron 1350 at OSC. October 19-20, 2010
Using the IBM Opteron 1350 at OSC October 19-20, 2010 Table of Contents Hardware Overview The Linux Operating System User Environment and Storage 2 Hardware Overview Hardware introduction Login node configuration
More informationChoosing Resources Wisely Plamen Krastev Office: 38 Oxford, Room 117 FAS Research Computing
Choosing Resources Wisely Plamen Krastev Office: 38 Oxford, Room 117 Email:plamenkrastev@fas.harvard.edu Objectives Inform you of available computational resources Help you choose appropriate computational
More informationINTRODUCTION TO THE CLUSTER
INTRODUCTION TO THE CLUSTER WHAT IS A CLUSTER? A computer cluster consists of a group of interconnected servers (nodes) that work together to form a single logical system. COMPUTE NODES GATEWAYS SCHEDULER
More informationIntroduction to RCC. September 14, 2016 Research Computing Center
Introduction to HPC @ RCC September 14, 2016 Research Computing Center What is HPC High Performance Computing most generally refers to the practice of aggregating computing power in a way that delivers
More informationIntroduction to HPC Numerical libraries on FERMI and PLX
Introduction to HPC Numerical libraries on FERMI and PLX HPC Numerical Libraries 11-12-13 March 2013 a.marani@cineca.it WELCOME!! The goal of this course is to show you how to get advantage of some of
More informationFEPS. SSH Access with Two-Factor Authentication. RSA Key-pairs
FEPS SSH Access with Two-Factor Authentication RSA Key-pairs access.eps.surrey.ac.uk Contents: Introduction - 3 RSA Key-pairs - 3 Where can I use my RSA Key-Pair? - 3 Step 1 Prepare to generate your RSA
More informationSubmitting batch jobs
Submitting batch jobs SLURM on ECGATE Xavi Abellan Xavier.Abellan@ecmwf.int ECMWF February 20, 2017 Outline Interactive mode versus Batch mode Overview of the Slurm batch system on ecgate Batch basic concepts
More informationFile transfer clients manual File Delivery Services
File transfer clients manual File Delivery Services Publisher Post CH Ltd Information Technology Webergutstrasse 12 CH-3030 Berne (Zollikofen) Contact Post CH Ltd Information Technology Webergutstrasse
More informationIntroduction to CINECA HPC Environment
Introduction to CINECA HPC Environment 23nd Summer School on Parallel Computing 19-30 May 2014 m.cestari@cineca.it, i.baccarelli@cineca.it Goals You will learn: The basic overview of CINECA HPC systems
More informationCS Fundamentals of Programming II Fall Very Basic UNIX
CS 215 - Fundamentals of Programming II Fall 2012 - Very Basic UNIX This handout very briefly describes how to use Unix and how to use the Linux server and client machines in the CS (Project) Lab (KC-265)
More informationIntroduction to RCC. January 18, 2017 Research Computing Center
Introduction to HPC @ RCC January 18, 2017 Research Computing Center What is HPC High Performance Computing most generally refers to the practice of aggregating computing power in a way that delivers much
More informationDistributed Memory Programming With MPI Computer Lab Exercises
Distributed Memory Programming With MPI Computer Lab Exercises Advanced Computational Science II John Burkardt Department of Scientific Computing Florida State University http://people.sc.fsu.edu/ jburkardt/classes/acs2
More informationAn Overview of SSH. Presentation to Linux Users of Victoria. Melbourne, August 26, 2017
An Overview of SSH Presentation to Linux Users of Victoria Melbourne, August 26, 2017 http://levlafayette.com Utilisation and Rationale The most common use of SSH (secure shell) is remote login access
More informationIntroduction to the ITA computer system
Introduction to the ITA computer system Tiago M. D. Pereira Slides: https://folk.uio.no/tiago/teaching/unix2017 Institute of Theoretical Astrophysics Today s lecture in a nutshell 1. Network and users,
More informationHigh Performance Computing (HPC) Using zcluster at GACRC
High Performance Computing (HPC) Using zcluster at GACRC On-class STAT8060 Georgia Advanced Computing Resource Center University of Georgia Zhuofei Hou, HPC Trainer zhuofei@uga.edu Outline What is GACRC?
More informationCommon Report Engine Leipzig. Ref. 0003
Common Report Engine 08.05.2018 Leipzig Ref. 0003 Table of Contents 1. Introduction 3 1.1 SSH File Transfer Protocol 3 1.2 SSH/SFTP tools 3 2. Setup Process 4 2.1 Generation process for public / private
More informationIntroduction to HPC Using zcluster at GACRC
Introduction to HPC Using zcluster at GACRC On-class PBIO/BINF8350 Georgia Advanced Computing Resource Center University of Georgia Zhuofei Hou, HPC Trainer zhuofei@uga.edu Outline What is GACRC? What
More informationKey File Generation. November 14, NATIONAL STUDENT CLEARINGHOUSE 2300 Dulles Station Blvd., Suite 220, Herndon, VA 20171
Key File Generation NATIONAL STUDENT CLEARINGHOUSE 2300 Dulles Station Blvd., Suite 220, Herndon, VA 20171 Table of Contents Introduction... 2 PuTTY Installation... 2 Key Generation... 7 Configuring PuTTY
More informationUsing Compute Canada. Masao Fujinaga Information Services and Technology University of Alberta
Using Compute Canada Masao Fujinaga Information Services and Technology University of Alberta Introduction to cedar batch system jobs are queued priority depends on allocation and past usage Cedar Nodes
More informationIntroduction to HPC Using zcluster at GACRC
Introduction to HPC Using zcluster at GACRC Georgia Advanced Computing Resource Center University of Georgia Zhuofei Hou, HPC Trainer zhuofei@uga.edu Outline What is GACRC? What is HPC Concept? What is
More informationDuke Compute Cluster Workshop. 11/10/2016 Tom Milledge h:ps://rc.duke.edu/
Duke Compute Cluster Workshop 11/10/2016 Tom Milledge h:ps://rc.duke.edu/ rescompu>ng@duke.edu Outline of talk Overview of Research Compu>ng resources Duke Compute Cluster overview Running interac>ve and
More informationIntroduction to HPC2N
Introduction to HPC2N Birgitte Brydsø HPC2N, Umeå University 4 May 2017 1 / 24 Overview Kebnekaise and Abisko Using our systems The File System The Module System Overview Compiler Tool Chains Examples
More informationSCALABLE HYBRID PROTOTYPE
SCALABLE HYBRID PROTOTYPE Scalable Hybrid Prototype Part of the PRACE Technology Evaluation Objectives Enabling key applications on new architectures Familiarizing users and providing a research platform
More informationAn Introduction to Cluster Computing Using Newton
An Introduction to Cluster Computing Using Newton Jason Harris and Dylan Storey March 25th, 2014 Jason Harris and Dylan Storey Introduction to Cluster Computing March 25th, 2014 1 / 26 Workshop design.
More informationIntroduction to HPC2N
Introduction to HPC2N Birgitte Brydsø, Jerry Eriksson, and Pedro Ojeda-May HPC2N, Umeå University 12 September 2017 1 / 38 Overview Kebnekaise and Abisko Using our systems The File System The Module System
More informationDEBUGGING ON FERMI PREPARING A DEBUGGABLE APPLICATION GDB. GDB on front-end nodes
DEBUGGING ON FERMI Debugging your application on a system based on a BG/Q architecture like FERMI could be an hard task due to the following problems: the core files generated by a crashing job on FERMI
More informationName Department/Research Area Have you used the Linux command line?
Please log in with HawkID (IOWA domain) Macs are available at stations as marked To switch between the Windows and the Mac systems, press scroll lock twice 9/27/2018 1 Ben Rogers ITS-Research Services
More informationEffective Use of CCV Resources
Effective Use of CCV Resources Mark Howison User Services & Support This talk... Assumes you have some familiarity with a Unix shell Provides examples and best practices for typical usage of CCV systems
More informationAllinea DDT Debugger. Dan Mazur, McGill HPC March 5,
Allinea DDT Debugger Dan Mazur, McGill HPC daniel.mazur@mcgill.ca guillimin@calculquebec.ca March 5, 2015 1 Outline Introduction and motivation Guillimin login and DDT configuration Compiling for a debugger
More information