UL HPC School 2017[bis] PS1: Getting Started on the UL HPC platform

Size: px
Start display at page:

Download "UL HPC School 2017[bis] PS1: Getting Started on the UL HPC platform"

Transcription

1 UL HPC School 2017[bis] PS1: Getting Started on the UL HPC platform UL High Performance Computing (HPC) Team C. Parisot University of Luxembourg (UL), Luxembourg 1 / 34

2 Latest versions available on Github: UL HPC tutorials: UL HPC School: PS1 tutorial sources: / 34

3 Introduction Summary 1 Introduction 2 SSH Secure Shell 3 UL HPC Tutorial: Getting Started Step by step program of this practical session 4 Hands-On: Getting Started on ULHPC 3 / 34

4 Introduction Main Objectives of this Session Understand SSH Connect to the UL HPC Platform SSH configuration Generate your SSH key pair overcome port filtering Discovering, visualizing and reserving UL HPC resources Working environment Web monitoring interfaces OAR vs. SLURM Batch Scheduler Job management Software / Environement Modules 4 / 34

5 Summary 1 Introduction 2 SSH Secure Shell 3 UL HPC Tutorial: Getting Started Step by step program of this practical session 4 Hands-On: Getting Started on ULHPC 5 / 34

6 SSH: Secure Shell Ensure secure connection to remote (UL) server establish encrypted tunnel using asymmetric keys Public id_rsa.pub vs. Private id_rsa (without.pub) typically on a non-standard port (Ex: 8022) limits kiddie script Basic rule: 1 machine = 1 key pair the private key is SECRET: never send it to anybody Can be protected with a passphrase 6 / 34

7 SSH: Secure Shell Ensure secure connection to remote (UL) server establish encrypted tunnel using asymmetric keys Public id_rsa.pub vs. Private id_rsa (without.pub) typically on a non-standard port (Ex: 8022) limits kiddie script Basic rule: 1 machine = 1 key pair the private key is SECRET: never send it to anybody Can be protected with a passphrase SSH is used as a secure backbone channel for many tools Remote shell i.e remote command line File transfer: rsync, scp, sftp versionning synchronization (svn, git), github, gitlab etc. 6 / 34

8 SSH: Secure Shell Ensure secure connection to remote (UL) server establish encrypted tunnel using asymmetric keys Public id_rsa.pub vs. Private id_rsa (without.pub) typically on a non-standard port (Ex: 8022) limits kiddie script Basic rule: 1 machine = 1 key pair the private key is SECRET: never send it to anybody Can be protected with a passphrase SSH is used as a secure backbone channel for many tools Remote shell i.e remote command line File transfer: rsync, scp, sftp versionning synchronization (svn, git), github, gitlab etc. Authentication: password (better) public key authentication (disable if possible) 6 / 34

9 SSH: Public Key Authentication Client Local Machine local homedir ~/.ssh/ id_rsa owns local private key id_rsa.pub known_hosts logs known servers 7 / 34

10 SSH: Public Key Authentication Client Local Machine Server Remote Machine local homedir ~/.ssh/ id_rsa owns local private key remote homedir ~/.ssh/ authorized_keys knows granted (public) key id_rsa.pub known_hosts logs known servers 7 / 34

11 SSH: Public Key Authentication Client Local Machine Server Remote Machine local homedir ~/.ssh/ id_rsa owns local private key remote homedir ~/.ssh/ authorized_keys knows granted (public) key id_rsa.pub known_hosts logs known servers SSH server config /etc/ssh/ sshd_config ssh_host_rsa_key ssh_host_rsa_key.pub 7 / 34

12 SSH: Public Key Authentication Client Local Machine Server Remote Machine local homedir ~/.ssh/ id_rsa owns local private key remote homedir ~/.ssh/ authorized_keys knows granted (public) key id_rsa.pub 7 / 34

13 SSH: Public Key Authentication Client Local Machine Server Remote Machine local homedir ~/.ssh/ id_rsa id_rsa.pub owns local private key 1. Initiate connection 3. solve challenge using private key return response 2. create random challenge, encrypt using public key remote homedir ~/.ssh/ authorized_keys knows granted (public) key 4. allow connection iff response == challenge Restrict to public key authentication: /etc/ssh/sshd_config: PermitRootLogin no # Disable Passwords PasswordAuthentication no ChallengeResponseAuthentication no # Enable Public key auth. RSAAuthentication yes PubkeyAuthentication yes 7 / 34

14 SSH Setup on Linux / Mac OS OpenSSH natively supported; configuration directory : ~/.ssh/ package openssh-client (Debian-like) or ssh (Redhat-like) SSH Key Pairs (public vs private) generation: ssh-keygen specify a strong passphrase protect your private key from being stolen i.e. impersonation drawback: passphrase must be typed to use your key 8 / 34

15 SSH Setup on Linux / Mac OS OpenSSH natively supported; configuration directory : ~/.ssh/ package openssh-client (Debian-like) or ssh (Redhat-like) SSH Key Pairs (public vs private) generation: ssh-keygen specify a strong passphrase protect your private key from being stolen i.e. impersonation drawback: passphrase must be typed to use your key ssh-agent 8 / 34

16 SSH Setup on Linux / Mac OS OpenSSH natively supported; configuration directory : ~/.ssh/ package openssh-client (Debian-like) or ssh (Redhat-like) SSH Key Pairs (public vs private) generation: ssh-keygen specify a strong passphrase protect your private key from being stolen i.e. impersonation drawback: passphrase must be typed to use your key ssh-agent DSA and RSA 1024 bit are deprecated now! 8 / 34

17 SSH Setup on Linux / Mac OS OpenSSH natively supported; configuration directory : ~/.ssh/ package openssh-client (Debian-like) or ssh (Redhat-like) SSH Key Pairs (public vs private) generation: ssh-keygen specify a strong passphrase protect your private key from being stolen i.e. impersonation drawback: passphrase must be typed to use your key ssh-agent DSA and RSA 1024 bit are deprecated now! $> ssh-keygen -t rsa -b o -a 100 # 4096 bits RSA (better) $> ssh-keygen -t ed o -a 100 # new sexy Ed25519 Private (identity) key ~/.ssh/id_{rsa,ed25519} Public Key ~/.ssh/id_{rsa,ed25519}.pub 8 / 34

18 SSH Setup on Windows: the OLD way Putty Suite, includes: - PuTTY, the free SSH client - Pageant, an SSH authentication agent for PuTTY tools - PLink, th PuTTy CLI - PuTTYgen, an RSA and DSA key generation utility 9 / 34

19 SSH Setup on Windows: the OLD way Putty Suite, includes: - PuTTY, the free SSH client - Pageant, an SSH authentication agent for PuTTY tools - PLink, th PuTTy CLI - PuTTYgen, an RSA and DSA key generation utility PuTTY OpenSSH 9 / 34

20 SSH Setup on Windows: the OLD way Putty Suite, includes: - PuTTY, the free SSH client - Pageant, an SSH authentication agent for PuTTY tools - PLink, th PuTTy CLI - PuTTYgen, an RSA and DSA key generation utility PuTTY OpenSSH Putty keys are NOT supported by OpenSSH (yet can be exported) Binding Pageant with OpenSSH agent is NOT natively supported Third-party tools like ssh-pageant are made for that Combine nicely with Git bash with PLink, hostnames eventually refer to PuTTY Sessions NEVER to SSH entries in ~/.ssh/config This usage might be hidden... Ex: $GIT_SSH etc. 9 / 34

21 SSH Setup on Windows: the NEW way Use MobaXterm! [tabbed] Sessions management X11 server w. enhanced X extensions Graphical SFTP browser SSH gateway / tunnels wizards [remote] Text Editor / 34

22 SSH Basic Usage *.<domain>-intern.com gitlab bastion1.<domain>.com 11 / 34

23 SSH Basic Usage *.<domain>-intern.com gitlab bastion1.<domain>.com SSH 11 / 34

24 SSH Basic Usage *.<domain>-intern.com gitlab bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com SSH 11 / 34

25 SSH Basic Usage *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> gitlab bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com SSH 11 / 34

26 SSH Advanced Usage: SOCKS Proxy *.<domain>-intern.com gitlab bastion1.<domain>.com 12 / 34

27 SSH Advanced Usage: SOCKS Proxy *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> gitlab bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com SSH 12 / 34

28 SSH Advanced Usage: SOCKS Proxy *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> gitlab bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com SSH DOMAIN <domain>-intern.com NOT FOUND 12 / 34

29 SSH Advanced Usage: SOCKS Proxy *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> - Connection/SSH/Tunnels: Port 1080, Dynamic gitlab bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com SOCKS LISTEN: localhost:1080 SSH -D 1080 (SOCKS 5 Proxy) 12 / 34

30 SSH Advanced Usage: SOCKS Proxy *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> - Connection/SSH/Tunnels: Port 1080, Dynamic gitlab bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com SOCKS LISTEN: localhost:1080 SSH -D 1080 (SOCKS 5 Proxy) 12 / 34

31 SSH Advanced Usage: SOCKS Proxy *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> - Connection/SSH/Tunnels: Port 1080, Dynamic gitlab bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com SOCKS LISTEN: localhost:1080 SSH -D 1080 (SOCKS 5 Proxy) 12 / 34

32 SSH Advanced Usage: SOCKS Proxy *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> - Connection/SSH/Tunnels: Port 1080, Dynamic gitlab bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com SOCKS LISTEN: localhost:1080 SSH -D 1080 (SOCKS 5 Proxy) 12 / 34

33 SSH Advanced Usage: SOCKS Proxy *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> - Connection/SSH/Tunnels: Port 1080, Dynamic gitlab bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com SOCKS LISTEN: localhost:1080 SSH -D 1080 (SOCKS 5 Proxy) 12 / 34

34 SSH Advanced Usage: ProxyCommand *.<domain>-intern.com gitlab bastion1.<domain>.com 13 / 34

35 SSH Advanced Usage: ProxyCommand *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> gitlab bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com SSH 13 / 34

36 SSH Advanced Usage: ProxyCommand *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> gitlab Session uni_gitlab - Hostname: gitlab.<domain>-intern.com - Port: Connection/Data: username: <LOGIN> - Connection/Proxy: - type: local - Proxy hostname: bastion1.<domain>.com - Port: Username: <LOGIN> - Local proxy command: plink -load uni_bastion1 -nc %host:%port bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com Host uni_gitlab Hostname gitlab ProxyCommand ssh -q uni_bastion1 "nc %h %p" SSH + ProxyCommand + netcat 13 / 34

37 SSH Advanced Usage: ProxyCommand *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> gitlab Session uni_gitlab - Hostname: gitlab.<domain>-intern.com - Port: Connection/Data: username: <LOGIN> - Connection/Proxy: - type: local - Proxy hostname: bastion1.<domain>.com - Port: Username: <LOGIN> - Local proxy command: plink -load uni_bastion1 -nc %host:%port bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com Host uni_gitlab Hostname gitlab ProxyCommand ssh -q uni_bastion1 "nc %h %p" SSH + ProxyCommand + netcat 13 / 34

38 SSH Advanced Usage: ProxyCommand *.<domain>-intern.com PuTTY / PLink / Pageant (Windows) Session uni_bastion1 - Hostname: bastion1.<domain>.com - Port: Connection/Data: username: <LOGIN> gitlab Session uni_gitlab - Hostname: gitlab.<domain>-intern.com - Port: Connection/Data: username: <LOGIN> - Connection/Proxy: - type: local - Proxy hostname: bastion1.<domain>.com - Port: Username: <LOGIN> - Local proxy command: plink -load uni_bastion1 -nc %host:%port bastion1.<domain>.com OpenSSH ~/.ssh/config (Mac / Linux) Host uni_* User <LOGIN> Port 8022 Host uni_bastion1 Hostname bastion1.<domain>.com Host uni_gitlab Hostname gitlab ProxyCommand ssh -q uni_bastion1 "nc %h %p" SSH + ProxyCommand + netcat 13 / 34

39 SSH in Practice ~/.ssh/config $> ssh [-X] [-p <port>] # Example: ssh -p 8022 Host <shortname> Port <port> User <login> Hostname <hostname> ~/.ssh/config: Simpler commands Bash completion $> ssh cha<tab> 14 / 34

40 SSH in Practice ~/.ssh/config $> ssh [-X] [-p <port>] # Example: ssh -p 8022 Host *.ext_ul ProxyCommand ssh -q chaos-cluster \ "nc -q 0 %h %p" # UL HPC Platform -- Host chaos-cluster Hostname access-chaos.uni.lu Host gaia-cluster Hostname access-gaia.uni.lu Host iris-cluster Hostname access-iris.uni.lu Host *-cluster User login #ADAPT accordingly Port 8022 ForwardAgent no Host <shortname> Port <port> User <login> Hostname <hostname> ~/.ssh/config: Simpler commands Bash completion $> ssh cha<tab> 14 / 34

41 SSH in Practice ~/.ssh/config $> ssh [-X] [-p <port>] # Example: ssh -p 8022 Host *.ext_ul ProxyCommand ssh -q chaos-cluster \ "nc -q 0 %h %p" # UL HPC Platform -- Host chaos-cluster Hostname access-chaos.uni.lu Host gaia-cluster Hostname access-gaia.uni.lu Host iris-cluster Hostname access-iris.uni.lu Host *-cluster User login #ADAPT accordingly Port 8022 ForwardAgent no Host <shortname> Port <port> User <login> Hostname <hostname> ~/.ssh/config: Simpler commands Bash completion $> ssh cha<tab> $> ssh chaos-cluster $> ssh work $> ssh work.ext_ul 14 / 34

42 SSH in Practice: Main CLI commands 15 / 34

43 DSH - Distributed / DancerâĂŹs Shell SSH wrapper that allows to run commands over multiple machines. Linux / Mac OS only $> { apt-get yum brew } install dsh # Installation 16 / 34

44 DSH - Distributed / DancerâĂŹs Shell SSH wrapper that allows to run commands over multiple machines. Linux / Mac OS only $> { apt-get yum brew } install dsh # Installation Configuration: in ~/.dsh/ ~/.dsh/dsh.conf: main configuration file ~/.dsh/machines.list: list of all nodes ~/.dsh/group/: holds group definition <name> Group definition: ~/.dsh/group/<name>: simply list SSH shortnames (one name by line) Bash completion file for DSH: 16 / 34

45 DSH configuration ~/.dsh/dsh.conf ############################################################ # ~/.dsh/dsh.conf # Configuration file for dsh (Distributed / Dancer s Shell). # man dsh.conf for details ############################################################ verbose = 0 remoteshell = ssh showmachinenames = 1 # Specify 1 to make the shell wait for each individual invocation. # See -c and -w option for dsh(1) waitshell = 0 # whether to wait for execution # Number of parallel connection to create at the same time. #forklimit=8 remoteshellopt = -q 17 / 34

46 DSH Basic Usage $> dsh [-c -w] { -a -g <group> -m <hostname> } <command> Option Description -c run the commands in parallel (default) -w run the commands in sequential -a run the command on all nodes listed in machines.list -g <group> restrict the commands to the hosts group <group> -m <hostname> run the command only on hostname FAQ: sudo: sorry, you must have a tty to run sudo requires to change the default configuration of sudo Ex: to not requiring a tty to launch a sudo command Defaults:<login>!requiretty 18 / 34

47 UL HPC Tutorial: Getting Started Summary 1 Introduction 2 SSH Secure Shell 3 UL HPC Tutorial: Getting Started Step by step program of this practical session 4 Hands-On: Getting Started on ULHPC 19 / 34

48 UL HPC Tutorial: Getting Started Reference Tutorial Source Tutorial Page: 20 / 34

49 UL HPC Tutorial: Getting Started Platform overview. Quick presentation of UL HPC platform and the new Iris cluster as of 2017: TFlops, TB (shared) For more details: Site <sitename> Other Clusters network Local Institution Network 10/40 GbE QSFP+ 10 GbE [Redundant] Load balancer Site router [Redundant] Site access server(s) [Redundant] Adminfront(s) OAR Puppet Kadeploy supervision Slurm etc... GPFS / Lustre Fast local interconnect (Infiniband EDR) 100 Gb/s Site Computing Nodes Disk Enclosures Site Shared Storage Area 21 / 34

50 UL HPC Tutorial: Getting Started First connection & SSH setup Obj: Connecting for the 1st time & preparing your SSH environment Step 1a: Connect to UL HPC (Linux / Mac OS / Unix) Step 1b: Connect to UL HPC (Windows) using MobaXTerm or Putty. Step 2: Connect from one cluster to the other Learn how to connect from one cluster to the other. Step 2bis: Using SSH proxycommand to access the clusters allow access from everywhere despite port filtering use of SSH aliases to easier connection Step 3: Transferring data files from your laptop to the clusters cover both Linux / Mac and Windows users 22 / 34

51 UL HPC Tutorial: Getting Started First connection & SSH setup Step 3a:Transferring data files on Linux / OS X / Unix use command line tools (SCP, Rsync) Step 3b: Windows / Linux / OS X / Unix GUI tools Learn how to configure Filezilla a graphical tool to transfer files to/from the clusters. Step 3c: Windows [MobaXterm] file transfert 23 / 34

52 UL HPC Tutorial: Getting Started Discovering & reserving HPC resources Obj: How to reserve resources & use them to run your code on it? Step 1: the working environment What software is installed on the nodes where can I put my files, my data, my results? How many space is available? Step 2: web monitoring interfaces What is the status of the platform? How many ressources are available and when? Why is my job in pending state? 24 / 34

53 UL HPC Tutorial: Getting Started Discovering & reserving HPC resources Step 3a: Reserving resources with Slurm Now I want to run my script on the platform. What should I do? How to use Slurm scheduler on iris cluster? Step 3b: Reserving resources with OAR As above, yet using the OAR scheduler available on gaia, chaos, g5k clusters? 25 / 34

54 UL HPC Tutorial: Getting Started Discovering & reserving HPC resources Step 4: Using modules I want to run a specific version of my software. What software is available? How can I use them? Step 5 (advanced): Job management and Persistent Terminal Sessions using GNU Screen Each time I close my SSH connection, my job is killed. How can I make persistent terminal sessions... to execute my code wthout disconnections. Pre-requisite: screen configuration file ~/.screenrc Basic commands Sample Usage on the UL HPC platform: Kernel compilation 26 / 34

55 Hands-On: Getting Started on ULHPC Summary 1 Introduction 2 SSH Secure Shell 3 UL HPC Tutorial: Getting Started Step by step program of this practical session 4 Hands-On: Getting Started on ULHPC 27 / 34

56 Hands-On: Getting Started on ULHPC Hands-On 1: SSH Setup Your Turn! Generating you SSH Key pair Connect to UL HPC (Linux / Mac OS / Unix / Windows) Connect from your laptop/workstation to UL HPC access Connect from one cluster to the other Transferring files 28 / 34

57 Hands-On: Getting Started on ULHPC Hands-on 2: First steps on UL HPC Debian 8 (chaos/gaia) CentOS 7 (iris) module avail module load./a.out mpirun rsync ssh access Computing Nodes icc Internet oarsub [-I] srun / sbatch UL HPC Environment Computing Nodes GPU Operating System: Debian 7 on gaia, chaos $HOME $WORK $SCRATCH SpectrumScale/GPFS Lustre (gaia only) CentOS 7 on iris Job Management: { oarsub srun/sbatch } Environment modules: modules Not available on frontends, *Only* on compute nodes (advanced) discovering GNU screen ssh rsync Infiniband [Q E]DR nvcc Directory Max size Max #files Backup $HOME (gaia, chaos) 100 GB YES $HOME (iris) 500 GB YES $WORK (except iris) 3 TB NO $SCRATCH (except iris) 10 TB NO 29 / 34

58 Hands-On: Getting Started on ULHPC ULHPC Web monitoring interfaces 30 / 34

59 Hands-On: Getting Started on ULHPC ULHPC Web monitoring interfaces 30 / 34

60 Hands-On: Getting Started on ULHPC ULHPC Web monitoring interfaces 30 / 34

61 Hands-On: Getting Started on ULHPC ULHPC Web monitoring interfaces 30 / 34

62 Hands-On: Getting Started on ULHPC ULHPC Web monitoring interfaces 31 / 34

63 Hands-On: Getting Started on ULHPC Job management If there are not enough resources available, use our reservations, add the parameters in red to your submission commands: OAR (Gaia) $> oarsub -I -t inner= SLURM (Iris) $> srun reservation=hpcschool pty bash 32 / 34

64 Hands-On: Getting Started on ULHPC Programming, quick start choose a command line text editor load modules run a Matlab script run a R script use the available compilers compile and run a simple MPI program 33 / 34

65 Thank you for your attention... Questions? High Performance UL Prof. Pascal Bouvry Dr. Sebastien Varrette & the UL HPC Team (V. Plugaru, S. Peter, H. Cartiaux & C. Parisot) University of Luxembourg, Belval Campus Maison du Nombre, 4th floor 2, avenue de l Université L-4365 Esch-sur-Alzette mail: hpc@uni.lu 1 Introduction 2 SSH Secure Shell 3 UL HPC Tutorial: Getting Started Step by step program of this practical session 4 Hands-On: Getting Started on ULHPC 34 / 34

Tutorial: SSH. Secure SHell: Connect remotely anything, anywhere. UL High Performance Computing (HPC) Team Sebastien Varrette

Tutorial: SSH. Secure SHell: Connect remotely anything, anywhere. UL High Performance Computing (HPC) Team Sebastien Varrette Secure SHell: Connect remotely anything, anywhere UL High Performance Computing (HPC) Team Sebastien Varrette University of Luxembourg (UL), Luxembourg http://hpc.uni.lu 1 / 26 Summary 1 Introduction 2

More information

Cryptography Application : SSH. Cyber Security & Network Security March, 2017 Dhaka, Bangladesh

Cryptography Application : SSH. Cyber Security & Network Security March, 2017 Dhaka, Bangladesh Cryptography Application : SSH Cyber Security & Network Security 20-22 March, 2017 Dhaka, Bangladesh Issue Date: [31-12-2015] Revision: [v.1] What is Safely Authentication I am Assured of Which Host I

More information

Cryptography - SSH. Network Security Workshop May 2017 Phnom Penh, Cambodia

Cryptography - SSH. Network Security Workshop May 2017 Phnom Penh, Cambodia Cryptography - SSH Network Security Workshop 29-31 May 2017 Phnom Penh, Cambodia What is Safely Authentication I know who I am talking with Our communication is Encrypted Telnet Servers Terminal Routers

More information

Cryptography - SSH. Network Security Workshop. 3-5 October 2017 Port Moresby, Papua New Guinea

Cryptography - SSH. Network Security Workshop. 3-5 October 2017 Port Moresby, Papua New Guinea Cryptography - SSH Network Security Workshop 3-5 October 2017 Port Moresby, Papua New Guinea 1 What is Secure Authentication I know who I am talking to Our communication is Encrypted Telnet Servers Terminal

More information

SSH. What is Safely 6/19/ June 2018 PacNOG 22, Honiara, Solomon Islands Supported by:

SSH. What is Safely 6/19/ June 2018 PacNOG 22, Honiara, Solomon Islands Supported by: SSH 25-29 June 2018 PacNOG 22, Honiara, Solomon Islands Supported by: Issue Date: Revision: 1 What is Safely Authentication I am Assured of Which Host I am Talking With Authentication - The Host Knows

More information

FEPS. SSH Access with Two-Factor Authentication. RSA Key-pairs

FEPS. SSH Access with Two-Factor Authentication. RSA Key-pairs FEPS SSH Access with Two-Factor Authentication RSA Key-pairs access.eps.surrey.ac.uk Contents: Introduction - 3 RSA Key-pairs - 3 Where can I use my RSA Key-Pair? - 3 Step 1 Prepare to generate your RSA

More information

Cryptography Application : SSH. 7 Sept 2017, Taichung, Taiwan

Cryptography Application : SSH. 7 Sept 2017, Taichung, Taiwan Cryptography Application : SSH 7 Sept 2017, Taichung, Taiwan What is Safely Authentication I am Assured of Which Host I am Talking With Authentication - The Host Knows Who I Am The Traffic is Encrypted

More information

What is Secure. Authenticated I know who I am talking to. Our communication is Encrypted

What is Secure. Authenticated I know who I am talking to. Our communication is Encrypted Crypto App - SSH 1 What is Secure Authenticated I know who I am talking to Our communication is Encrypted Telnet clear text Servers Terminal clear text Routers SSH encrypted channel encrypted text Servers

More information

An Overview of SSH. Presentation to Linux Users of Victoria. Melbourne, August 26, 2017

An Overview of SSH. Presentation to Linux Users of Victoria. Melbourne, August 26, 2017 An Overview of SSH Presentation to Linux Users of Victoria Melbourne, August 26, 2017 http://levlafayette.com Utilisation and Rationale The most common use of SSH (secure shell) is remote login access

More information

This is a guide about using Putty on Windows with OpenSSH on Linux. You would learn about how to:

This is a guide about using Putty on Windows with OpenSSH on Linux. You would learn about how to: 1 of 7 5/16/2011 5:37 PM Submitted by: Man-wai CHANG Update by: Man-wai CHANG Date Submitted: 31 May 2006 Document Version: 1.0 Last Updated: 08/01/2010 13:48:54 Last Updated: 02/02/2008 23:46:43 This

More information

SSH and keys. Network Startup Resource Center

SSH and keys. Network Startup Resource Center SSH and keys Network Startup Resource Center www.nsrc.org These materials are licensed under the Creative Commons Attribution-NonCommercial 4.0 International license (http://creativecommons.org/licenses/by-nc/4.0/)

More information

Introduction to the ITA computer system

Introduction to the ITA computer system Introduction to the ITA computer system Tiago M. D. Pereira Slides: https://folk.uio.no/tiago/teaching/unix2017 Institute of Theoretical Astrophysics Today s lecture in a nutshell 1. Network and users,

More information

Setting up a Chaincoin Masternode

Setting up a Chaincoin Masternode Setting up a Chaincoin Masternode Introduction So you want to set up your own Chaincoin Masternode? You ve come to the right place! These instructions are correct as of April, 2017, and relate to version

More information

Introduction to Joker Cyber Infrastructure Architecture Team CIA.NMSU.EDU

Introduction to Joker Cyber Infrastructure Architecture Team CIA.NMSU.EDU Introduction to Joker Cyber Infrastructure Architecture Team CIA.NMSU.EDU What is Joker? NMSU s supercomputer. 238 core computer cluster. Intel E-5 Xeon CPUs and Nvidia K-40 GPUs. InfiniBand innerconnect.

More information

Supercomputing environment TMA4280 Introduction to Supercomputing

Supercomputing environment TMA4280 Introduction to Supercomputing Supercomputing environment TMA4280 Introduction to Supercomputing NTNU, IMF February 21. 2018 1 Supercomputing environment Supercomputers use UNIX-type operating systems. Predominantly Linux. Using a shell

More information

Using a Linux System 6

Using a Linux System 6 Canaan User Guide Connecting to the Cluster 1 SSH (Secure Shell) 1 Starting an ssh session from a Mac or Linux system 1 Starting an ssh session from a Windows PC 1 Once you're connected... 1 Ending an

More information

Linux Network Administration

Linux Network Administration Secure Remote Connections with OpenSSH Objective At the conclusion of this module, the student will be able to: Configure the ssh daemon start, stop, and restart sshd 17 January 2005 NETW 111 - SSH 2 SSH

More information

2-1-1 ssh Secure SHell

2-1-1 ssh Secure SHell 2-1-1 ssh Secure SHell Using Public Key Cryptography Keying, Key Exchange, and Session Setup 1 Communicate Safely with Remote Systems 2 What is Safely Authentication I am Assured of Which Host I am Talking

More information

Graham vs legacy systems

Graham vs legacy systems New User Seminar Graham vs legacy systems This webinar only covers topics pertaining to graham. For the introduction to our legacy systems (Orca etc.), please check the following recorded webinar: SHARCNet

More information

Security with SSH. Network Startup Resource Center

Security with SSH. Network Startup Resource Center Security with SSH Network Startup Resource Center http://www.nsrc.org/ These materials are licensed under the Creative Commons Attribution-NonCommercial 4.0 International license (http://creativecommons.org/licenses/by-nc/4.0/)

More information

Introduction to Linux Workshop 2. The George Washington University SEAS Computing Facility

Introduction to Linux Workshop 2. The George Washington University SEAS Computing Facility Introduction to Linux Workshop 2 The George Washington University SEAS Computing Facility Course Goals SSH and communicating with other machines Public/Private key generation,.ssh directory, and the config

More information

Siemens PLM Software. HEEDS MDO Setting up a Windows-to- Linux Compute Resource.

Siemens PLM Software. HEEDS MDO Setting up a Windows-to- Linux Compute Resource. Siemens PLM Software HEEDS MDO 2018.04 Setting up a Windows-to- Linux Compute Resource www.redcedartech.com. Contents Introduction 1 On Remote Machine B 2 Installing the SSH Server 2 Configuring the SSH

More information

NoMachine NX Client Configuration Guide

NoMachine NX Client Configuration Guide NoMachine NX Client Configuration Guide Configuration steps for ssh key login Step 1: NoMachine NX requires keys in OpenSSH format, therefore the existing key needs to be converted into OpenSSH format.

More information

2-1-1 ssh Secure SHell

2-1-1 ssh Secure SHell 2-1-1 ssh Secure SHell Using Public Key Cryptography Keying, Key Exchange, and Session Setup Communicate Safely with Remote Systems What is Safely Authentication I am Assured of Which Host I am Talking

More information

ssh and handson Matsuzaki maz Yoshinobu 1

ssh and handson Matsuzaki maz Yoshinobu  1 ssh and handson Matsuzaki maz Yoshinobu maz@iij.ad.jp 1 Secure Shell (ssh) Replacement for unsecure tools/protocols rsh and telnet Usually listen on tcp/22 Whole communication is encrypted

More information

LAB :: Secure SHell (SSL)

LAB :: Secure SHell (SSL) LAB :: Secure SHell (SSL) In this example we are using apnictraining.net as domain name. # super user command. $ normal user command. X replace with your group no. Username apnic and password training

More information

Ftp Command Line Manual Windows Username Password Linux

Ftp Command Line Manual Windows Username Password Linux Ftp Command Line Manual Windows Username Password Linux Midnight Commander is a console based full-screen text mode File Manager that allows you to copy, MC has many features which are useful for a user

More information

Project #6: Using ssh, scp and sftp with Key-Based Authentication

Project #6: Using ssh, scp and sftp with Key-Based Authentication Project #6: Using ssh, scp and sftp with Key-Based Authentication ssh, scp and sftp Going beyond Password Protection Creating Keys Creating Keys with a Passphrase Using Key-Based Authentication in Our

More information

GIT. A free and open source distributed version control system. User Guide. January, Department of Computer Science and Engineering

GIT. A free and open source distributed version control system. User Guide. January, Department of Computer Science and Engineering GIT A free and open source distributed version control system User Guide January, 2018 Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Table of Contents What is

More information

Siemens PLM Software. HEEDS MDO Setting up a Windows-to- Windows Compute Resource.

Siemens PLM Software. HEEDS MDO Setting up a Windows-to- Windows Compute Resource. Siemens PLM Software HEEDS MDO 2018.04 Setting up a Windows-to- Windows Compute Resource www.redcedartech.com. Setting up a Windows-to-Windows Compute Resource Contents Introduction 1 On Remote Machine

More information

1. INTRODUCTION to AURO Cloud Computing

1. INTRODUCTION to AURO Cloud Computing 1. INTRODUCTION to AURO Cloud Computing Welcome to AURO! The purpose of this document is to help you get started with using AURO s Public Cloud Compute. This document covers how to setup and launch a virtual

More information

For Dr Landau s PHYS8602 course

For Dr Landau s PHYS8602 course For Dr Landau s PHYS8602 course Shan-Ho Tsai (shtsai@uga.edu) Georgia Advanced Computing Resource Center - GACRC January 7, 2019 You will be given a student account on the GACRC s Teaching cluster. Your

More information

XSEDE New User Training. Ritu Arora November 14, 2014

XSEDE New User Training. Ritu Arora   November 14, 2014 XSEDE New User Training Ritu Arora Email: rauta@tacc.utexas.edu November 14, 2014 1 Objectives Provide a brief overview of XSEDE Computational, Visualization and Storage Resources Extended Collaborative

More information

School of Computing Science Gitlab Platform - User Notes

School of Computing Science Gitlab Platform - User Notes School of Computing Science Gitlab Platform - User Notes Contents Using Git & Gitlab... 1 Introduction... 1 Access Methods... 2 Web Access... 2 Repository Access... 2 Creating a key pair... 2 Adding a

More information

TELE301 Lab16 - The Secure Shell

TELE301 Lab16 - The Secure Shell TELE301 Lab16 - The Secure Shell Department of Telecommunications May 7, 2002 Contents 1 Introduction 2 2 OpenSSH 2 3 Replacing Telnet 2 4 Logging in without a password 2 5 SSH Agent 3 5.1 SSH Add..............................

More information

HPC Workshop. Nov. 9, 2018 James Coyle, PhD Dir. Of High Perf. Computing

HPC Workshop. Nov. 9, 2018 James Coyle, PhD Dir. Of High Perf. Computing HPC Workshop Nov. 9, 2018 James Coyle, PhD Dir. Of High Perf. Computing NEEDED EQUIPMENT 1. Laptop with Secure Shell (ssh) for login A. Windows: download/install putty from https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html

More information

Introduction to High-Performance Computing (HPC)

Introduction to High-Performance Computing (HPC) Introduction to High-Performance Computing (HPC) Computer components CPU : Central Processing Unit cores : individual processing units within a CPU Storage : Disk drives HDD : Hard Disk Drive SSD : Solid

More information

CSC- Bioweek 2018 Using cpouta for cloud computing Kimmo Mattila, Shubham Kapoor, Ari-Matti Saren (Jukka Nousiainen)

CSC- Bioweek 2018 Using cpouta for cloud computing Kimmo Mattila, Shubham Kapoor, Ari-Matti Saren (Jukka Nousiainen) CSC- Bioweek 2018 Using cpouta for cloud computing 8.2.2018 Kimmo Mattila, Shubham Kapoor, Ari-Matti Saren (Jukka Nousiainen) CSC Finnish research, education and public administration ICT knowledge centre

More information

Common Report Engine Leipzig. Ref. 0003

Common Report Engine Leipzig. Ref. 0003 Common Report Engine 08.05.2018 Leipzig Ref. 0003 Table of Contents 1. Introduction 3 1.1 SSH File Transfer Protocol 3 1.2 SSH/SFTP tools 3 2. Setup Process 4 2.1 Generation process for public / private

More information

Bitnami Apache Solr for Huawei Enterprise Cloud

Bitnami Apache Solr for Huawei Enterprise Cloud Bitnami Apache Solr for Huawei Enterprise Cloud Description Apache Solr is an open source enterprise search platform from the Apache Lucene project. It includes powerful full-text search, highlighting,

More information

Bastion Hosts. Protected Access for Virtual Cloud Networks O R A C L E W H I T E P A P E R F E B R U A R Y

Bastion Hosts. Protected Access for Virtual Cloud Networks O R A C L E W H I T E P A P E R F E B R U A R Y Bastion Hosts Protected Access for Virtual Cloud Networks O R A C L E W H I T E P A P E R F E B R U A R Y 2 0 1 8 Disclaimer The following is intended to outline our general product direction. It is intended

More information

Package ssh. June 4, 2018

Package ssh. June 4, 2018 Type Package Title Secure Shell (SSH) Client for R Version 0.2 Author Jeroen Ooms Package ssh June 4, 2018 Maintainer Jeroen Ooms Connect to a remote server over SSH to transfer files

More information

SSH Tips and Tricks. Ferry Boender. Warning

SSH Tips and Tricks. Ferry Boender. Warning Revision History Revision 1.0 May 02, 2011 Revised by: FB Ferry Boender 1. Authentication 1.1. Passwordless key One very useful feature of SSH are passwordless keys. They are especially useful when you

More information

Secure SHell Explained!

Secure SHell Explained! Open Gurus How To Secure SHell Explained! Here re some insights into SSH (Secure Shell), an essential tool for accessing remote machines. S SH is used to access or log in to a remote machine on the network,

More information

Bitnami MEAN for Huawei Enterprise Cloud

Bitnami MEAN for Huawei Enterprise Cloud Bitnami MEAN for Huawei Enterprise Cloud Description Bitnami MEAN Stack provides a complete development environment for mongodb and Node.js that can be deployed in one click. It includes the latest stable

More information

SSH to your DC/OS cluster on Unix/Linux (macos, Ubuntu, etc) SSH to your DC/OS cluster on Windows

SSH to your DC/OS cluster on Unix/Linux (macos, Ubuntu, etc) SSH to your DC/OS cluster on Windows SSHing into Nodes These instructions explain how to set up an SSH connection to your DC/OS cluster from an outside network. If you are on the same network as your cluster or connected by using VPN, you

More information

Slurm basics. Summer Kickstart June slide 1 of 49

Slurm basics. Summer Kickstart June slide 1 of 49 Slurm basics Summer Kickstart 2017 June 2017 slide 1 of 49 Triton layers Triton is a powerful but complex machine. You have to consider: Connecting (ssh) Data storage (filesystems and Lustre) Resource

More information

IT Services Security. The Dark Arts Of SSH. Author: John Curran Version: 0.1

IT Services Security. The Dark Arts Of SSH. Author: John Curran Version: 0.1 IT Services Security The Dark Arts Of SSH Author: John Curran Version: 0.1 STATUS\ REVISION HISTORY Date Version Description 0.1-0.9 Review preparation 1.0 Presented to business & retained by Information

More information

Ftp Command Line Manual Windows Example Port 22

Ftp Command Line Manual Windows Example Port 22 Ftp Command Line Manual Windows Example Port 22 Session, Logging, Console/scripting mode, Operations, Configuration enables passive ( =on ) or active ( =off ) transfer mode (FTP protocol only). winscp.exe

More information

HPC system startup manual (version 1.20)

HPC system startup manual (version 1.20) HPC system startup manual (version 1.20) Document change log Issue Date Change 1 12/1/2012 New document 2 10/22/2013 Added the information of supported OS 3 10/22/2013 Changed the example 1 for data dowonload

More information

CMTH/TYC Linux Cluster Overview. Éamonn Murray 1 September 2017

CMTH/TYC Linux Cluster Overview. Éamonn Murray 1 September 2017 CMTH/TYC Linux Cluster Overview Éamonn Murray 1 September 2017 Cluster Documentation Everything I ll talk about is covered in the cluster documentation Web: http://www.cmth.ph.ic.ac.uk/computing Local

More information

About the Tutorial. Audience. Prerequisites. Copyright & Disclaimer. Gerrit

About the Tutorial. Audience. Prerequisites. Copyright & Disclaimer. Gerrit Gerrit About the Tutorial Gerrit is a web-based code review tool, which is integrated with Git and built on top of Git version control system (helps developers to work together and maintain the history

More information

MarketC - Masternode Setup Guide

MarketC - Masternode Setup Guide MarketC - Masternode Setup Guide Preface In this guide we will be focusing on setting up a masternode for Marketc (CMK). This guide will focus on a typical "hot node" / "cold wallet" scenario. The "hot

More information

LECTURE 7. Readings: - SSH: The Definitive Guide; D.J. Barret et al.; O Reilly Lecture outline: - SSH. Marco Spaziani Brunella, Manuel Campo

LECTURE 7. Readings: - SSH: The Definitive Guide; D.J. Barret et al.; O Reilly Lecture outline: - SSH. Marco Spaziani Brunella, Manuel Campo LECTURE 7 Readings: - SSH: The Definitive Guide; D.J. Barret et al.; O Reilly Lecture outline: - SSH Remote Managing In real life, physical access to network nodes is not always an option. Often, we need

More information

Linux Administration

Linux Administration Linux Administration This course will cover all aspects of Linux Certification. At the end of the course delegates will have the skills required to administer a Linux System. It is designed for professionals

More information

Ftp Command Line Manual Windows User Password

Ftp Command Line Manual Windows User Password Ftp Command Line Manual Windows User Password SSH Guide SFTP on Mac OS X SFTP on Windows The most straight forward way to use SFTP is through the command line. a command line, you can alternately use an

More information

HP OO 10.x Network Architecture

HP OO 10.x Network Architecture Technical white paper HP OO 10.x Network Architecture Table of Contents Overview 2 Advancing to a Scalable Model 2 The Old Model 2 The New Model 3 Configuring the New Model 4 Firewall Configuration 4 Worker

More information

No More Passwords (with SSH)

No More Passwords (with SSH) No More Passwords (with SSH) Ted Dustman March 30, 2009 Contents 1 Introduction 1 1.1 Local or Remote?................................. 1 1.2 SSH Command Set................................ 1 2 Authentication

More information

Ftp Get Command Line Linux Proxy Settings Via

Ftp Get Command Line Linux Proxy Settings Via Ftp Get Command Line Linux Proxy Settings Via How to change system proxy settings from the command line on Ubuntu desktop gsettings get _schema key_ If you want to change HTTPS/FTP proxy to manual as well,

More information

Introduction to the Cluster

Introduction to the Cluster Follow us on Twitter for important news and updates: @ACCREVandy Introduction to the Cluster Advanced Computing Center for Research and Education http://www.accre.vanderbilt.edu The Cluster We will be

More information

If you prefer to use your own SSH client, configure NG Admin with the path to the executable:

If you prefer to use your own SSH client, configure NG Admin with the path to the executable: Each Barracuda NG Firewall system is routinely equipped with an SSH daemon listening on TCP port 22 on all administrative IP addresses (the primary box IP address and all other IP addresses that administrative

More information

HPC Introductory Course - Exercises

HPC Introductory Course - Exercises HPC Introductory Course - Exercises The exercises in the following sections will guide you understand and become more familiar with how to use the Balena HPC service. Lines which start with $ are commands

More information

Linux. Computer networks - Administration 1DV202. fredag 30 mars 12

Linux. Computer networks - Administration 1DV202. fredag 30 mars 12 Linux Computer networks - Administration 1DV202 Configuration files /etc/hosts /etc/resolv.conf /etc/network/interfaces /etc/hosts 127.0.0.1 localhost.localdomain localhost 127.0.1.1 lokal.kalmar.se lokal

More information

Practical Magic with SSH. By David F. Skoll Roaring Penguin Software Inc. 1 February

Practical Magic with SSH. By David F. Skoll Roaring Penguin Software Inc. 1 February Practical Magic with SSH By David F. Skoll Roaring Penguin Software Inc. 1 February 2001 http://www.roaringpenguin.com dfs@roaringpenguin.com Overview of Presentation Why SSH? Problems with Telnet & Friends

More information

Build your own Lightweight Webserver - Hands-on I - Information Network I. Marius Georgescu. Internet Engineering Laboratory. 17 Apr

Build your own Lightweight Webserver - Hands-on I - Information Network I. Marius Georgescu. Internet Engineering Laboratory. 17 Apr Build your own Lightweight Webserver - Hands-on I - Information Network I Marius Georgescu Internet Engineering Laboratory 17 Apr. 2015 iplab Prerequisites Prerequisites Download and Install VirtualBox

More information

Bitnami JRuby for Huawei Enterprise Cloud

Bitnami JRuby for Huawei Enterprise Cloud Bitnami JRuby for Huawei Enterprise Cloud Description JRuby is a 100% Java implementation of the Ruby programming language. It is Ruby for the JVM. JRuby provides a complete set of core built-in classes

More information

Installing Cisco MSE in a VMware Virtual Machine

Installing Cisco MSE in a VMware Virtual Machine Installing Cisco MSE in a VMware Virtual Machine This chapter describes how to install and deploy a Cisco Mobility Services Engine (MSE) virtual appliance. Cisco MSE is a prebuilt software solution that

More information

High Performance Computing: Planing, usage, and Application

High Performance Computing: Planing, usage, and Application High Performance Computing: Planing, usage, and Application Yudi Rosandi Workshop on High Performance Computing: from design to application Outline Planing Hardware components Infrastructures Operating

More information

File transfer clients manual File Delivery Services

File transfer clients manual File Delivery Services File transfer clients manual File Delivery Services Publisher Post CH Ltd Information Technology Webergutstrasse 12 CH-3030 Berne (Zollikofen) Contact Post CH Ltd Information Technology Webergutstrasse

More information

Windows Subsystem for Linux Guide Documentation

Windows Subsystem for Linux Guide Documentation Windows Subsystem for Linux Guide Documentation Release Kenneth Reitz Aug 30, 2017 Contents: 1 Background on WSL (Windows Subsystem for Linux) 3 1.1 What is WSL?..............................................

More information

Adobe Marketing Cloud Using FTP and sftp with the Adobe Marketing Cloud

Adobe Marketing Cloud Using FTP and sftp with the Adobe Marketing Cloud Adobe Marketing Using FTP and sftp with the Adobe Marketing Contents Using FTP and sftp with the Adobe Marketing...3 Setting Up FTP Accounts Hosted by Adobe...3 Classifications...3 Data Sources...4 Data

More information

Remote Access Via Remote Desktop

Remote Access Via Remote Desktop Remote Access Via Remote Desktop Remotely connecting to your office desktop requires a client-server interface using Remote Desktop. The following page describes the procedures for establishing a SSH Tunnel

More information

Setting up my Dev Environment ECS 030

Setting up my Dev Environment ECS 030 Setting up my Dev Environment ECS 030 1 Command for SSHing into a CSIF Machine If you already have a terminal and already have a working ssh program (That is, you type ssh into the terminal and it doesn

More information

Singularity: container formats

Singularity: container formats Singularity Easy to install and configure Easy to run/use: no daemons no root works with scheduling systems User outside container == user inside container Access to host resources Mount (parts of) filesystems

More information

UL HPC Monitoring in practice: why, what, how, where to look

UL HPC Monitoring in practice: why, what, how, where to look C. Parisot UL HPC Monitoring in practice: why, what, how, where to look 1 / 22 What is HPC? Best Practices Getting Fast & Efficient UL HPC Monitoring in practice: why, what, how, where to look Clément

More information

Beginner's Guide for UK IBM systems

Beginner's Guide for UK IBM systems Beginner's Guide for UK IBM systems This document is intended to provide some basic guidelines for those who already had certain programming knowledge with high level computer languages (e.g. Fortran,

More information

A Quick Guide for the pbdrpc Package

A Quick Guide for the pbdrpc Package A Quick Guide for the pbdrpc Package Wei-Chen Chen pbdr Core Team Silver Spring, MD, USA Contents 1. Introduction 1 1.1. Basic ssh and srpc()................................. 2 1.2. Basic ssh()......................................

More information

Security with SSH. SANOG VI IP Services Workshop. Hervey Allen

Security with SSH. SANOG VI IP Services Workshop. Hervey Allen Security with SSH SANOG VI IP Services Workshop Hervey Allen Topics Where to get SSH (Secure SHell) How to enable and configure SSH Where to get SSH clients for Windows Authentication of the server to

More information

Topics. Security with SSH. Cryptographic Methods and Apps. SSH Application Layer Security

Topics. Security with SSH. Cryptographic Methods and Apps. SSH Application Layer Security Security with SSH SANOG VI IP Services Workshop Hervey Allen Topics Where to get SSH (Secure SHell) How to enable and configure SSH Where to get SSH clients for Windows Authentication of the server to

More information

Sherlock for IBIIS. William Law Stanford Research Computing

Sherlock for IBIIS. William Law Stanford Research Computing Sherlock for IBIIS William Law Stanford Research Computing Overview How we can help System overview Tech specs Signing on Batch submission Software environment Interactive jobs Next steps We are here to

More information

Introduction to High-Performance Computing (HPC)

Introduction to High-Performance Computing (HPC) Introduction to High-Performance Computing (HPC) Computer components CPU : Central Processing Unit cores : individual processing units within a CPU Storage : Disk drives HDD : Hard Disk Drive SSD : Solid

More information

Let s see some real traffic

Let s see some real traffic LINUX NETWORK TOOLS Let s see some real traffic wireshark tcpdump tcpdump the command line network analizer For documentation: man tcpdump (program usage) http://danielmiessler.com/study/tcpdump/ (nice

More information

Ftp Get Command Line Linux Proxy Settings

Ftp Get Command Line Linux Proxy Settings Ftp Get Command Line Linux Proxy Settings How to change system proxy settings from the command line on Ubuntu If you want to change HTTPS/FTP proxy to manual as well, use these commands: Do you want to

More information

Introduction to BioHPC

Introduction to BioHPC Introduction to BioHPC New User Training [web] [email] portal.biohpc.swmed.edu biohpc-help@utsouthwestern.edu 1 Updated for 2015-06-03 Overview Today we re going to cover: What is BioHPC? How do I access

More information

Bitnami Ruby for Huawei Enterprise Cloud

Bitnami Ruby for Huawei Enterprise Cloud Bitnami Ruby for Huawei Enterprise Cloud Description Bitnami Ruby Stack provides a complete development environment for Ruby on Rails that can be deployed in one click. It includes most popular components

More information

HOW TO SECURELY CONFIGURE A LINUX HOST TO RUN CONTAINERS

HOW TO SECURELY CONFIGURE A LINUX HOST TO RUN CONTAINERS HOW TO SECURELY CONFIGURE A LINUX HOST TO RUN CONTAINERS How To Securely Configure a Linux Host to Run Containers To run containers securely, one must go through a multitude of steps to ensure that a)

More information

SSH Deploy Key Documentation

SSH Deploy Key Documentation SSH Deploy Key Documentation Release 0.1.1 Travis Bear February 03, 2014 Contents 1 Overview 1 2 Source Code 3 3 Contents 5 3.1 Alternatives................................................ 5 3.2 Compatibility...............................................

More information

Titan FTP Server SSH Host Key Authentication with SFTP

Titan FTP Server SSH Host Key Authentication with SFTP 2016 Titan FTP Server SSH Host Key Authentication with SFTP A guide for configuring and maintaining SSH Host Key Authentication for SFTP connections in Titan FTP Server. QuickStart Guide 2016 South River

More information

Preparing Your Google Cloud VM for W4705

Preparing Your Google Cloud VM for W4705 Preparing Your Google Cloud VM for W4705 August 27, 2017 1. Get a cloud.cs.columbia.edu account 1. Sign up for a cloud Columbia CS account using this link. Note that is is an entirely new account and is

More information

Some SSH tips & tricks you may enjoy (plus, iptables)

Some SSH tips & tricks you may enjoy (plus, iptables) Some SSH tips & tricks you may enjoy (plus, iptables) D. H. van Dok (Nikhef) 2014-05-19 getting the most (security) out of your openssh The user s perspective: least amount of hassle tradeoff between anxiety,

More information

Ftp Command Line Manual Windows Passive Mode Linux

Ftp Command Line Manual Windows Passive Mode Linux Ftp Command Line Manual Windows Passive Mode Linux To Connect to Box with Command Line (Advanced users on Mac or Linux) Windows built-in ftp command line utility cannot support passive mode (even. open

More information

Most of the programming assignments in CS 2505 will require using the C language. We will use a (relatively) current version of the GCC C compiler.

Most of the programming assignments in CS 2505 will require using the C language. We will use a (relatively) current version of the GCC C compiler. Programming Tools 1 Most of the programming assignments in CS 2505 will require using the C language. We will use a (relatively) current version of the GCC C compiler. What s GCC? - the Gnu Compiler Collection

More information

curl Manager Manual Contents Intro

curl Manager Manual Contents Intro curl Manager Manual Contents Intro... 1 Creating and Changing Your Password... 2 Add a Known Host... 3 Remove a Known Host... 4 Figure Out What s in an Existing known_hosts File... 4 Export Known Hosts

More information

Docker task in HPC Pack

Docker task in HPC Pack Docker task in HPC Pack We introduced docker task in HPC Pack 2016 Update1. To use this feature, set the environment variable CCP_DOCKER_IMAGE of a task so that it could be run in a docker container on

More information

Introduction to CARC. To provide high performance computing to academic researchers.

Introduction to CARC. To provide high performance computing to academic researchers. Introduction to CARC To provide high performance computing to academic researchers. Machines Metropolis Nano Galles Poblano Pequena Gibbs Ulam Machines (cont.) Machine details, just for example: Nano (supercomputer)

More information

ICS-ACI System Basics

ICS-ACI System Basics ICS-ACI System Basics Adam W. Lavely, Ph.D. Fall 2017 Slides available: goo.gl/ss9itf awl5173 ICS@PSU 1 Contents 1 Overview 2 HPC Overview 3 Getting Started on ACI 4 Moving On awl5173 ICS@PSU 2 Contents

More information

New User Tutorial. OSU High Performance Computing Center

New User Tutorial. OSU High Performance Computing Center New User Tutorial OSU High Performance Computing Center TABLE OF CONTENTS Logging In... 3-5 Windows... 3-4 Linux... 4 Mac... 4-5 Changing Password... 5 Using Linux Commands... 6 File Systems... 7 File

More information

Using Cartesius and Lisa. Zheng Meyer-Zhao - Consultant Clustercomputing

Using Cartesius and Lisa. Zheng Meyer-Zhao - Consultant Clustercomputing Zheng Meyer-Zhao - zheng.meyer-zhao@surfsara.nl Consultant Clustercomputing Outline SURFsara About us What we do Cartesius and Lisa Architectures and Specifications File systems Funding Hands-on Logging

More information

MVAPICH MPI and Open MPI

MVAPICH MPI and Open MPI CHAPTER 6 The following sections appear in this chapter: Introduction, page 6-1 Initial Setup, page 6-2 Configure SSH, page 6-2 Edit Environment Variables, page 6-5 Perform MPI Bandwidth Test, page 6-8

More information

Relay Proxy User Guide

Relay Proxy User Guide Relay Proxy User Guide DBL Technology Company Limited www.dbltek.com sales@dbltek.com support@dbltek.com 2016/5/30 Relay Proxy Relay Proxy is a server software developed by DBL Technology. Its main purposed

More information