Research. We make it happen. Unix Basics Presented by: Patton Fast User Support Group help-line: help@msi.umn.edu 612-626-0802 personal: pfast@msi.umn.edu 612-625-6573
Outline I. Warnings! II. III. IV. Basic Concepts Basic Commands The Fun Stuff V. Resources Please Ask Questions!
Warnings! If you delete a file, it is GONE! The file can only be retrieved from a backup. If you overwrite a file it has been changed forever. The original file can only be restored from a backup. Home directories are backed up nightly. The scratch spaces are NOT backed up! Bottom line? Be Careful With Your Files!
Warnings! scratch space Scratch Space Time Limits! Regatta (/regscratch1 and /regscratch2) 7 days, 50 Gb SP (/scratch1 and /scratch2) 7 days, 50 Gb Altix (/scratch1 and /scratch2) 14 days Netfinity (/scratch1 and /scratch2) 14 days SDVL (/scratch) 14 days BSCL (/wrk and /wrk2) 14 days Bottom line? Be Careful With Your Files!
Basic Concepts resources IBM pseries 312 Power4 processors 664 Gb of memory 7 Tb of storage SGI Altix 176 Itanium2 processors 352 Gb of memory 5 Tb of storage http://www.msi.umn.edu/pseries http://www.msi.umn.edu/altix IBM SP 370 Power3 processors 663 Gb of memory 4 Tb of storage Netfinity Linux Cluster 160 Xeon processors 224 Gb of memory 2.0 Tb of storage http://www.msi.umn.edu/sp http://www.msi.umn.edu/netfinity www.msi.umn.edu/user_support
Basic Concepts resources Basic Sciences Computing Lab 48 CPU SGI Altix 16 CPU SGI Onyx4 9 SGI Octanes 8 CPU Sun UltraSparcIII 1 SunBlade 2000 2.5 Tb of storage http://www.msi.umn.edu/bscl Med. Chem./Supercomputing Inst. Vis. Workstation Lab 3 SGI Octanes 3 SGI O2 s 2 SunBlade 2000s http://www.msi.umn.edu/vwl Scientific Dev. & Vis. Lab 9 SunBlade 2000s 7 SGI Octanes 1 SGI Onyx http://www.msi.umn.edu/sdvl Computational Genetics Lab 16 CPU SunFire 6800 4 Dell PCs 1 Mac G5 1 SGI Fuel 1 SGI Tezro 1 IBM IntelliStation http://www.msi.umn.edu/cgl www.msi.umn.edu/user_support
Basic Concepts console Console Login: Enter username Enter password Consoles normally have a graphical-windows type interface, and always have priority over remote logins. Console Logout: Logout procedures vary between consoles. Make sure you logout! Otherwise your files and account are open to the next person. You could be logged out by an administrator.
Basic Concepts changing passwords After your first login you should change your initial password to something that is easy for you to remember, but not easy for someone to guess. %> yppasswd (Institute machines) %> passwd (general) You should NEVER give your password to anyone! If you forget your password, send a message to help@msi.umn.edu and your password will be reset. Note: Never share accounts or passwords!
Basic Concepts remote login ssh [options] machine secure shell; a program for logging into a remote machine providing encrypted communications between machines options: -l login name -X sets environment variables for porting X-display example: type: ssh -l pfast sp type: ssh bi2 SSH clients: http://www.msi.umn.edu/user_support/ssh open a secure connection for the user pfast on the machine sp.msi.umn.edu open a secure connection to bi2.msi.umn.edu with current username
Basic Concepts forwarding email Why? How? So that you get all of your email sent to one account. Makes it easier for sys admins to communicate with you. 1) Create the file.forward in your home directory 2) On the first line: your.email@desired.location 3) Create this same file in all of your accounts that you want to forward.
Basic Concepts command line help man [options] command displays manual pages about command -k keyword search for a command info command displays Info document of command When in doubt, check the man/info pages! Need extra details? Check the man/info pages!
Basic Concepts file structure overview / bin@ home/ lib@ scratch/ usr/ user1/ user2/ bin/ lib/ local/ file1.cshrc mail/ source/ work/ more* proj1/ proj2/ user1/ user2/ input output dir1/ dir2/ directory ( / ) executable ( * ) link ( @ ) file1/ file2/
Basic Concepts metacharacters? single character wild card * wild card, arbitrary number of characters ~ home directory of current user ~name home directory of user name. current directory.. parent directory (back one)
Basic Concepts command anatomy command [options] argument example: [options] (usually preceded by a dash) ls -al /scratch command name argument (quite often a filename or directory) Note: UNIX commands, options, and arguments are all CasE sensitive!
Basic Commands Command ls [options] dirname pwd mkdir dirname cd dirname mv [options] source destination cp [options] source destination rm [options] source ssh [options] hostname scp [options] user@host1:file user@host2:file Description list the contents of dirname display full pathname create the directory dirname change to the directory dirname move a file or directory copy a file or directory remove a file or directory remote login remote copy; file transfer skip basic commands
Listing Contents ls [options] dirname list the contents of dirname options: -a list all files including hidden files [hidden files are preceded by a. ; eg.cshrc] -l long listing showing ownership, permissions, and links examples: type: ls /home/user/temp view the contents of a directory with absolute pathname /home/user/temp type: ls../../temp list the contents of a directory using a relative path.
Make Directory mkdir dirname create the directory dirname examples: type: mkdir work create the directory work/ in the current working directory type: mkdir work/proj1 create the directory proj1/ in the work/ directory type: mkdir /wrk/user2 create the directory user2/ in the /wrk/ directory
Change Directory cd cd dirname change to your home directory change to the directory dirname examples: type: cd ~tom change to tom s home directory type: cd /wrk/user2 change to the directory /wrk/user2
Moving or Renaming mv [options] presname newname mv [options] srcfile destdir rename a file move a file to another directory options: example: -i confirm overwrites type: mv ~user1/file./outputfile moves file from user1 s home directory to the current working directory and renames it output file Note: Be careful when overwriting files!
Copying cp [options] srcfile destfile cp [options] srcfile destdir copy a file to another filename copy a file to another directory options: example: -i confirm overwrites -R recursively copy a directory and its contents, copies symbolic links type: cp -R proj1 proj2 copy the directory proj1/ and name it proj2/ Note: Be careful when overwriting files!
Copying cp [options] srcfile destfile cp [options] srcfile destdir Tip: copy a file to another filename copy a file to another directory options: If you have large files to -i move from confirm the scratch overwrites -R directories,copy recursively (cp) copy them a directory and its rather than contents, moving copies (mv) symbolic links them. example: type: cp -R proj1 proj2 copy the directory proj1/ and name it proj2/ Note: Be careful when overwriting files!
Removing rm [options] filename remove a file options: example: -f remove without prompting -i confirm removal -r recursively remove a directory and its contents type: rm -rf /scratch/user2/dir1 remove the directory /scratch2/user2/dir1 and its contents Note: Be careful when removing files! Especially with wild cards (*,?)!
Remote Copy scp [options] host:file1 host:file2 secure copy; copy files between hosts on a network using the ssh protocol options: -r recursive (used for directories) example: type: scp pfast@wind: output/*.out. type: scp -r mail origin:mail_copy copy the output files from host wind with username pfast to current directory copy the mail directory to the origin and rename the directory mail_copy Note: You can use sftp if scp is not available.
Remote Copy scp [options] host:file1 host:file2 Tip: secure copy; copy files between hosts on a network using the ssh protocol options: -r recursive (used for directories) If you have large files to example: transfer, use sftp instead of scp. type: scp pfast@wind: output/*.out. type: scp -r mail origin:mail_copy copy the output files from host wind with username pfast to current directory copy the mail directory to the origin and rename the directory mail_copy Note: You can use sftp if scp is not available.
Changing Permissions chmod [options] who ops permission filename change file permissions use ls -l to view permissions options R recursively change permissions who can be any combination of: u (user) g (group) o (other/world) a (all or ugo) ops adds or takes away permission, and can be: + (add permission) (remove permission) permission can be any combination of: r (read) w (write) x (execute) X (check user first)
Command Repetition history!string up/down arrows left/right arrows Control-E Control-A display list of most recent commands repeat command beginning with string scroll up and down through most recent commands; csh and tcsh use for editing current line end of line beginning of line examples: type:!ls repeat last ls command
Command Completion TAB or Control-D finish the current command, filename, or directory if possible, or show the possible completers (works with c and tc shells)
Tarring tar [options] files collect or extract files or directories into or from an archive options: c create a new archive x extract files from archive f store files in archive v print function letter and name of files store to standard output example: type: tar xvf file.tar extract the files and or directories from the archive, file.tar
Compression compress file uncompress file.z gzip file gunzip file.gz compress file and rename it file.z (the program automatically renames the file) uncompress file.z and rename it file compress file and rename it file.gz (the program automatically renames the file) uncompress file.gz and rename it file
Process Control Control-C Control-Z jobs bg fg & example: cancel a foreground job stop (interrupt) a foreground job list of background jobs type: blastp swissprot test.seq > test.out & run stopped job in the background run stopped job in the foreground appended to the end of a command will place that job in the background run WU-BLAST job in the background
Process Control ps [options] display the status of the current processes and the process id-number options: -a all processes -u display processes owned by a particular user jobs shows any jobs that are currently running in the background or suspended
Process Control kill 9 id-number terminate a process owned by you; idnumber (process id) can be found with the ps command or kill %job-number terminate a process owned by you; jobnumber can be found with the jobs command
Other Commands du -sk file df stat file quota -v display total disk usage of file or directory in kilobytes display the current disk usage for all connected file systems view attributes of a file or directory list your disk quota
Viewers cat [options] files dumps the contents of files to the screen more [options] files less file a pager, that lets you view one page at a time similar to more, but better
Editors Graphical: nedit filename xemacs filename if not available, ask to have it installed (jot no longer works on SGI) Command-line: pico filename vi filename full-screen text editor available on most UNIX systems full-screen text editor available on virtually every UNIX system
The Fun Stuff redirect > command output redirection (create new, overwrites existing file) >> command output redirection (append) < command input redirection (from file) examples: how to: type: how to: type: combine individual sequence files into one file cat seq1 seq2 > seq email a file to a collaborator mail -s results collab@umn.edu < new_blast_results
The Fun Stuff piping command1 command2 directs standard output of one command into standard input for the next command examples: type: ls -al more type: qstat -a grep user1 look at the ls output one page at a time check the queue on the origin for any processes being run by user1
The Fun Stuff pattern searching grep [options] string file options: find and print out lines containing the string in a file or multiple files examples: -i ignore case how to: extract sequence information from protein structure files type: grep SEQRES pdbfile cut -c20-70 > seqs
The Fun Stuff.cshrc file.cshrc Some things to add to your.cshrc file: alias alias-string com-string examples: shell startup file abbreviate a command string with an alias string type: alias la ls -a create a shortcut for ls -a type: alias q qstat -u pfast module load(unload) software source /usr/local/gcg/gcgstartup check if you have any jobs running in the queue (origin) load or unload the environment settings for a particular software runs the gcgstartup script to set the necessary env. variables
Resources UNIX in a Nutshell by O Reilly Developing Bioinformatics Computer Skills by O Reilly http://www.iats.missouri.edu/servlets/knowledgebase/article/422 http://www.geek-girl.com/unix.html http://riceinfo.rice.edu/computer/documents/unix/unix1.01.pdf