Linux environment Graphical interface X-window + window manager Text interface terminal + shell
ctrl-z put running command to background (come back via command fg) Terminal basics Two basic shells - slightly different command syntax csh/tcsh ksh/bash Type commands on command line, send command by pressing Enter Useful key combinations: ctrl-a start of line ctrl-e end of line ctrl-c cancel text typed on line
Basic commands ls list contents of a directory pwd display current directory cd change to directory (cd test) cp copy file (cp from_file to_file) mv move file (mv from_file to_file) rm delete file (rm test1) mkdir make directory (mkdir test) rmdir remove directory (rmdir test) man help for given command (man cp)
Wildcards more files can be specified via wildcards * - matches any number of letters incl. none? - matches any single character [] - encloses set of characters that can match the single given position - used within [] denotes range of characters ~ - followed by user name = home directory (~mcuma) e.g. - *.txt,?igure.jpg, file[0-9].dat
Command flags commands can take flags that modify their behavior flags are formed with (dash) and letter consult man pages of each command for list of available flags e.g. ls -l list files in long format rm -r * remove both files and directories (very dangerous)
Exercise 1 Try to make a directory, e.g. test List contents of a directory, e.g. /uufs/chpc.utah.edu/common/home/mcuma/talks/unix Try to copy some files to this directory, e.g. source files (*.c, *.f) from /uufs/chpc.utah.edu/common/home/mcuma/talks/unix Open man page for some command (e.g. ls) and try some of its flags (e.g. -l, -lt, -ltr)
File view commands cat display contents of file more display contents of file with page breaks (next page with Space key) head display top of file tail display end of file grep search for pattern in file (grep pattern test1) vi edit file (more on this later)
Command output redirection > redirect to a new file (cat test1 > test3) >> - append to a file (cat test2 >> test3) - pipe redirect command output to another command ls -l more some useful websites http://www.ctssn.com/ http://unix.t-a-y-l-o-r.com/unix.html
Exercise 2 View program files via cat, more, head and tail Vary number of lines viewed with head and tail Search for a string in a file with grep Use cat to concatenate two files Use pipe, e.g. paginate result of ls
Some other useful commands wc e.g. wc -l file.txt cut e.g. cut -f 2 -d : file.txt du e.g. du -h df e.g. df -h chmod e.g. chmod a+r file.txt ln e.g. ln -s ~/bin/prog.exe prog1.exe Use man pages to find out what these commands do.
Scripting programming in shell All commands can be combined in a script Other features of script include flow control statements (if, foreach), variables Syntax different for ksh/bash and csh/tcsh Example #!/bin/tcsh set AGE=1 find `pwd` -type f -mtime +$AGE -exec touch "{}"
The vi editor two modes command, input command mode commands input via keyboard keys i, a, r, R enter input mode - insert, append, replace character, replace G go to (1G go to line 1, G go to end of file) x, dd delete character, line : - enter external command (:w write file, :q quit, :q! - quit discarding changes, :wq write and quit) /,? - search forward, backward (/test) input mode works like any other text editor
Use of the vi editor to input text, enter input mode to quit input mode, push Esc key searching, deleting,... done in command mode search and replace: :s/old_text/new_text replace next occurence on current line :s/old_text/new_text/g replace all occurence on current line :%s/old_text/new_text/g replace all occurences in the whole file
Exercise 3 Open a file with vi - vi script.csh Enter edit mode (i) and write text: #/bin/tcsh ls -1 wc -1 Exit edit mode (Esc), and save the file (:w). Oops we made a typo last letter is l, not 1. Use replace to fix it. Save file again and quit (:wq) Change mode of the script to be able to execute it. chmod u+x script.csh Practice some more editing with vi
Programming basic Source code must be compiled in order to run Source most often C or Fortran program (text file) Executable binary form of program that can be executed Compilation translates source into binary form, and combines it (links) with other necessary system libraries
Compilers Compilers are system-specific, but, there are quite a few vendors GNU gcc, g++, g77, gfortran open source, free Intel icc, icpc, ifort commercial but free for academia PGI pgcc, pgcc, pgf77, pgf90 Pathscale pathcc, pathcc, pathf90
Program compilation C program compilation (GNU) gcc cpi_ser.c -o cpi_ser.exe Fortran compilation g77 pi3_ser.f -o pi3_ser.exe Mixed C/Fortran compilation gcc -c ctimer.c g77 pi3_ser.f ctimer.o -o pi3_ser.exe
Compiler optimization and debugging Compiler can perform internal program optimizations that can significantly improve performance common flags -O3 (GNU), -fast (Intel), -fastsse (PGI), -Ofast (Pathscale) In order to debug program in debugger, symbolic information must be included flag -g The easiest debugging is to just add printf or write statements
Exercise 4 Examine performance improvement when using compiler optimization Compile cpi_ser.c without any flags, and run: gcc cpi_ser.c -o cpi_ser.exe./cpi_ser.exe Compile with -O3 optimization, and run again: gcc cpi_ser.c -O3 -o cpi_ser.exe./cpi_ser.exe Calculation of π 1 0 4 1 x 2
Exercise 5 Simple debugging with printf statements Open cpi_ser.c in vi, set up numbering of lines (:set nu) Add below line30: printf( Step is %f\n,h); Save file, recompile and run
More on scripting Scripts are useful in automation of routine tasks multiple/cyclic job submission processing of output files complex run environment Different shells have different syntax We focus on csh/tcsh More advanced text file processing can be done with tools such as perl, python, etc.
Looping in scripts foreach statement loop over set of variables need to specify: loop index variable, loop list variable example set NODES=(da001 da002 da003 da004) foreach NODE ($NODES) rsh $NODE uptime end while statement loop till expression is zero while (expression) commands end break breaks out of the loop continue returns to top of the loop
Exercise 6 Create script to loop over all files in current directory and report number of lines in each file Hints command output into variable is done by enclosing to reverse quotes `command` loop list variable can be supplied on the fly e.g. `ls *.c` line count in the file with wc -l use caption output with echo e.g. echo $filename...
Conditions in scripts if statement if (condition) then else endif switch statement can replace multiple if statements switch (string) case pattern 1: commands breaksw default: commands breaksw endsw
Conditional expressions ==,!=, >, <,&&, - equal, not equal, larger, smaller than, and, or!($?var) true if variable does not exist -d $var true if $var is directory -f $var true if $var is a file -e $var true if $var exists (file, or directory) examples if (($myvar==$var1) ($myvar==$var1)) then
Some other tricks Setting value of a string variable set e.g. set NODES=(da001 da002 da003) Numerical variables denoted by @, e.g. @i = 1, @i = $i + 1 convert string to number expr statement e.g. @i = `expr $number + 0` Specify shell by #!/bin/tcsh Change attributes of the script to be executable chmod u+x my_script.csh
Exercise 7 make a script that prints files larger than 10 kb hints ls l lists files with file lengths can loop over output from ls with foreach cut command can cut out string containing length expr can convert string to number compare value of this number with 10k