General Information. Scripting with Bash and Python. Contents. General Information. Contents (cont.) Contents (cont.) Contents (cont.

Transcription

1 General Information Scripting with Bash and Python Compact Max-Planck Tobias Neckel February 16-26, 2015 Organization About 1.5 hours lecture Slides and live demonstrations About 1.5 hours tutorials We suggest tasks to practise You can work on whatever you want I ll try to answer any questions Breaks :-) Outline 4 days Bash 5 days Python One day off in second week Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, General Information About this course Scripting in Bash and Python (and beyond) Your background? Prior knowledge? Feedback!! More information Slides and tutorials on website: Bash_and_Python_-_2015 Contents Bash (ca. 4 days): Overview, most important basics: Files and directories: ls, dir, cd, mkdir, nd, touch,... Variables, manipulation of variables, arrays, special variables Control structures, functions, parameters Shell conguration Aliases and variables Conguration examples for bash (bashrc) Pipes and wildcards Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Contents (cont.) Bash (ca. 4 days): Advanced topics Regular expressions String-manipulation, regular expressions: grep, cut, sed, head,... Operating on les Grouping, subshells awk Working remote ssh, scp, rsync, unison, nx... Process management Visualization with gnuplot Parsing, preprocessing data Interaction with shell tools Contents (cont.) Scripting with Python... and beyond (ca. 5 days) Overview Interpreted vs. compiled programming languages Python command shell vs. executable Python programs, IPython Where to use it and where not Data types and control structures Objects, type declarations Numeric types, sequences (list, tuple, dict, strings),... Control structures: if, for, break, while,... Arithmetic operations Functions and parameters String handling: strip, trim, split,... File I/O Objects, classes, modules, and packages Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Contents (cont.) Scripting with Python... and beyond (ca. 5 days) Object oriented programming A short introduction to OOP OOP in Python Common and useful libraries os, sys Regular expressions: re Command-line options: optparse math, random GUIs with tkinter Scientic Computing in Python NumPy (vectors, matrices,... ) Matplotlib Calling gnuplot Compact Max-Planck, February 16-26, Requirements Bash Bash as shell (yes, tcsh works, too, but slightly different) Editor of your choice (vim, emacs,... ) Collection of standard tools gnuplot If you are using Windows, we recommend to install cygwin full/extended Python Python 2.7.x, as not all packages have been ported to Python 3.X Packages: Scipy (which should include IPython as interactive Python shell; if not, get IPython) PyLab/Matplotlib Gnuplot Compact Max-Planck, February 16-26,

2 Some Types of Programming Langages Part I Introduction Declarative programming Functional (Lisp, Scheme, make,...) Logical (Prolog) Other (Sql,...) Imperative programming Procedural (C, Fortran, Python, Pascal,...) Modular (Bash,...) Object-oriented (C++, Java, Python,...) Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Scripting vs. Compiled Languages Shell vs. Python Scripting languages Bash, Python, Perl, postscript, matlab/octave,... Interactive mode Few optimisations Easy to use No binary les (hardly useful for commercial software) Compiled languages C, C++, Fortran,... Efcient for computational expensive tasks Source code is compiled to binary code Other Java Shell (Bash) Seperate program for each simple task Gluing together programs with a script Not really a full programming language Powerful tools available Suitable for small tools (1-100 lines of code) Python Full programming language Large number of libraries available Intuitive naming conventions Suitable for almost any task Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Use of Shell Scripts Where to use them: System administration Automating everyday terminal tasks Searching in and manipulating ASCII-Files Where not to use them: Lots of mathematical operations Computational expensive tasks Large programs which need structuring and modularisation Arbitrary le access Data structures Platform-independent programs General Programming Rules Comments Comments Comments Problem algorithm program Modular programming Generic where possible, specic where necessary Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Some Basic Commands The rst interactive example: Part II Bash Basics cd mkdir bash_course cd bash_course touch hello. sh chmod u+x hello. sh <editor > hello. sh File manipulation Files: touch, ls, rm, mv, cp Directories: cd, mkdir, rmdir, pwd Access with chmod: read, write and execute Editors: vi, emacs, gedit, nedit,... Documentation: man, info, apropos, - - help Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26,

3 Hello World! The rst non-interactive example (hello.sh): echo " Hello World!" Convention: shell script sufx.sh #! sha-bang (#: sharp,!: bang) Sha-bang is followed by interpreter (try #!/bin/rm) echo is a shell builtin Hello World! is (as almost everything) a string hello.sh has to be executable (chmod) Variables STR =" Hello World!" echo $STR NO WHITESPACE in assignments STR: name of the variable used for assignment $STR: reference to the value $STR is a short form of ${STR} Quoting STR =" Hello World!" echo $STR echo " $STR " echo '$STR ' Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Operations Arrays a =3+5 # does not work a=` expr 3 + 5` # does work a=` expr 3+5 ` # does not work let "a = 3 + 5" # does work let "a =3+5 " # does work a=$ ((3+5)) # does work (( a ++)) # does work ; result? a=$ [3+5] # does work No direct mathematical operations (everything is a string) command is used to call a program expr: evaluate expressions Only integer operations possible Operators for expr: comparison (<,...) and arithmetics (+, -, \*, /) Operators for let: comp., arith., +=,..., bitwise and logical arr [0]=1 arr [1]= $(( ${ arr [0]}*2)) arr [2]=$[${ arr [1]}*2] arr [5]=32 echo ${ arr [0]} echo ${ arr [1]} echo ${ arr [2]} echo ${ arr [3]} echo ${ arr [4]} echo ${ arr [5]} echo ${# arr [*]} echo ${ arr [*]} Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Special Variables within a script $? Exit status variable $$ process ID $0, $1, $2,... command line parameters $* all command line parameters (single string) $@ all command line parameters (one string per parameter) $# number of command line parameters ${# variable } string length of the variable value shift n shift all command line parameters to the left by n (rst n parameters are lost!) Environment Variables env Control Structures - if if [ 1 = 1 ] echo " Hello World!" Usually (most languages), something is done if test is true Usually, 0 is false, non-zero values are true In bash, test gives a return value Return value 0 means no error test 1 = 2 # equals [ 1 = 2 ] test 1 = 1 Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Control Structures - test Usage: test EXPRESSION or [ EXPRESSION ] Exits with the status determined by the given expression And: EXPRESSION1 -a EXPRESSION2 Or: EXPRESSION1 -o EXPRESSION2 Negation:! EXPRESSION String comparison: =,!= ( test 1 = 1 is string comparison!) Integer comparison: -eq, -ge, -gt, -le, -lt, -ne File comparison: -ef, -nt, -ot File test with single operand: -e, -d,... More details: man test test 1 -eq 1 [ 1 -eq 1 ] [[ 1 -eq 2 1 -eq 1 ]] First Useful Example # encryption program clear which $EDITOR if [ $?!= "0" ] echo " Which editor would you like to use?" read EDITOR $EDITOR plaintext # gedit does not work! detached from console! gpg -a --no - options -o cryption. gpg -c plaintext shred -u plaintext clear cat cryption. gpg shred -u cryption. gpg exit Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26,

4 Control Structures - loops Functions for i in <list > do <commands > done List can be any possible list of elements seq -s <s> <x> produces a list of numbers from 1 to x with separator s (jot - 1 x on MAC) for i in {1..x} does the same (no variable expansion!) break/break n : stop the loop (n levels of nested loops) continue/continue n : continue with the next iteration Other loops: while [ condition ]; do command; done Other loops: until [ condition ]; do command; done function function_name { command } function_name () { command } Both syntactic variants do the same The round brackets are NOT used for parameters Functions have to be dened before they are used Functions may not be empty (use :) Parameter passing to functions equal to programs Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Functions with Parameters Functions with Parameters (2) function min { if [ $1 -lt $2 ] return $1 else return $2 } a=` min 4 6` echo $a # does not work min 4 6 a=$? # works min a=$? # does not work! why? function min { if [ $1 -lt $2 ] echo $1 else } echo $2 a=$( min 4 6) # equals a=` min 4 6` echo $a Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Command Substitution Allows output of a command to replace command itself $( command ) `command ` Both perfom expansion by executing command Replacing command with standard output of command, trailing newlines deleted Not return code! May be nested: escape inner backquotes with backslashes Control Structures - if (2) rand=$[ $RANDOM %2] if [ $# -eq 0 ] echo " Usage : $0 <0 or 1>" else if [[ $1 -ne 0 && $1 -ne 1 ]] echo " parameter has to be 0 or 1" elif [ $1 -eq $rand ] echo " won " else echo " lost " Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Control Structures - case and select echo " Hit a key, hit return." read Key case " $Key " in [[: lower :]] ) echo " Lowercase letter ";; [[: upper :]] ) echo " Uppercase letter ";; [0-9] ) echo " Digit ";; * ) echo " something else ";; esac PS3=' Choose your favorite language : ' select language in " bash " " python " " brainfuck " "C++" do echo " Your favorite language is $language." break done Compact Max-Planck, February 16-26, Partial List of Commands/Variables/... so far man mkdir bla $? $STR continue rmdir clear #! shell builtin $RANDOM break let emacs ls $# $3 touch echo $STR ${#a} $a -le $b chgrp if a = 3 mv a=$((3+5)) expr test $$ ((a++)) cp pwd script sufx cd echo for chmod $* $PS3 esac $@ u+x ${STR:- x } echo $STR info apropos rm vi read - - help a=$[3+5] Compact Max-Planck, February 16-26,

5 Special Directories Part III Shell Cong. current directory.. parent directory ~ own home directory ~user home directory of user ~- previous directory (works also w/o ) cd / usr / local / bin pwd cd ~ pwd cd ~- pwd cd.. pwd Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Aliases and Variables alias - short form for any command alias alias ll=' ls -l ' environment variables Variables are not only within scripts, but also in the shell By setting a variable, it is present in the current script/shell By exporting it (export VARIABLE=value), it is also present at all child processes (NOT at the parent) Many environment variables are already set by default PATH =/ home / neckel / bin /: $PATH ; echo $PATH env local variables set shows all local and global variables and functions unset deletes a variable Compact Max-Planck, February 16-26, Bash cong prole and rc especially aliases are used in every session should not be dened each time You might even run arbitrary code on login or logout e.g. clean up (kill jobs) on logout Prole is used for complete session bashrc is used for a single terminal Files /etc/prole /etc/bash.bashrc $HOME/.bash prole $HOME/.bashrc $HOME/.bash logout Compact Max-Planck, February 16-26, Pipes stdin/stdout: Standard input/output for programs Most Linux programs read from stdin and write to stdout Pipes are used to redirect input and output cmd1 cmd2 connect the output of cmd1 with the input of cmd2 cmd > le redirect the output of cmd to le cmd 2> le redirect stderr of cmd to le cmd > le 2>&1 redirect stdout to le and stderr to stdout cmd &> le redirect all output to le cmd >> le redirect the output of cmd and append it to le cmd < le use the content of le as stdin for cmd cmd <<Endmark Read from stdin until Endmark is inserted in a seperate line ; command separator Wildcards Wildcards are expanded by the shell * zero or more characters? exactly one character [abcd] one of the characters a-d [a-d] same [!a-d] any other character {rst,second} either rst or second Similar pattern exist in other contexts as well (compare regular expressions), but always a bit different... :-( Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Regular Expressions??? Part IV Bash Advanced: Regular Expressions and More The set of regular languages over an alphabet Σ and the corresponding regular expressions are dened recursively as follows: The empty language Ø is a regular language, and the corresponding regular expression is Ø. The empty string { } is a regular language, and the corresponding regular expression is. For each a in Σ, the singleton language { a } is a regular language, and the corresponding regular expression is a. If A and B are regular languages, and r1 and r2 are the corresponding regular expressions, Then A U B (union) is a regular language, and the corresponding regular expression is (r1+r2) AB (concatenation) is a regular language, and the corresponding regular expression is (r1r2) A* (Kleene star) is a regular language, and the corresponding regular expression is (r1*) Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26,

6 Regular Expressions!!! What does all this mean to you, as a user? Absolutely nothing. As a user, you don t care if it s regular, nonregular, unregular, irregular, or incontinent. So long as you know what you can expect from it, you know all you need to care about. Jeffrey Friedl, author of Mastering Regular Expressions Did you ever search for a character or string in a text le?... use tab for auto-completion?... use the * in a terminal for selecting a group of les?... Then you ve already somehow used regular expressions. Regular Expressions When searching for a string, exactly the given character sequence is searched regular expressions get powerful as a tool to nd patterns Additionally to ordinary characters, which stand for themselves special characters are used which are interpreted in a special way. The exact syntax for special characters differs between different implementations Example to identify an address: [^@]\+@.\+\.[^.]\+ regular expressions usually look very cryptic! But imagine you would have to write a normal program to identify an address! Regular expressions can be used to nd/replace groups of strings which can be described by a pattern Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Special Characters for Regular Expressions char A character maching itself * matches zero or more occurences (greedy!) of the previous expression \+ matches one or more occurences (GNU extension) \? matches zero or one occurence (GNU extension) \{i\} matches i occurences \{i,\} matches i or more occurences \{i,j\} matches i to j occurences. matches an arbitrary character ˆ matches the beginning of a string $ matches the end of a string [list] matches a single character from the list ([ˆlist] any character not in the list) \n matches newline \(\) denes a group, reuse via \1 (1st group), \2 (2nd group) \ allows for a logical OR Character Classes [:digit:] 0 to 9 (alternative: [0-9]). [:alnum:] alphanumeric character 0-9 or A-Z or a-z. [:alpha:] character A-Z or a-z. [:xdigit:] Hexadecimal notation 0-9, A-F, a-f. [:punct:] Punctuation symbols, e.g..,?! ; : # $ % & ( ) [:print:] Any printable character. [:space:] whitespace (space, tab,...). [:upper:] uppercase character A-Z (alternative: [A-Z]). [:lower:] lowercase character a-z (alternative: [a-z]). Attention: Additional brackets are necesssary! [[:digit:]] [0 9] Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, programs using regular expressions grep print lines matching a pattern grep -i case insensitive grep -v invert match grep -n additionally print line number tr translate: echo "lower/upper" tr "A-Z""a-z" sed stream editor awk pattern scanning and processing language nd search for les in a directory hierarchy most editors can handle regular expressions Compact Max-Planck, February 16-26, sed - Stream Editor works (as most bash programs) on a stream of data data is processed linewise no way of going back a line ( efcient) apply some action on selected lines (using addresses to select) Addresses address: line number or regular expression zero, one or two (comma-separated) addresses can be used zero addresses: all lines are processed one address: all lines matching the address are processed two addresses: match from rst to second address n match only line number n n~step starting from line n, match every stepth line /pattern/ lines matching the given regular expression i,j all lines from i to j (including both) $ match the last line Compact Max-Planck, February 16-26, sed - Editing Commands parameters -n supress output (per default, all lines are printed) -e editing command follows in command line -f script to be read from le commands p print line = print line number i\text and a\text insert text before/after matched line c\text replace matched line by text s replace pattern, e.g. sed -e 's/search/replace/g' (note: g at end optional, but replaces globally!) (note: use single quotes to avoid problems with white space and variable expansion for $) w write line into given le, sed -n '/patt/ w out.txt' demo.txt Operating on Files less is more than more more displays the content of text les pagewise only downward-scrolling is possible less is an extended and more flexible more outputting les cat outputs the content of a le to stdout (try tac) cat concatenates several les head outputs the rst part of les tail outputs the last part of les for i in `seq 30 `; do echo $i >> temp. txt ; sleep 1; \ done & tail -f temp. txt Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26,

7 Operating on Files (2) le nd out more about le type cmp Compare les byte by byte diff Compare les line by line (graphical: kdiff3) patch apply patch (created by diff) to a le tar pack and compress les (try tar -xzf and tar -czf) sort sort lines of text les (and write to stdout) uniq report or omit repeated lines wc print newline, word, and byte counts for each le cut remove sections from each line of les echo " 1;2;3;4 " cut -b 4-5 echo " 1;2;3;4 " cut -d ";" -f 3 cut -d ' ' -f1,2 / etc / mtab Operating on Files (3) paste merge lines of les join basic database functionality (join tables) le1. txt line1 line2 line3 line4 le2. txt a b c d paste -d.- le?. txt paste le [12]. txt > j1. txt paste le [13]. txt > j2. txt join j?. txt le3. txt Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, nd Tool to retrieve recursively(!) les and directories depending on search criteria Perform operations on the les found In connection with other bash tools perform tasks such as Find all les not older than two days Find all.txt les containing the phrase Hello World! Find all.bak les and remove them General syntax: nd [path...] [expression] An expression can be Options, such as --maxdepth n Tests, such as -amin n (last access less than n minutes ago) Actions, such as -delete If path is not specied, assume./ Compact Max-Planck, February 16-26, nd (2) wget h t t p : / / www5. i n. tum. de / l e h r e / vorlesungen / \ progcourse / bashpython / story. tgz t a r xzf story. tgz Examples nd - maxdepth 2 -name "*. txt " nd./ -amin iname "*. TXT " nd - atime +1 -name "*. txt " nd -path "* folder1 *" nd -size +400c -name " le *.* " -ls nd -name "*. txt " - printf "%h, %f\n" nd -name " le??. txt " -exec grep -H " Bilbo " {} \; nd -regex '.*[^0-9][0-4]. txt ' nd -name "*. txt " - delete # take care!! Parameter -regextype to determine type of regular expression Compact Max-Planck, February 16-26, Executing Commands For executing a single command, it rst has to be found! Either path is provided, or command is in $PATH which, whereis and type: nd out about program type and path For commands with more than one line: echo " This command prints a lot of useless text \ which does not t into one line " multiple commands One way to execute several commands: connecting IO via pipes cmd1; cmd2;... executing several commands cmd1 && cmd2 &&... execute following commands if previous successful cmd1 cmd2... execute following commands if previous fails Compact Max-Planck, February 16-26, awk - by Aho, Weinberger and Kernighan awk is the most powerful bash tool awk <options> '<pattern> {<commands>}' <le> <pattern> or {<commands>} has to be specied (or both) Options awk -f <commandle> <le> applies commands from commandle to the contents of le -v var=value denes a variable to be available within the executed commands -F <sep> set the eld-separator to be available within the executed commands Internal variables NF Number of elds in current line NR current row $n n-th eld of the current line echo -e " 1;2;3\ n4 ;5;6 " awk -F ';' '$1 >3 { print $2+$3}' echo -e " 1;2\ n4 ;5" awk '{ line = $0 } END { print line }' Compact Max-Planck, February 16-26, awk - Patterns /regular expression/ relational expression Operators: <, >, <=, >=,!=, ==, ~,!~ Example: $1 ~ /regex/ (matching a regex) pattern && pattern pattern pattern pattern? pattern : pattern (conditional assignment) (pattern)! pattern pattern1, pattern2 (specifying a range) awk 'length >10 ' input. txt awk '$2 > $1 ' input. txt awk '$1!= prev { print ; prev = $1 }' input. txt awk - Commands Operators ++, -- Increment and decrement +, -, *, /, % Basic arithmetics ^ Exponentiation <, >,... Relational operators, && Logical or/and?: C conditional expression =, +=, *=,... Assignment operations Control statements if, for, while,... awk IS a programming language awk '$2 > $1 { print $3 + $4}' input. txt awk '{ if($1 >$2) print " foo "; else print " bar " }' input. txt Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26,

8 awk - More Commands I/O statements print prints the whole line print expr-list prints the list of expressions print... >> "le" prints the result to a le print... command prints the result to a pipe next jumps to the next line printf C-type print command Numeric functions sin(expr), cos(expr), sqrt(expr), log(expr),... Further functionality string functions (sort, substrings, matching,...) Time functions Bit manipulation User functions Compact Max-Planck, February 16-26, Part V Working Remotely And More Compact Max-Planck, February 16-26, Working remotely - ssh, scp, screen and unison ping ping hostname Useful to nd out whether some machine is down or has no connection. ssh and scp - security by encryption Developed 1995 to allow secure connections between computers ssh username@computername log in onto computername as username scp -r * username@computername:<path_rel_to_home> scp *.txt username@computername:/<absolute_path> scp "username@computername:path/*.*"./ copy all remote les in folder path to current directory Quotes necessary for remote wildcards Compact Max-Planck, February 16-26, Working remotely - ssh, scp, screen and unison Tunneling connections local XX ssh 22 You can ssh from local machine to gate You can (e.g.) ssh from gate to remote You cannot directly access remote Then use ssh for tunneling the connection XY ssh 22 ssh -L 8888: < remote >:22 <gname >@ <gate > cat - # in other terminal scp -P 8888 le. txt <rname : some / dir ssh -p 8888 <rname Compact Max-Planck, February 16-26, local local XX XX 8888 ssh ssh Synchronising data - Unison Working remotely - screen very nice to sync 2 locations GUI and backup support 2-way sync (including merge etc.) supports win and linux (MAC?) needs gtk and ssh download and infos: during a typical session, many terminals are used if working remote, this means opening many ssh connections for each login/logout, everything has to be closed/opened again screen solves all these problems by creating virtual terminals C-a c creates a new terminal C-a " shows a selection-list of available terminals C-a d or screen -d detach the screen C-a D D detach the screen and log out screen -r reattach the screen C-a A give a name to a terminal C-a w show the currently opened terminals C-a k kill a terminal Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Process management schedule jobs at 4:01 pm < job. txt # use atq for list, atrm to delete crontab -e # -l to list # m h dom mon dow command * * * touch / home / neckel / cron. txt priority nice -n <value> <command> starts command with lower priority top shows processes with top computing time within top, r can renice a process nohup run a command immune to hangups ps -ef list all processes pidof nd out the process id for a program name kill <ID> kills process with the given Id (-9 to force) killall <name> kills processes with given name bg and fg send programms to background/foreground Compact Max-Planck, February 16-26, Some more commands you should know basename " this / is/ the / path / to/a. le " dirname " this / is/ the / path / to/a. le " time sleep 3 time for i in `seq `; do a=$ [3* $i ]; done users groups du -sh df -h ln -s <le > <link > Compact Max-Planck, February 16-26,

9 Now for Something Completely Different?! Part VI Visualizing with Gnuplot Why gnuplot? Fast way to visualize scientic data Vast functionality It s free and available on Windows, Linux, and MAC Often bash-tools helpful to preprocess/use data Next steps Shortly introduce basic functionality Visualization of 2d and 3d scientic data Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Interactive Interactive Use $ gnuplot G N U P L O T Version 4.4 p a t c h l e v e l 3 l a s t modied March 2011 System : Linux generic Copyright (C) , 1998, 2004, Thomas Williams, Colin Kelley and many others gnuplot home : h t t p : / / www. gnuplot. i n f o faq, bugs, etc : type help seeking assistance immediate help : type help p l o t window : h i t h Command interpreter Help functionality help for general help help commands for overview help plot for help on command plot quit or <Ctrl+D> to exit Terminal type set to wxt gnuplot> help Gnuplot i s a p o r t a b l e command l i n e d riven graphing u t i l i t y f o r Linux, OS/ 2, MS Windows, OSX, VMS, and many other platforms. [... ] Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, A rst example Plotting sin(x) gnuplot> p l o t s i n ( x ) gnuplot>... leads to Useful Commands Comments Comments start with # Plotting plot <function> for 2d data splot <function> for 3d data plot [xmin:xmax] sin(x) specify range for x-axis plot sin(x), cos(x) multiple plots replot repeat last plot command Escape linebreaks with \ help plot help on further options Functions Variables x, y, z Usual operators; 2 * x ** 2+sin(x)/3.2 4 Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Useful Commands Plotting Some plot modiers plot sin(x) title A nice plot specify title plot sin(x) with <style> specify style Syntax : with <s t y l e > { { l i n e s t y l e l s <l i n e s t y l e >} {{ l i n e t y p e l t <l i n e t y p e >} { l i n e w i d t h lw <l i n e w i d t h >} { l i n e c o l o r l c <colorspec >} { pointtype pt <point type >} { p o i n t s i z e ps <p o i n t s i z e >} { f i l l f s < f i l l s t y l e >} {nohidden3d} { p a l e t t e }} } where <s t y l e > i s e i t h e r l i n e s, p o i n t s, l i n e s p o i n t s, impulses, dots, steps,... use test to see available style of your terminal Compact Max-Planck, February 16-26, Setting Variables and Parameters Use set, e.g. gnuplot> set xrange [ 0 : 3 * p i ] gnuplot> set yrange [ 2:2] gnuplot> set x l a b e l X axis gnuplot> set y l a b e l Y axis gnuplot> p l o t s i n ( x ) Use unset to reset settings, see help unset For xrange it s restored with set xrange restore Compact Max-Planck, February 16-26,

10 Setting Variables and Parameters Save and Load Choose terminal (default: wxt) gnuplot> set t e r m i n a l gnuplot> set t e r m i n a l png t r u e c o l o r f o n t a r i a l 20 \ enhanced Save to le gnuplot> set out [ put ] lename gnuplot> p l o t s i n ( x ) Save and load settings Save settings: save lename. plt Load settings: load lename. plt load works also for plot commands use also for scripts or collection of commands Batch mode Similar for batch mode: $ gnuplot lename.gplt run script lename.gplt Helpful hint: use parameter -persist $ gnuplot -persist lename.gplt Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, More Useful Commands Functions, variables Use variables, calculations, functions gnuplot> a = s q r t (16) * 10+2 gnuplot> p r i n t a 42.0 gnuplot> p r i n t p i gnuplot> f ( x ) = a * s i n ( x+ p i ) gnuplot> p l o t f ( x ) Shell access Leaving Gnuplot to list directory discards settings Therefore shell-ecape: Single commands with! command Spawn shell with shell, return with exit pwd and cd "folder" directly accessible (tab works) Compact Max-Planck, February 16-26, Plot Scientic Data Assume data le gaussian1.dat # x f(x) error Plot with p l o t gaussian1. dat w l i n e s p o i n t s lw 2 ps 2 pt 5 p l o t gaussian1. dat w errorbars lw 2 pt 3 p l o t gaussian1. dat u 1:3 w impulses lw 2 p l o t gaussian1. dat w e r r o r b a r s l t 1 lw 2, \ w l i n e s l t 1 lw 2, \ gaussian2. dat w errorbars l t 2 lw 2 pt 5, \ w l i n e s l t 2 lw 2 Compact Max-Planck, February 16-26, Data within the Gnuplot le Apart from data in les ( lename and ), data can be provided inline ( ) Easy way to pipe data to gnuplot without having to create intermediate les set term png set out t e s t. png p l o t w l i n e s end 3d Data Unordered data sets s p l o t d i s t i l l e d. dat u 1 : 2 : 8 replot doesn t work Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, d Data 3d data on a grid # x y f ( x, y ) If grid has no missing entries, gnuplot creates surface 3d Grid Data There are much more options for plotting We don t cover them all, just few examples s p l o t s i n ( x ) * y+11 w l i n e s set hidden3d r e p l o t set isosample 40 show view set view 60,15 set contour base set pm3d at b unset contour set pm3d s p l o t 3 d f i l e. dat w l i n e s Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26,

11 Modify values Using Bash Commands File one2ten.dat: Use of built-in modiers p l o t one2ten. dat u 1 : ( $2 * $2 ) w l i n e s, \ u 1 : ( s q r t ( $2 ) ) w l i n e s, \ u 1 : ( $1+$2 ) w l i n e s Bash commands such as sed, awk, sort, tail, cat, paste,... can be used as well Allows powerful data processing on the fly Examples: p l o t < cat one2ten. dat eleven2twelve. dat \ u 1 : ( $2 * $2 ) w l i n e s p l o t one2ten. dat u 1 : ( $2 * $2 ) w l i n e s, \ < sed s / 4/ 5/ g one2ten. dat u 1 : ( $2 * $2 ) \ w l i n e s p l o t < awk { p r i n t $1, ( $2<4)? 0 : $2 } one2ten. dat \ w l i n e s p l o t < awk { x=x+$2 ; p r i n t $1, x /NR} one2ten. dat \ w steps Compact Max-Planck, February 16-26, Compact Max-Planck, February 16-26, Finally More information... gnuplot website: Demo scripts for gnuplot: not so Frequently Asked Questions : Compact Max-Planck, February 16-26,