UNIX for Smar0es. compu0ng environments. Aaron J. Mackey Bill Pearson

Size: px

Start display at page:

Download "UNIX for Smar0es. compu0ng environments. Aaron J. Mackey Bill Pearson"

Mae Berry
5 years ago
Views:

1 UNIX for Smar0es Aaron J. Mackey Bill Pearson compu0ng environments UNIX compu0ng: the command line "shell" environment, built- in tools infinitely extensible: download/install tools most bioinforma0cs algorithms/tools are implemented as UNIX command line u0li0es or libraries or, write your own algorithms/tools from scratch highly automatable by scrip0ng (Perl, Python, etc.) interopera0on between tools only limited by your ability to glue together input/output formats almost en0rely free access to tools demo 1

2 UNIX file editors UNIX newlines are "\n" PC is "\r\n"; Mac is \r"; Use a UNIX editor on UNIX files: nano emacs vs. vi/vim When programming, use an IDE eclipse ( Komodo Edit ( edit) do not use: Word, NotePad/WordPad, TextEdit, etc. every editor has pros and cons, try a few filesystem naviga0on cd change directory pwd print working directory (current dir.) ls list files pushd/popd cd, but remember stack find search through filesystem basename/dirname extract filename pieces 2

3 filesystem manipula0on cp copy files mv move files rm remove files rmdir remove directories touch make a new, empty file mkdir make a new, empty directory file inspec0on more read/browse through a file/stdin cat dump file contents to stdout head/tail print first/last N lines od look at the raw data sort sort the lines in the file uniq report unique lines cut extract specific columns grep search for matching lines wc count words/lines/characters 3

4 UNIX permisions chmod change the permissions on a file/dir chown change the ownership of a file/dir chgrp change the group of a file/dir UNIX host status top/ps what processes/apps are running kill force- quit running processes/apps df -h available disk resources du disk space usage 4

5 other UNIX commands builtins list available shell commands which/where find path of commands time measure how long something take echo/tee print/report text wget/curl download files gzip/gunzip/bunzip/zcat compressed files ssh/scp login/copy to/from remote hosts history what have I done previously man get help redirec0on, pipes, replacements > - redirect stdout into file, replace exis0ng >> - redirect stdout into file, appending - redirect/pipe stdout to stdin of next command `backticks`- replace with captured stdout 5

6 globs * wildcard matching {a,b,c} mul0ple choice [a-c], [1-5,9] range/set choice ^ nega0on ls l *.bam ls chr[1-23,x,y].bed environment variables $USER who you are $SHELL what shell you are running $PWD your current working dir $PATH where the shell will go to look for commands $EDITOR your default editor set in your.cshrc/.bashrc, set/setenv 6

7 UNIX editors: learn (at least) one nano simple, easy vi no mouse, use arrow keys how to quit: ctrl- X (all commands at screen bo^om) not so simple to use guaranteed to be on any UNIX machine o_en the default $EDITOR how to quit: [colon]q![enter] emacs also not so simple to use incredibly versa0le, customizable, programmable how to quit: ctrl- X ctrl- C Using emacs" sh>emacs! ^x^c!exit! sh>emacs filename! type some stuff! ^f,^b,^p,^n forward,back,prev, next! ^x^s!save it! ^x^c!exit! sh>! 7

8 Intermediate emacs" sh>emacs random.pl! ^s, ^r search forward, reverse! ^a, ^e start, end of line! esc = M-! M-<, M-> start, end of buffer! M-%!query-replace! ^k!kill-line (and put in kill buffer)! ^k^k!delete line and linefeed (EOL)! ^y!(yank insert kill buffer)! ^x 2, ^x 1, ^x o (multiple windows)! ^u!!(repeat number)! ^h!(help,^h-t tutorial, ^h-a apropos)! (bash) shell scripts files ending with.sh suffix shebang: #!/bin/bash or #!/bin/sh useful to capture (poten0ally long) history of UNIX commands into a reproducible analysis you will always need to repeat your analysis you will never remember all the necessary steps with some modifica0on, your script can be made generic, and reusable for other data 8

9 control flow statements for name in [ ] ; do [ ] ; done do something for each item in a list if [ ] ; then [ ] ; elif [ ]; then [ ]; else [ ] fi specify behavior depending on condi0ons alterna0ve scrip0ng languages Perl once the mainstay of WWW/CGI programming long history == lots of reusable packages PHP mainly limited to dynamic WWW pages Python extremely popular Ruby compact, expressive 9

10 Running Perl" Running a script: % perl myscript.pl Perl one-liners : % perl e 'print "Howdy\n";' Spontaneous Perl: % perl Print "Here we are.\n"; <ctrl-d> Executable scripts: % chmod +x myscript.pl % myscript.pl Literals: strings and numbers" % perl e 'print 2 + 2; print "\n";' % perl e 'print "abc"; print "def\n";' # string addition (concatenation operator) % perl e 'print "one two". " and three\n";' # mixing numbers and strings: % perl e 'print (2 * 2). \n ; % perl e 'print "2 + 2 = ". (2 + 2). "\n";' # decimals and concatenations: % perl e 'print "\n";' % perl e 'print "\n";' 10

11 Perl vs. bash scripts ".pl" file extension, e.g. "myscript.pl" begins with a "shebang" #!/usr/bin/perl.pl scripts need chmod +x to be executable: invoked with perl: perl myscript.pl or directly:./myscript.pl Perl variables scalar: begins with s: $ $name = "Aaron"; $day = "Friday"; $url = " array: begins = ("Mon", "Tue", "Wed","etc."); hash: begins with h: % %lookup = ( "key1" => "value1", "key2" => "value2" ); 11

12 use strict; always begin every Perl script with: use strict; use warnings; #for longer help: use diagnostics; now every variable must be declared with my my $name = "Aaron"; = ("Mon", "Tue", "etc."); strict ensures that the variable names you use are what you meant, not typos or accidental syntax errors warnings provides brief messages about inappropriate usage/syntax your first Perl script #!/usr/bin/perl use strict; use warnings; my $name = "Bob"; print "Hello world.\n"; print "My name is $name.\n"; 12

13 where to get help: perldoc perldoc perl provides list of all the "chapters" available perldoc perltoc gives more detail/searchable perldoc perlintro shows the intro to Perl chapter: read this! perldoc perlrequick introduc0on to regular expressions perldoc perlopentut how to open/read/write files/stdin/stdout 13

Bioinformatics and Functional Genomics Unix at the command line biol4230 Friday, Jan 19, 2018

Bioinformatics and Functional Genomics Unix at the command line biol4230 Friday, Jan 19, 2018 Goals of today's lecture: introduction to the unix command line unix file manipulation ls, cp, mv, mkdir, cd,