Essential Unix and Linux! Perl for Bioinformatics, 140.636! F. Pineda
Generic computer architecture Memory Storage Fig. 1.2 From Designing Embedded Hardware, 2 nd Ed. by John Catsoulis
OS concepts Shell interprets typed commands GUI interprets mouse! and keyboard events Applications Excel, R, Perl scripts Operating system (e.g. Linux, Unix, MacOS The file system Devices e.g. Keyboard, printer screen, ethernet, mouse, etc. Processes e.g. editors, word processors, spreadsheets, scripts, etc.
The Operating System (OS) orchestrates and coordinates computing on the machine n Unix n Linux n OS X, MacOS n Windows n CPM
Unix Phylogeny
Essential unix (operating system) concepts 1. A process Everything that runs under unix is a process, e.g. commands, spreadsheets, terminal windows, etc. Relevant commands: ps, top 2. A file & the hierarchical file system directories(folders), data files, applications. On the disk, everything is a file. Even directories! Files are not processes! Relevant commands: ls, df 3. A shell A shell is a process. The shell accepts command lines and feed them to the unix kernel. Command lines consist of commands or programs to execute along with information needed to execute them
Processes What processes am I running from my shells?! ps What processes is everyone running from shells?! ps -a What processes of all kinds are running?! ps -ax Which processes are using the most cpu?! top
File system concepts
The hierarchical file system / The file system is a tree bin boot dev etc home2 home lib mnt opt proc root sbin tmp usr var vmlinux 140.636F08 faculty public bin local fernando stu08-00 stu08-01 stu08-02 Stu08-03 public_html private_html example.txt /home2/140.636f08/stu08-00/private_html
Navigating the hierarchical file system 1. Navigation commands n ls -- list directory contents n pwd -- prints the absolute path to the working directory n cd -- change directory to a directory specified by a path n df -- free disk space n du -- disk usage statistics 2. Paths n Absolute path n n A fully qualified path must be specified from the root directory / is the symbol for the root directory n n Relative path n Always defined relative to the working directory n. is the symbol that means the working directory n.. is the symbol that means the directory above the working directory Executing an executable that resides in the current working directory n./filename
Review: commands to get around the file system n pwd the command that tells you where you are, i.e. print working directory n ls the command that lists the contents! of the current directory n cd the command that changes the! current directory. In other words! it is how you move around the! file system Memorize these three navigation commands or die!
Files permissions n to see what permissions a file has use ls command with -l option: n ls -l filename filetype user read write execute group read write execute other read write execute d r w x r - x r - x n to change file permissions use the chmod command (see linux quickstart notes)
Basic Bash shell commands
n n n n Commands simple commands n examples: clear, cd, ls, man, wc n Syntax: command [options] parameters! Options n An option is a way of telling Linux to perform a command in a particular way (like pressing alt-key while clicking a mouse) n Typically a minus sign followed by one or more characters! and sometimes an argument! Parameters n Often filenames or arguments for the command! Wildcards n used to represent filenames n? matches any single character n * matches any number of characters
some example commands wc helloworld.pl wc -l helloworld.pl wc -l *.pl ls *.*
On-line help for commands n man command n command -help n apropos keywords
Standard input & output two very important Unix concepts that you absolutely must understand
! Standard I/O n Commands are processes, so they have a default standard input and a default standard output!!! standard input process standard output n For commands executed in shell window: n n standard input is connected to the keyboard standard output is connected to the screen STDIN shell process STDOUT Keyboard device graphics device
When you invoke the Perl interpreter it is a process! STDIN perl process STDOUT Keyboard device graphics device
Redirection and pipes two more very important and related ideas that you absolutely must understand
Standard i/o streams can be Redirected n The standard output of a command can be redirected to a file by the > redirector ls -l > results.txt ls -l STDIN ls process STDOUT Storage device graphics device results.txt file
Standard i/o streams can be Redirected n a file can be redirected to the standard input of a command by the < redirector. wc -l < results.txt wc -l results.txt STDIN wc process STDOUT Keyboard device file graphics device
a pipeline of processes process1 STDOUT STDIN process2 STDOUT STDIN process3 file1.txt file2.txt
a pipeline of processes! (without the intermediate files) process1 STDOUT STDIN process2 STDOUT STDIN process3
Pipes The standard output stream of one command can be piped to the standard input of another command by using the pipe symbol. No intermediate file is required. Example: What does this do? ls -l wc -l ls -l STDOUT STDIN wc -l
Let s use perl to illustrate pipes! 1 st recall that the Perl interpreter is a process STDIN perl process STDOUT Keyboard device graphics device In the shell start the perl interpreter and feed it a text file of perl statements (examine the file of perl statements) perl helloworld.txt In the shell start the perl interpreter and feed it another text file of perl statements (examine the file of perl statements) perl helloperl.txt
What does this command line do? perl hello.txt perl helloperl.txt perl process perl STDOUT STDIN STDIN STDOUT process demo
Execution of unix commands and scripts
How the shell executes commands and applications 1. If the shell does not recognize the command name as one of it s built-in commands, it starts a search for an executable file with the same name. 2. An executable file is either an application binary or a special kind of text file called a script 3. The shell searches through all the directories on the disk specified by the PATH shell variable (demo) 4. And executes the first executable that it finds with the specified name. 5. The file must have execute permission in order to be executed, otherwise you will see a permission error
How the shell executes a script 1. If the executable is a text file, the shell treats it as a script. A script must have a special first line 1. The first line must start with #!, the rest of the first line is the path to an executable 2. The executable is launched 3. The file itself is passed to the standard input (STDIN) of the the executable. 2. Examples helloworld.pl helloperl.pl
Equivalent command lines perl hello.txt perl helloperl.txt./helloworld.pl./helloperl.pl Perl scripts are the standard way of invoking the perl interpreter. You almost never see perl invoked by hand, we did it just to illustrate what is going on under the hood.